8.1 Misunderstanding Python's Scope Rules
Python's scope is based on what they call the LEGB rule, which stands for:
-
Local(names assigned in any way within a function (deforlambda), and not declared global in that function); Enclosing(names in the local scope of any statically enclosing functions (deforlambda), from inner to outer);Global(names assigned at the top-level of a module file, or by executing aglobalstatement in adefwithin the file);-
Built-in(names preassigned in the built-in names module:open,range,SyntaxError, etc.).
Seems pretty straightforward, right?
But there's some nuance in how it works in Python, which leads us to the tricky issue of programming in Python. Consider the following example:
x = 10
def foo():
x += 1
print(x)
foo()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in foo
UnboundLocalError: local variable 'x' referenced before assignmentWhat's the problem?
The error above occurs because when you assign a value to a variable in a scope, Python automatically considers it local to that scope and it hides any variable of the same name in any enclosing scopes.
This is why many are surprised when they get a UnboundLocalError in previously working code when it is modified by adding an assignment statement somewhere in the body of a function.
This feature is particularly confusing when using lists. Consider the following example:
lst = [1, 2, 3]
def foo1():
lst.append(5) # This works just fine...
foo1()
print(lst)
[1, 2, 3, 5]
lst = [1, 2, 3]
def foo2():
lst += [5] # ... but this crashes!
foo2()
Traceback (most recent call last):
File "
", line 1, in
File "
", line 2, in foo UnboundLocalError: local variable 'lst' referenced before assignment
Why does foo2 crash while foo1 works?
The answer is the same as in the previous example, but it is perhaps more subtle here. foo1 does not apply an assignment operator to lst, whereas foo2 does. Remembering that lst += [5] is really just shorthand for lst = lst + [5], we see that we are trying to assign to lst (and so Python assumes it is in the local scope). However, the value we are trying to assign to lst depends on the value of lst itself (which, again, is assumed to be in the local scope), which has not been defined yet. And we get an error.
8.2 Modifying a List While Iterating Over It
The problem in the following piece of code should be fairly obvious:
odd = lambda x: bool(x % 2)
numbers = [n for n in range(10)]
for i in range(len(numbers)):
if odd(numbers[i]):
del numbers[i] # BAD: Deleting item from a list while iterating over it
Traceback (most recent call last):
File "
", line 2, in
IndexError: list index out of range
Deleting an item from a list or array while iterating over it is a Python pitfall well-known to any seasoned software developer. But while the above example might be obvious enough, even experienced developers might occasionally stumble over this in much more complicated code.
Fortunately, Python includes a number of elegant programming paradigms that, when properly used, can result in significant simplification and optimization of code. An additional nice side effect of this is that in simpler code, the chance of falling into a pitfall like accidentally deleting an item from a list while iterating over it is drastically reduced.
One such paradigm is list comprehensions. Moreover, understanding how list comprehensions work is particularly useful for avoiding this particular problem, as demonstrated in the following alternative implementation of the code above, which works perfectly:
odd = lambda x: bool(x % 2)
numbers = [n for n in range(10)]
numbers[:] = [n for n in numbers if not odd(n)] # just filtering new items
print(numbers)
# [0, 2, 4, 6, 8]Important! This doesn't assign a new list object. Using numbers[:] is a group assignment to all elements of the list.
8.3 Misunderstanding Python's Variable Binding in Closures
Consider the following example:
def create_multipliers():
return [lambda x: i * x for i in range(5)] # Returns a list of functions!
for multiplier in create_multipliers():
print(multiplier(2))You might expect the following output:
0
2
4
6
8
But what you actually get is this:
8
8
8
8
8
Surprise!
This happens because of late binding in Python, which means that the values of variables used in closures are looked up at the time the inner function is called.
Therefore, in the code above, whenever any of the returned functions are called, the value of i is looked up in the surrounding scope at call time (by which time the loop has already completed, so i has been assigned its final value of 4).
The fix for this common Python problem would be:
def create_multipliers():
return [lambda x, i = i : i * x for i in range(5)]
for multiplier in create_multipliers():
print(multiplier(2))
# 0 # 2 # 4 # 6 # 8Voilà! We use default arguments here to generate anonymous functions to achieve the desired behavior. Some might call this solution elegant. Some might call it subtle. Some might hate such shenanigans. But if you're a Python developer, it's important to understand this either way.
GO TO FULL VERSION