10 Python quirks you should know about in your code
Python is one of the most popular, beginner-friendly languages to learn. It’s super simple to read by being very direct in its syntax. As long as you know the basics, there really is no question as to what the language is doing at any given time.
However, just like any other language you might study, Python does have some quirks about it. This article will introduce you to some of the idiosyncrasies of Python by telling you what’s going on under the hood.
Note: for this article, we will only be referring to quirks that are relevant to Python 3 since Python 2 was deprecated in January 2020.
We’ll take a look at:
- Variables, Namespace, and Scope
- Deleting a list item while iterating
- Modifying the dictionary while iterating over it
- Name resolution ignoring class scope
- Beware of default mutable arguments
- Same operands, different story
- What’s wrong with Booleans?
- Class attributes and instance attributes
split()method- Wild imports
- What to learn next
Improve your code with the top Python tricks
This course sheds light on some of the interesting parts of Python you might not know. Learn the tricks that every Python programmer should know.
Variables, Namespace, and Scope#
There are two things we need to talk about when it comes to looking at Python under the hood: namespace and scope.
Namespace#
In Python, because it is an object-oriented programming language, everything is considered an object. A namespace is just a container for mapping an object’s variable name to that object.
function_namespace = { name_of_obj_a: obj_1, name_of_obj_b: obj_2 }for_loop_namespace = { name_of_obj_a: obj_3, name_of_obj_b: obj_4 }
We can think of namespaces as just Python dictionaries, where the variable name for the object is the key, and the value is the object itself. We create a new, independent namespace every time we define a loop, a function, or a class. Each namespace has its own hierarchy called scope.
Scope#
Scope, at a very high level, is the hierarchy at which the Python interpreter can “see” a defined object. The interpreter starts with the smallest scope, local, and looks outward if it can’t find the declared variable to the enclosed scope. If the interpreter can’t find it in the enclosed scope, it looks to the global scope.
Take this example:
i = 1def foo():i = 5print(i, 'in foo()')print("local foo() namespace", locals())return iprint("global namespace", globals())foo()
Here we have a global namespace and we have a foo() namespace. You can take a look at the individual namespaces by printing globals() and printing locals() at the given spots in the code.
The local namespace is pretty straightforward. You can clearly see i and its value. The global namespace is a little different in that it also includes some extraneous stuff from Python.
Here, it shows the foo function as a place in memory rather than the actual function value itself as well as the value for i in the global namespace.
That being said, you can alter a variable in the global namespace. Just use the global keyword in front of the variable name prior to your logic:
i = 1def foo():global ii = 5print(i, 'in foo()')print("local namespace", locals())return iprint("global i before func invocation", globals()["i"])foo()print("global i after func invocation", globals()["i"])
Deleting a list item while iterating#
When working with lists in Python, we need to take a look at what happens when we remove items from a list when we loop over it. In general, it’s not a good idea to iterate and remove items from a list due to unintended consequences. Take these examples:
del keyword#
The del keyword only deletes the instance of that item in the local namespace, but not the actual item itself in the global namespace. So the globally defined list_1 is unaffected.
list_1 = ["apples", "oranges", "bananas", "strawberries"]for item in list_1:del itemprint("list_1: ",list_1); # ['apples', 'oranges', 'bananas', 'strawberries']
remove() method#
In the remove() method, once Python removes an item from the list, all of the other items will shift to the left once, but the iteration doesn’t happen until after everything has been moved.
list_2 = ["apples", "oranges", "bananas", "strawberries"]for item in list_2:list_2.remove(item)print("list_2: ",list_2)# ['oranges', 'strawberries']
Here is a step-by-step rundown of how it happens:
- First iteration: remove
apples.orangesmoves to left and is now the current index.bananasmoves to left and becomes the next index.strawberriesmovies to left, and loop goes to the next index. - Second iteration:
bananasis at current index, so method removesbananas.strawberriesmoves to left and is now the current index. No more index values, so iteration is done. - Result: This leaves
orangesandstrawberriesin the list.
pop(idx) method#
For the same reason that we don’t use the remove method when looping over a list, we don’t use the pop(idx) method. When an index is not passed in as an argument, Python removes the last index in the list.
list_3 = ["apples", "oranges", "bananas", "strawberries"]for item in list_3:list_3.pop()print("list_3: ",list_3) # ['apples', 'oranges']
- First iteration: Remove
strawberries, so the list’s length is now three. Move to the next iteration. - Second iteration: Remove
bananas, so the list’s length is now two. No more index values, and iteration is done. - Result: This leaves
applesandorangesin the list.
Note: If an index is passed into the
pop()method and it doesn’t exist, it will raise anIndexError.
So, what does work?#
The secret to iterating and manipulating a list in Python is by slicing, or making a copy of the list. It’s as simple as using [:]:.
list_4 = ["apples", "oranges", "bananas", "strawberries"]for item in list_4[:]:list_4.remove() #pop() would also work here.print("list_4: ",list_4) # []
list_4[:] This operator makes a copy of the list in memory. The original list is unaffected as we loop through it, but does affect the original when all done.
Enjoying the article? Scroll down to sign up for our free, bi-monthly newsletter.
Modifying the dictionary while iterating over it#
Python dictionaries can be tricky objects to work with. One thing that is absolutely certain, though, is that these dictionaries cannot necessarily be modified at all when they are being looped over.
Depending on the Python version you have, you will either get a Runtime Error, or the loop will run a certain number of times (between 4 and 8) until the dictionary needs to be resized.
You can make a workaround by using list comprehensions, but it’s generally not in best practice.
for i in x:del x[i]x[i+1] = i + 1print(i)print(x)
Keep the learning going.#
Learn the interesting parts of Python without scrubbing through videos or documentation. Educative’s text-based courses are easy to skim and feature live coding environments, making learning quick and efficient.
Name resolution ignoring class scope#
According to the creator of Python, Guido van Rossum, Python 2 had some “dirty little secrets” that allowed for certain leaks to happen. One of these leaks allowed for the loop control variable to change the value of a in the list comprehension.
That’s been fixed in Python 3 by giving list comprehensions their own enclosing scope. When the list comprehension doesn’t find a definition for a in the enclosing scope, it looks to the global scope to find a value. This is why Python 3 ignores a = 17 in the class scope.
a = 5class Example:# global aa = 17b = [a for i in range(20)]print(Example.y[0])
Beware of default mutable arguments#
Default arguments in Python are fallback values that are set up as parameters if the function is invoked without arguments. They can be useful, but if you call the function several times in a row, there can be some unintended consequences.
def num_list(nums=[]):num = 1nums.append(num)return numsprint(num_list())print(num_list())print(num_list([]))print(num_list())print(num_list([4]))
The first two times num_list() is invoked, a 1 will be appended to nums list both times. The result is [1, 1]. To reset the list, you have to pass in an empty list to the next invocation.
Trick! To prevent bugs where you use default arguments, use
Noneas the initial default.
Same operands, different story#
Reassignments in Python can be tricky if you are not sure of how they work. The = and the += operators carry two different meanings in Python when used in conjunction with lists.
# reassignmenta = [1, 2, 3, 4]b = aa = a + [5, 6, 7, 8]print(a)print(b)# extendsa = [1, 2, 3, 4]b = aa += [5, 6, 7, 8]print(a)print(b)
When manipulating lists, the = operator just means reassignment. When b is assigned as a, it created a copy of a as it was at the time. When a is reassigned to a + [5, 6, 7, 8], it concatenated the original a with [5, 6, 7, 8] to create [1, 2, 3, 4, 5, 6, 7, 8]. The b list remains unchanged from its original assignment.
With the += operator, when it pertains to lists, is a shortcut for the extends() method. This results in the list changing in place, giving us [1, 2, 3, 4, 5, 6, 7, 8] for both a and b.
What’s wrong with Booleans?#
When it comes to Boolean values, it seems pretty straightforward. In this mixed array, how many Boolean values do we have and how many integer values do we have?
mixed_type_list = [False, 4.55, "educative.io", 3, True, [], False, dict()]integers_count = 0booleans_count = 0for item in mixed_type_list:if isinstance(item, int):integers_count += 1elif isinstance(item, bool):booleans_count += 1print(integers_count)print(booleans_count)
Why is the output 4-0? In short, a Boolean value in Python is a subclass of integers. True in Python equates to 1, and False equates to 0.
Class attributes and instance attributes#
In object oriented Python, a class is a template, and an instance is a new object based on that template. What would happen if we were to try to change or mix up the assignments to class variables and instance variables?
class Animal:x = "tiger"class Vertebrate(Animal):passclass Cat(Animal):passprint(Animal.x, Vertebrate.x, Cat.x)Vertebrate.x = "monkey"print(Animal.x, Vertebrate.x, Cat.x)Animal.x = "lion"print(Animal.x, Vertebrate.x, Cat.x)a = Animal()print(a.x, Animal.x)a.x += "ess"print(a.x, Animal.x)
Here we have three classes: Animal, Vertebrate, and Cat. When we assign a variable in the Animal class, and the other classes are extensions of the Animal class, those other classes have access to the variable created in the Animal class.
Be certain of your reassignment when working with classes and instances. If you want to alter the template, use the class name, and when you want to alter the instance, use the variable you assigned to the new instance of the class name.
split() method#
The split() method has some unique properties in Python. Take a look at this example:
print(' foo '.split(" ")) # ['', '', '', '', '', '', '', '', '', 'foo', '']print(' foo bar '.split()) # ['foo', 'bar']print(''.split(' ')) #['']
When we give the split method a separator, in this case (" "), and use it on a string of any length, it’ll split on the whitespace. No matter how many whitespace characters you have in a row, it’ll split on each one.
If there is no separator indicated, the Python interpreter will compress all of the repeating whitespace characters into one, and split on that character, leaving only the groups of non-whitespace characters separated.
An empty string split on a whitespace character will return a list with an empty string as its first index.
Wild imports#
Wildcard imports can be useful when you know how to use them. They have some idiosyncrasies that can make them more often confusing than not. Take this example:
def hello_world(str):return str;def _hello_world(str):return str
from helpers import *hello_world("hello world -- WORKS!")_hello_world("_hello_world -- WORKS!")
If we were to try to run this in the directory these files were in, the first invocation of the hello_world function would work fine. The second, not so much. When using wildcard imports, the functions that start with an underscore do not get imported.
For those methods, you will either have to directly import the function, or use the __all__ list to use your wildcard import.
from helpers import *hello_world("hello world -- WORKS!")_private_hello_world("__all__ -- WORKS!")
Note: The
__all__variable is surrounded by two underscores on either side.
What to learn next#
Congrats! You’ve now learned about ten common quirks in Python that can improve your code. It’s important to understand what’s going on under the hood of Python to get the most of the language. But there is till more to learn to truly master Python.
Next you should learn about:
deloperation- Tricks with strings
- Subclass relationships
- Bloating instance dicts
- Non-reflexive class method
- and more
To get started with these quirks and more, check out Educative’s course Python FTW: Under the Hood Instructor Satwik Kansal shares more about how Python works and the reasons for certain errors or responses in the interpreter. You can think of the course as a “Python hacks” handbook. Mind bending and fun!
Happy learning!
Continue reading about Python#
Free Resources
