Modules and Packages

Why do we need a module?

Now we know how to create classes and instantiate objects. We don’t need to write too many classes (or non-object-oriented code, for that matter) before we start to lose track of them. For small programs, we generally put all our classes into one file and add a little script at the end of the file to start them interacting. However, as our projects grow, it can become difficult to find the one class that needs to be edited among the many classes we’ve defined. This is where modules come in. Modules are Python files, nothing more. The single file in our small program is a module. Two Python files are two modules. If we have two files in the same folder, we can load a class from one module for use in the other module.

Module name

The Python module name is the file’s stem; the name without the .py suffix. A file named model.py is a module named model. Module files are found by searching a path that includes the local directory and the installed packages.

Import module

The import statement is used for importing modules or specific classes or functions from modules. We’ve already seen an example of this in our Point class in the previous section. We used the import statement to get Python’s built-in math module and use its hypot() function in the distance calculation. Let’s start with a fresh example.

Real-world example

If we are building an e-commerce system, we will likely be storing a lot of data in a database. We can put all the classes and functions related to database access into a separate file (we’ll call it something sensible: database.py). Then, our other modules (for example, customer models, product information, and inventory) can import classes from the database module in order to access the database.

Let’s start with a module called database. It’s a file, database.py, containing a class called Database. A second module called products is responsible for product-related queries. The classes in the products module need to instantiate the Database class from the database module so that they can execute queries on the product table in the database. There are several variations on the import statement syntax that can be used to access the Database class. One variant is to import the module as a whole:

import database
db = database.Database("path/to/data")

This version imports the database module, creating a database namespace. Any class or function in the database module can be accessed using the database.<something> notation. Alternatively, we can import just the one class we need using the from...import syntax:

from database import Database 
db = Database("path/to/data")

This version imported only the Database class from the database module. When we have a few items from a few modules, this can be a helpful simplification to avoid using longer, fully qualified names like database.Database. When we import a number of items from a number of different modules, this can be a potential source of confusion when we omit the qualifiers.

If for some reason, products already have a class called Database, and we don’t want the two names to be confused, we can rename the class when used inside the products module:

from database import Database as DB 
db = DB("path/to/data")

We can also import multiple items in one statement. If our database module also contains a Query class, we can import both classes using the following code:

from database import Database, Query

We can import all classes and functions from the database module using the following syntax:

from database import *

Tip: Most experienced Python programmers will ask to never use this syntax (a few will tell us there are some very specific situations where it is useful, but we can disagree). One way to learn why to avoid this syntax is to use it and try to understand our code two years later. We can save some time and two years of poorly written code with a quick explanation now!

Avert from module import * statement

We’ve got several reasons for avoiding this:

• When we explicitly import the Database class at the top of our file using from database import Database, we can easily see where the Database class comes from. We might use db = Database() 400 lines later in the file, and we can quickly look at the imports to see where that Database class came from. Then, if we need clarification as to how to use the Database class, we can visit the original file (or import the module in the interactive interpreter and use the help ( database.Database ) command). However, if we use the from database import * syntax, it takes a lot longer to find where that class is located. Code maintenance becomes a nightmare.

• If there are conflicting names, we’re doomed. Let’s say we have two modules, both of which provide a class named Database. Using from module_1 import * and from module_2 import * means the second import statement overwrites the Database name created by the first import. If we used import module_1 and import module_2, we’d use the module names as qualifiers to disambiguate module_1.Database from module_2.Database.

• In addition, most code editors are able to provide extra functionality, such as reliable code completion, the ability to jump to the definition of a class, or inline documentation, if normal imports are used. The import * syntax can hamper their ability to do this reliably.

• Finally, using the import * syntax can bring unexpected objects into our local namespace. Sure, it will import all the classes and functions defined in the module being imported from, but unless a special __all__ list is provided in the module, this import will also import any classes or modules that were themselves imported into that file.

Every name used in a module should come from a well-specified place, whether it is defined in that module, or explicitly imported from another module. There should be no magic variables that seem to come out of thin air. We should always be able to immediately identify where the names in our current namespace originated. If we use this evil syntax, one day we’ll have extremely frustrating moments of where on earth can this class be coming from?

Tip: Try typing the import this into your interactive interpreter. It prints a nice poem (with a couple of inside jokes) summarizing some of the idioms that Pythonistas tend to practice. Specific to this discussion, note the line “Explicit is better than implicit.” Explicitly importing names into our namespace makes our code much easier to navigate than the implicit from module import * syntax.

Practice all the from .. import types of commands in the playground below: