Python 3: An interactive deep dive/

...

/

Fixing What 2to3 Can’t

Fixing What 2to3 Can’t

We'll cover the following...

False is invalid syntax

You do have tests, right?

Now for the real test: running the test harness against the test suite. Since the test suite is designed to cover all the possible code paths, it’s a good way to test our ported code to make sure there aren’t any bugs lurking anywhere.

Press + to interact
C:\home\chardet> python test.py tests\*\*
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    from chardet.universaldetector import UniversalDetector
  File "C:\home\chardet\chardet\universaldetector.py", line 51
    self.done = constants.False
                              ^
SyntaxError: invalid syntax

Hmm, a small snag. In Python 3, False is a reserved word, so you can’t use it as a variable name. Let’s look at constants.py to see where it’s defined. Here’s the original version from constants.py, before the 2to3 script changed it:

Press + to interact
import __builtin__
if not hasattr(__builtin__, 'False'):
    False = 0
    True = 1
else:
    False = __builtin__.False
    True = __builtin__.True

This piece of code is designed to allow this library to run under older versions of Python 2. Prior to Python 2.3, Python had no built-in bool type. This code detects the absence of the built-in constants True and False, and defines them if necessary.

However, Python 3 will always have a bool type, so this entire code snippet is unnecessary. The simplest solution is to replace all instances of constants.True and constants.False with True and False, respectively, then delete this dead code from constants.py.

So this line in universaldetector.py:

Press + to interact
self.done = constants.False

Becomes

Press + to interact
self.done = False

Ah, wasn’t that satisfying? The code is shorter and more readable already.

No module named constants

Time to run test.py again and see how far it gets.

Press + to interact
C:\home\chardet> python test.py tests\*\*
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    from chardet.universaldetector import UniversalDetector
  File "C:\home\chardet\chardet\universaldetector.py", line 29, in <module>
    import constants, sys
ImportError: No module named constants

What’s that you say? No module named constants? Of course there’s a module named constants. It’s right there, in chardet/constants.py.

Remember when the 2to3 script fixed up all those import statements? This library has a lot of relative imports — that is, modules that import other modules within the same library — but the logic behind relative imports has changed in Python 3. In Python 2, you could just import constants and it would look in the chardet/ directory first. In Python 3, all import statements are absolute by default. If you want to do a relative import in Python 3, you need to be explicit about it:

Press + to interact
from . import constants

But wait. Wasn’t the 2to3 script supposed to take care of these for you? Well, it did, but this particular import statement combines two different types of imports into one line: a relative import of the constants module within the library, and an absolute import of the sys module that is pre-installed in the Python standard library. In Python 2, you could combine these into one import statement. In Python 3, you can’t, and the 2to3 script is not smart enough to split the import statement into two.

The solution is to split the import statement manually. So this two-in-one import:

Press + to interact
import constants, sys

Needs to become two separate imports:

Press + to interact
from . import constants
import sys

There are variations of this problem scattered throughout the chardet library. In some places it’s “import constants, sys”; in other places, it’s “import constants, re”. The fix is the same: manually split the import statement into two lines, one for the relative import, the other for the absolute import.

Onward!

Name ‘file’ is not defined

open() is the new file(). PapayaWhip is the new black.

And here we go ...

Access this course and 1400+ top-rated courses and projects.