Python modules are one of the foundational concepts you have to know if you want to become an expert Python developer.
A Python module is a file with extension .py that contains statements (instructions that the interpreter can execute) and definitions (e.g. function definitions). Modules help you make your code easier to maintain and they can be either executed or imported from other modules.
Without understanding modules you won’t truly understand how Python code works.
That’s why with this tutorial I want to help you get to a point where you are comfortable creating and using Python modules.
Let’s start writing some code!
What is a Python Module?
A module is a file with .py extension that contains statements and definitions. The name of the module is the name of the file with .py extension. For example, you will write the code for the module xyz in a file called xyz.py.
How do you create a Python module?
Let’s say we want to create a Python module called words_parser that contains two functions.
Create a file called words_parser.py and add the following code to it:
import random
def select_word(words):
return random.choice(words)
def get_unique_letters(word):
return "".join(set(word))In the first line of our new module we use an import statement to import the random module.
Why are we importing the random module in our words_parser module?
Because we want to use it in the logic of one of the two functions we will define in our module: the select_word() function.
- The first function selects a random word from a list of words.
- The second function returns a string that contains all the characters part of a word without including duplicates.
How Can You Use a Python Module?
To import the module we have created open the Python shell and use the following command:
>>> import words_parserAs you can see the name of the module we are importing is the name of the module’s file without .py extension.
Use the globals() built-in function to examine the current global symbol table.
Don’t worry too much if you don’t understand the meaning of symbol table right now…
Think about it as a list of names you can access in your code.
>>> globals()
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, 'words_parser': <module 'words_parser' from '/opt/coding/python/tutorials/words_parser.py'>}The module words_parser has been added to the current symbol table and for this reason we can call it in our code.
Also notice that the names of the functions defined inside the words_parser module have not been added to the symbol table.
Note: the name of the current module is __main__. This is the value of the global variable __name__.
To call the two functions defined in the words_parser module we will use the dot notation that follows the syntax below:
module_name.function_name()Call the two functions defined in our module to see if they work.
Test Function select_word
>>> words = ["module", "python", "symbol", "table"]
>>> words_parser.select_word(words)
'symbol'Test function get_unique_letters
>>> words_parser.get_unique_letters("programming")
'amgpnori'They both work fine!
I have created a separate tutorial if you want to learn how to execute a module as script.
How to Import a Module From Another Module
In the previous section we have imported the words_parser module from the Python shell.
Let’s see what happens when we import the words_parser module from another module. To do this we will create a second module called app_parser.
The idea is that the app_parser module could import more specific parsers like the words_parser.
Create a file called app_parser.py with the following content in the same directory where words_parser.py is.
import words_parser
print(f"The name of the current module is {__name__}")
print(f"The name of the imported module is {words_parser.__name__}")
words = ["module", "python", "symbol", "table"]
selected_word = words_parser.select_word(words)
print(f"The word selected is: {selected_word}")
word = "programming"
unique_letters = words_parser.get_unique_letters(word)
print(f"The letters of the word {word} without including duplicates are: {unique_letters}")In the first line we import the words_parser module.
Then we print the name of the current module (the value of the __name__ global variable).
Finally we call both functions from the words_parser module and we print the results we get back.
Here is the output:
The name of the current module is __main__
The name of the imported module is words_parser
The word selected is: table
The letters of the word programming without including duplicates are: gprnomaiThe name of the current module is __main__ because we are calling the module app_parser directly.
The name of the imported module is words_parser because we are importing it and we are not executing it directly (learn more about the difference between importing a module vs executing it directly).
To print the global symbol table you can use the following code:
print(f"The global symbol table of the importing module is: {globals()}")
[output]
The global symbol table of the importing module is: {'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x7fd858165730>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, '__file__': '/opt/coding/python/tutorials/app_parser.py', '__cached__': None, 'words_parser': <module 'words_parser' from '/opt/coding/python/tutorials/words_parser.py'>}You can see that the imported module words_parser is part of the importing module’s global symbol table.
What is “from import” in Python?
In the previous section we have seen how to import a module using the import statement.
We have also seen that after doing that the name of the imported module was added to the global symbol table of the importing module.
There is also another way to write import statements in Python: using the from import statement.
from words_parser import select_word, get_unique_lettersThis time we have imported the functions defined in the words_parser module instead of the module itself.
Let’s see how the global symbol table of the importing module looks like:
The global symbol table of the importing module is: {'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <_frozen_importlib_external.SourceFileLoader object at 0x7fe940165730>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, '__file__': '/opt/coding/python/tutorials/app_parser.py', '__cached__': None, 'select_word': <function select_word at 0x7fe94026a8b0>, 'get_unique_letters': <function get_unique_letters at 0x7fe9402d35e0>}This time we don’t see the words_parser module in the global symbol table but we see the two functions we have imported: select_word and get_unique_letters.
Do you see the difference compared to the previous import statement?
At this point when calling the functions part of the module you won’t have to use the dot notation (module.function), you can simply use the name of the functions:
words = ["module", "python", "symbol", "table"]
selected_word = select_word(words)
print(f"The word selected is: {selected_word}")
word = "programming"
unique_letters = get_unique_letters(word)
print(f"The letters of the word {word} without including duplicates are: {unique_letters}")
[output]
The word selected is: table
The letters of the word programming without including duplicates are: imnagproIt’s also possible to import all the names defined in our module using the following syntax:
from words_parser import *At the same time this is not a suggested approach considering that it’s a good programming practice to be as specific as possible in the code you write.
When you use the * in your import statements it’s not clear what you are importing in your program and what your intentions are.
In other words this makes your code harder to read for other developers.
What is “import as” in Python?
When importing a Python module you can also import it with a different name that you can use in your program when referring to that module.
Here is an example:
>>> import words_parser as wp
>>> globals()
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, 'wp': <module 'words_parser' from '/opt/coding/python/tutorials/words_parser.py'>}You can see that the name wp is added to the symbol table instead of words_parser.
After importing the words_parser module as wp we can use wp to call the functions defined in the module.
wp.select_word()
wp.get_unique_letters()You can also use the as keyword with from import.
from words_parser import select_word as sw
from words_parser import get_unique_letters as guq
words = ["module", "python", "symbol", "table"]
selected_word = sw(words)
print(f"The word selected is: {selected_word}")
word = "programming"
unique_letters = guq(word)
print(f"The letters of the word {word} without including duplicates are: {unique_letters}")This is not very readable so it’s something you might not want to do.
At the same time it’s good for you to be aware of all the options you have available when importing Python modules.
Where Does Python Search For Modules?
In this example we have imported a module we have created in the current directory.
But, how does the Python interpreter know how to import modules that are located in different parts of the filesystem?
For example, if I execute the following import it works fine:
>>> import os
>>> globals()
{'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__spec__': None, '__annotations__': {}, '__builtins__': <module 'builtins' (built-in)>, 'os': <module 'os' from '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/os.py'>}In this specific case the Python interpreter knows that the os module can be imported from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/os.py.
Note: This directory depends on your operating system and on your Python installation.
But, how does the Python interpreter know where to find the module os.py?
When importing a module Python looks for a built-in module with that name first. Then it looks for modules in the list of directories part of the PYTHONPATH (the value of the variable sys.path). For example if you import the module xyz Python looks for xyz.py in those directories. The first element in the PYTHONPATH list is an empty directory that represents the current directory.
Below you can see the list of directories part of the PYTHONPATH on my machine:
>>> import sys
>>> sys.path
['', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python38.zip', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/lib-dynload', '/opt/coding/python/tutorials/python-env/lib/python3.8/site-packages']
>>> type(sys.path)
<class 'list'>Note: the directories in the PYTHONPATH also depend on your Python installation.
In the next sections we will see:
- how to find the location of the file for a specific module.
- how to add more directories to the PYTHONPATH.
How Can You Find the Location of a Python Module?
Is there a quick way to find out where the file for a module is located on the filesystem?
The answer is yes!
Python modules provide a string attribute called __file__ that contains the filesystem location of the module.
Let’s test it with our custom module…
>>> import words_parser
>>> words_parser.__file__
'/opt/coding/python/tutorials/words_parser.py'
>>> type(words_parser.__file__)
<class 'str'>And also with the Python os built-in module…
>>> import os
>>> os.__file__
'/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/os.py'That’s quite useful!
How Can You Update the PYTHONPATH?
A few sections above we have seen how to import the words_parser module simply by being in the same directory where we have created the file for that module.
The import was working because the Python interpreter was looking for that module in the current directory.
Now let’s do a little experiment…
Create a directory called modules_tutorial and move the file words_parser.py to that directory.
$ mkdir modules_tutorial
$ mv words_parser.py modules_tutorialNow, don’t change directory and open the Python shell from the current directory that is the parent directory of the directory modules_tutorial.
>>> import words_parser
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'words_parser'When we import the words_parser module the Python interpreter cannot find it because we have moved it to a different directory.
One way to make this work is to update the PYTHONPATH.
As explained before the PYTHONPATH is a list of directories stored in sys.path.
Considering that sys.path is a list we can use the list append method to add directories to it.
Let’s try to add the new directory we have created (modules_tutorial) to the Python path:
>>> sys.path.append('/opt/coding/python/tutorials/modules_tutorial')
>>> sys.path
['', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python38.zip', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/lib-dynload', '/opt/coding/python/tutorials/python-env/lib/python3.8/site-packages', '/opt/coding/python/tutorials/modules_tutorial']
As you can see the new directory we have created has been appended at the end of the sys.path list.
Try to import the words_parser module again…
>>> import words_parser
>>> words_parser.__file__
'/opt/coding/python/tutorials/modules_tutorial/words_parser.py'This time the import is successful.
Note: the change to sys.path is lost if you exit from the Python shell and then you reopen it. So if you want to persist it you have to add it to your Python program.
Using the dir() Function with Python Modules
The dir() built-in function allows to see which names are defined in a specific Python module.
Here is the output of the dir() function when we apply it to our words_parser module.
>>> import words_parser
>>> dir(words_parser)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'get_unique_letters', 'random', 'select_word']You can see that some names are set by default considering that we haven’t set them (e.g. __builtins__, __name__, __package__, etc….).
Other names are related to the code we have added to the words_parser module:
- the random module imported at the beginning of the words_parser module.
- the select_word() function.
- the get_unique_letters() function.
Conclusion
In this tutorial we went through concepts that will help you understand, create and use Python modules.
You now know what modules are, how to create them, how to use them, how to import them, how to find their location and how to suggest new locations for modules to the Python interpreter.
And now it’s time for you to put this knowledge in practice!
I’m a Software Engineer and Programming Coach. I want to help you in your journey to become a Super Developer!
