How Python Virtualenv Works: Let’s Find Out!

Understanding how Python virtualenv works is one of the things you should know when you get started with Python.

Python virtualenv creates an isolated environment in which you can install all Python dependencies you need for your project. If you work on multiple projects that require different dependencies using virtual environments allows to keep those environment separate. It also allows to have an environment that doesn’t interfere with the global system Python.

In this guide you will learn what you need to start using virtualenv for your projects. We will go through practical examples that will help you understand how virtualenv works.

Virtualenv…here we go!

Why Would You Use a Virtual Environment in Python

As explained in the previous section Python virtualenv allows you to create an environment that contains all the dependencies (packages) needed by Python to execute projects.

A virtual environment is a single directory that can be created as I explain in the next section.

For now the important concept is that…

Using virtual environments allows you to manage multiple projects without risking Python dependencies of one project to break another project.

A simple example is the scenario in which ProjectA requires version 1.0.0 of a package and ProjectB only works with version 1.1.0 of the same package.

Running both projects using the system Python environment would not be possible and that’s where virtualenv can help.

How Do You Create a Virtual Environment in Python?

The examples I will be showing here are based on Linux. The concepts are very similar for Mac and Windows, certain commands are slightly different on Windows.

Firstly, I have created two aliases inside .bashrc for the ec2-user to refer to python3 and pip3 simply as python and pip:

alias python=python3
alias pip=pip3

First of all, I will confirm that Python and PIP are installed:

[ec2-user@host ~]$ python --version
Python 3.7.8
[ec2-user@host ~]$ pip --version
pip 9.0.3 from /usr/lib/python3.7/site-packages (python 3.7)

As you can see I have Python 3.7.8 and PIP 9.0.3 on my system. The versions on your system might differ from those depending on your Linux distribution and version.

The first step to create a virtual environment is to install the virtualenv package using pip:

[ec2-user@host ~]$ pip install virtualenv --user
Collecting virtualenv
  Using cached https://files.pythonhosted.org/packages/1d/09/9179b676c126b2687bf4110e5b88c8c52d9113f31bd5f8f6ab97d380e434/virtualenv-20.0.30-py2.py3-none-any.whl
Requirement already satisfied: appdirs<2,>=1.4.3 in /usr/local/lib/python3.7/site-packages (from virtualenv)
Requirement already satisfied: six<2,>=1.9.0 in /usr/local/lib/python3.7/site-packages (from virtualenv)
Requirement already satisfied: distlib<1,>=0.3.1 in /usr/local/lib/python3.7/site-packages (from virtualenv)
Requirement already satisfied: importlib-metadata<2,>=0.12; python_version < "3.8" in /usr/local/lib/python3.7/site-packages (from virtualenv)
Requirement already satisfied: filelock<4,>=3.0.0 in /usr/local/lib/python3.7/site-packages (from virtualenv)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/site-packages (from importlib-metadata<2,>=0.12; python_version < "3.8"->virtualenv)
Installing collected packages: virtualenv
Successfully installed virtualenv-20.0.30

The –user flag tells pip to install the virtualenv package in a local directory inside the home directory of the current user (ec2-user).

This command would fail if we don’t pass the –user flag because ec2-user doesn’t have access to install packages in the system Python library.

The following command confirms that virtualenv is installed successfully:

[ec2-user@host ~]$ virtualenv --version
virtualenv 20.0.30 from /home/ec2-user/.local/lib/python3.7/site-packages/virtualenv/__init__.py

Now, to create the virtual environment go to the directory where you want to develop your Python projects (called projects in this example) and run the following command:

virtualenv codefather

This creates a virtual environment called codefather. Here is the output on my Linux system:

[ec2-user@host blog]$ cd projects/
[ec2-user@host projects]$ virtualenv codefather
created virtual environment CPython3.7.8.final.0-64 in 909ms
  creator CPython3Posix(dest=/opt/blog/projects/codefather, clear=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/ec2-user/.local/share/virtualenv)
    added seed packages: pip==20.2.1, setuptools==49.2.1, wheel==0.34.2
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

The Structure of Python Virtual Environments

Before learning how to use the virtual environment we have created, let’s have a look at its directory structure. To do that I use the tree command with a display depth of the directory tree equal to 2.

The tree command is not available on my system and I can install it using:

sudo yum install tree

And here is the structure of the virtual environment we have created:

[ec2-user@host projects]$ tree -L 2 codefather
codefather
├── bin
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── activate.ps1
│   ├── activate_this.py
│   ├── activate.xsh
│   ├── easy_install
│   ├── easy_install3
│   ├── easy_install-3.7
│   ├── easy_install3.7
│   ├── pip
│   ├── pip3
│   ├── pip-3.7
│   ├── pip3.7
│   ├── python -> /usr/bin/python3
│   ├── python3 -> python
│   ├── python3.7 -> python
│   ├── wheel
│   ├── wheel3
│   ├── wheel-3.7
│   └── wheel3.7
├── lib
│   └── python3.7
├── lib64
│   └── python3.7
└── pyvenv.cfg

5 directories, 22 files

The Python binary you see in the bin directory is a symlink to the system Python 3 binary. In the bin directory you can also see a script that we will analyse in the next section: activate.

When the virtualenv is created it also includes the lib directory that contains modules and packages.

But, how does Python know to look for packages in the lib directory inside the virtual environment?

That’s because the Python binary looks for lib directories relative to its path, and the first lib directory it finds is ../lib/python3.7/.

To create a virtualenv that uses a different version of Python e.g. Python 2.7, I can use the following command:

virtualenv -p /usr/bin/python2.7 codefather

And now let’s see how to use the virtual environment we have created!

Activate a Virtual Environment

Creating a virtual environment is not enough, in order to use it you also have to activate it.

If you don’t activate the virtualenv after creating it you will still be installing any packages in the system-wide Python distribution.

Remember to activate your virtualenv using the source command:

[ec2-user@host projects]$ source codefather/bin/activate
(codefather) [ec2-user@host projects]$ 

As soon as you activate your virtualenv the Linux prompt changes. You will see the name of the virtual environment surrounded by parentheses on the left.

Ok, but what does actually happen when you activate a virtualenv?

The first thing I want to find out is which Python interpreter I’m using before and after activating the virtual environment. To do that I can use the Linux which command:

Before

[ec2-user@host projects]$ which python
alias python='python3'
	/usr/bin/python3

After

[ec2-user@host projects]$ source codefather/bin/activate
(codefather) [ec2-user@ip-172-31-28-249 projects]$ which python
alias python='python3'
	/opt/blog/projects/codefather/bin/python3

How can the Python interpreter used by default change after activating the virtual environment?

Virtualenv and the PATH Environment Variable

The first thing I can think about that could make this happen is a change in the value of the PATH Linux environment variable.

The PATH environment variable lists the directories used by Linux to execute binaries without specifying their full path.

A quick way to confirm this is by using the echo command to print the value of the PATH environment variable before and after activating the Python virtualenv.

Before

[ec2-user@host ~]$ echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin

After

[ec2-user@host projects]$ source codefather/bin/activate
(codefather) [ec2-user@ip-172-31-28-249 projects]$ echo $PATH
/opt/blog/projects/codefather/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin

That’s confirmed!

As you can see, after activating the virtual environment the first directory in the PATH is the bin directory inside the virtual environment we created and activated.

If we want to dig a bit deeper, we can have a look at the content of the activate script inside /opt/blog/projects/codefather/bin/.

The following lines show what the activate script does. It updates and exports the PATH environment variable:

VIRTUAL_ENV='/opt/blog/projects/codefather'
export VIRTUAL_ENV

PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH

Here you can learn more how export works with Linux environment variables.

And now let’s see what to do when we want to stop working in our virtual environment.

Deactivate a Virtual Environment

In the same way you activate your virtualenv to limit your actions to the environment, you can also deactivate it.

Deactivating a virtualenv brings you back to the original environment you came from.

Here’s an example…

(codefather) [ec2-user@host projects]$ deactivate 
[ec2-user@host projects]$ 

As you can see from the left part of the prompt, before running the deactivate command I’m in the codefather virtualenv.

After executing the deactivate command the name of the virtualenv disappears from the prompt. This indicates that we are not in the virtual environment anymore.

Before moving to the next section, I would like to find out where the deactivate command is. I didn’t see it in the directory structure of the virtualenv where only the activate script was present.

Let’s see if we can solve this mystery!

Got it, deactivate is not a command…

…it’s a function defined in the activate script:

deactivate () {
    unset -f pydoc >/dev/null 2>&1

    # reset old environment variables
    ... 
    ...
    ...
    [full content of the function not included]
}

Good to know!

Installing Modules in a Python Virtualenv

So, now that we know how the structure of a virtualenv is, how to activate it and how to deactivate it. The next step is…

…understanding how new packages are installed in the lib directory that belongs to the virtualenv and not in the system lib directory.

The reason is similar to what we have seen with the python binary…

The bin directory inside the virtual environment also contains a pip binary that when executed installs packages inside codefather/lib/python3.7/.

From the moment in which you activate a virtualenv, every package you install using pip will end up in the virtual environment and will not be visible to the global Python environment.

Delete a Python Virtual Environment

Deleting a virtual environment is very simple. You can just delete its folder using the rm command after deactivating the virtual environment.

For instance, in my system I would run the following commands:

(codefather) [ec2-user@host projects]$ deactivate 
[ec2-user@host projects]$ rm -fr codefather

Pretty simple!

Conclusion

You now know what a Python virtual environment is and how it works.

In this guide we have seen how to create a virtual env, how to activate it to start using it for one your projects and how to deactivate it.

In addition, we have also seen:

• How to install modules in a virtualenv.
• How Python keeps everything self-contained in that virtual environment.

Does it make sense to you? Do you have any questions?

Let me know in the comments!