1   Introduction and motivation

For more information about Python virtual environments, see the PEP (Python enhancement proposal): https://www.python.org/dev/peps/pep-0405/

Why would you want to create and use one or more Python virtual environments? If you are reading this, you likely already have a reason and a need. Here are a few more:

  • You will able to install Python packages, e.g. using pip, in a temporary environment without polluting your "main" or global Python environment. Doing some will enable you to fall back to you global Python environment, which will be unaltered by your use and experimentation with the temporary (or not so temporary) virtual environment.
  • You can create more than one virtual environment with a different version or release of Python in each one. Note, however, doing so requires that you have those different versions of Python installed on your machine.
  • You can create more than one virtual environment with different packages loaded in each one (or different versions of the same packages). Why? For example, your customer/user may need you to test with specific versions of specific Python packages.

Of the implementations described below, which should you choose?

  • virtualenvwrapper makes it especially easy to activate any one of several virtual environments and to switch between different virtual environments.
  • virtualenv is simpler.
  • venv is also simple, and it is included in the Python standard library (Python 3.3+), which means that you do not need to install it and that it is likely to have future support.

In thinking about using virtualenvwrapper, perhaps we should keep in mind that the ability to easily activate a different virtual environment in separate terminal windows/sessions and switch between those sessions is good enough and quick enough and convenient enough.

2   Versions and implementations of virtual environments

The implementations that we will consider are virtualenv, virtualenvwrapper, and venv.

2.1   virtualenv

You can read about virtualenv here -- https://pypi.org/project/virtualenv/

Install it with $ pip install virtualenv. You may have to use sudo.

Create a new virtual environment and the directory it will live in with the following: $ virtualenv path_to_new_virtualenv_directory.

Activate the virtual environment with this: $ source path_to_new_virtualenv_directory/bin/activate.

Deactivate with the following: $ deactivate.

If you need two virtual environments active at the same time, consider activating them in separate terminal sessions/windows.

And, to make it more convenient to activate a virtual environment, consider creating a shell alias, e.g. on Linux:

alias env1='source path_to_env01/bin/activate'
alias env2='source path_to_env02/bin/activate'

2.2   virtualenvwrapper

Install with $ pip install virtualenvwrapper. You may have to use sudo.

Follow the setup instructions here: https://virtualenvwrapper.readthedocs.io/en/latest/install.html.

Create a new virtual environment with the following: $ mkvirtualenv name_of_my_new_ve.

If you have more than one virtual environment, switch to using one or another with $ workon name_of_my_other_ve.

For help with the commands available for virtualenvwrapper, do: $ virtualenvwrapper.

If you need two virtual environments active at the same time, consider activating them in separate terminal sessions/windows, although switching between virtual environments with $ workon my_environment, is particularly easy with virtualenvwrapper.

2.3   venv

Installation -- No installation is needed. The venv module is part of the Python standard library for reasonably recent versions of Python (Python 3.3+). But on Ubuntu Linux, I had to install python3.9-venv ($ aptitude install python3.9-venv, or you can use apt).

For instructions on venv, see: https://docs.python.org/3/library/venv.html. For help, run $ python -m venv --help.

Create a new virtual environment with the following: $ python -m venv path_to_new_virtualenv_directory.

Activate the virtual environment with this: $ source path_to_a_new_directory/bin/activate. For those of you who are on MS Windows, that bin directory should contain something that you can use to activate the virtual environment.

Deactivate the virtual environment with this: $ deactivate. Note that a bash or shell function will have been defined for you.

If you need two virtual environments active at the same time, consider activating them in separate terminal sessions/windows.

3   A few notes on pip

pip is the tool you will likely use to manage the packages that you have installed in your virtual environment. So here are a few hints and usage notes.

pip does more than what I'll describe here. You can get help with:

$ pip help
$ pip help <cmd>

You can get a list of installed packages with either of the following:

$ pip list
$ pip freeze

You can get a list of installed packages that can be upgraded (that is, for which there are more recent versions than the one installed in your virtual environment) with the -o (--outdated) option to the list command:

$ pip list -o

You can upgrade the out-of-date packages with this:

$ pip install -U module1 module2 module3

Another method is to use pip freeze to get a list of installed packages. Then, fix it either (1) by removing the "==" and everything to the right of it on each line or (2) by changing the occurrence of "==" on each line to ">=". The following script will do the second:

#!/usr/bin/env python

# fix-freeze.py

import sys

for line in sys.stdin:
    sys.stdout.write(line.replace('==', '>='))

Once you have a file containing the list of packages, upgrade them with the following:

$ pip install -U -r <module-list-file>

Or, you can upgrade needed packages by creating a file containing a list of out-of-date packages with $ pip list -o and then delete the column header lines at the beginning of the file.

4   FAQ -- Frequently asked questions

How can I make a duplicate of a virtual environment?

Suppose you want to make a new virtual environment, for example with venv, and in this new virtual environment you want all the packages that are installed in an existing virtual environment.

You can follow these steps:

  1. Save the list of packages installed in the original (existing) virtual environment by running the following while that existing virtual environment is active:

    $ pip freeze > pkgs
    
  2. Create the new virtual environment, e.g. using venv. Do this:

    $ python -m venv my_new_ve
    
  3. Deactivate the old virtual environment (if necessarry) and activate the new one with the following:

    $ deactivate
    $ source my_new_ve/bin/activate
    
  4. Edit the package file pkgs and remove the "==" and everything after it on each line. For example, change:

    flake8==4.0.1
    

    to this:

    flake8
    
  5. Then install those packages (listed in file pkgs) with this:

    $ pip install -U -r pkgs
    

How can I upgrade the packages that I've installed in my virtual environment to the latest versions?

Here is a link to a discussion of ways to do this -- https://stackoverflow.com/questions/2720014/how-to-upgrade-all-python-packages-with-pip

And, here are a several alternative approaches:

Method 1:

  1. Capture the list of packages in your current virtual environment:

    $ pip freeze > pkgs
    
  2. Change occurrences of "==" to ">=". Use your text editor, or here is a Python script that you can use:

    #!/usr/bin/env python
    import sys
    
    for line in sys.stdin:
        sys.stdout.write(line.replace('==', '>='))
    
  3. Run the following to upgrade the packages in your current virtual environment:

    $ pip install -U -r pkgs
    

Or, create your list of packages to be upgraded with: $ pip list --outdated.

Method 2:

You can consider pipupgrade -- https://pypi.org/project/pipupgrade/. I tried it, but it did some things that I did not want done. But, perhaps it does what you want. And, it has additional features as well.


Published

Category

python

Tags

Contact