Dave's page

Compiling/building Python 3 etc. from source

1   Introduction

Building Python 3 from source is reasonably straight-forward, but I struggled with a few of the steps. The reason for that struggle is that I wanted to install Python 3 and Pelican on a machine where I do not have root access. So I had to figure our how to build those libraries and how to tell Python where to find them. Pelican is, by the way, a static Web site generator implemented in Python -- see https://docs.getpelican.com/en/latest/index.html.

Here is what we'll cover in this post:

  • How to configure, build, and install openssl from source.
  • How to configure, build, and install libffi from source.
  • How to configure, build, and install Python 3 from source.
  • How to configure, build, and install SQLite3 from source.

How to install and setup a few extra things, e.g. lxml, ipython, virtualenv, etc.

2   Building openssl

I needed openssl because using pip (the Python package installer) requires it.

You can find the source for openssl here: https://wiki.openssl.org/index.php/Compilation_and_Installation#Retrieve_source_code

I configured, compiled, and installed it with the following:

$ tar xf openssl-1.1.1k.tar.gz    # Will vary depending on version
$ ln -s openssl-1.1.1k openssl
$ cd openssl    # Will vary depending on version
$ ./config --prefix=<my-home-directory>/opt/opensll
$ make
$ make install

3   Building libffi

I needed the Python ctypes module, and that requires libffi.

You can find the source here: https://sourceware.org/libffi/

I configured, compiled, and installed it with the following:

$ ./configure --prefix=<my-home-directory>/opt
$ make
$ make install

4   Building SQLite3

I needed SQLite3 because I use IPython, and IPython uses SQLite3 to store command history.

You can find the source here: https://sqlite.org/src/doc/trunk/README.md

I configured, compiled, and installed it with the following:

$ mkdir sqlite_bld
$ cd sqlite_bld
$ ../sqlite/configure --prefix=<my-home-directory>/opt
$ make
$ make install

5   Building Python 3

I configured Python 3 with the following script:

#!/bin/bash -x
make distclean
./configure \
    --prefix=<my-home-directory>/opt \
    --with-openssl=<my-home-directory>/opt \
    --with-ensurepip=install \
    LDFLAGS='-L<my-home-directory>/opt/lib'

I'm actually not sure that the use of LDFLAGS above is necessary, but it worked for me.

I compiled with the following script:

#!/bin/bash -x
LD_LIBRARY_PATH=<my-home-directory>/opt/lib \
LD_RUN_PATH=<my-home-directory>/opt/lib \
make

Setting LD_RUN_PATH (above) enables the python executable to find the libraries that we've built and installed, for example openssl and libffi.

I installed with the following script:

#!/bin/bash -x
LD_LIBRARY_PATH=<my-home-directory>/opt/lib \
make install

6   Installing Python under a virtual environment

I used virtualenv to create a virtual environment under which I could install extra packages. That way I could leave my original/main Python 3 installation relatively clean.

You can do something like the following:

$ pip install virtualenv
$ mkdir Envs
$ cd Envs
$ virtualenv -p ~/opt/bin/python env01    # or some other name of your choice
$ source env01/bin/activate

Our new environment is now active. Now, when we use pip to install a package, it will be installed under that new environment/directory.

Whenever you want to activate that Python virtual environment, you will need to run:

$ source env01/bin/activate

You can consider adding that line to your logon startup file. In my case, that's .bashrc.

If you find that you want to switch between several different Python environments, be sure to investigate virtualenvwrapper. It enables you to install different packages under different Python virtual environments and to switch between them easily and quickly. You can find out about it here: https://virtualenvwrapper.readthedocs.io/en/latest/

7   Additional tools and packages

I also installed a few extra packages, for example:

$ pip install lxml
$ pip install ipython

Published

Category

python

Tags

Contact