Building an environment for python developments with GUIX

Introduction

On our computing servers, we provide Nix and GUIX as software package managers. With these tools, users can be autonomous in setting up their software environments (on the contrary to module command) while sharing the packages already installed (on the contrary of the conda command) to avoid to fill each user $HOME with common software packages.

However, these tools provide also very useful commands to work in isolated environments. In this tutorial, we will show how GUIX can replace Python virtualenv command and/or mechanisms, in a very powerful way and also applicable outside Python world.

Short reminder of main guix commands

Guix commandMain role
guix search stringbrowse the guix repositories to find packages matching string
guix install packages_nameinstall in the current profile the packages
guix install -p $GUIX_USER_PROFILE_DIR/profile_name package_nameinstall the package_name in the profile_name
guix remove package_nameremove in the current profile the package_name
guix pullupdate the package repository to have the last version of the packages (it may take a while :)
guix package –list-installed (or guix package -I)check what is actually installed
guix package -p $GUIX_USER_PROFILE_DIR/tuto1 -Isame as previous command in a specific profile (here tuto1)

Short reminder of specific to Gricad clusters guix commands

CommandMain role
source /applis/site/guix-start.shload guix default profile
refresh_guix $GUIX_USER_PROFILE_DIR/profile_nameload the profile_name

Introduction to the shell guix command

Role of the command

To create a development environment for python, independant from the packages already installed on the cluster, we introduce here the command: guix shell.

This role was previously carried out by the guix environment command, nowadays outdated.

This command takes one or more packages as argument, build the entries and generates a dedicated shell environement to use them, creating a new profile.

$ guix shell *options* *list of packages* 
$ guix shell emacs

Dependancies

A guix package comes with its dependancies for the building and for the execution. Then, guix shell automatically builds them. Consequently, the new shell environment is an improved version of the original shell. It contains necessary PATH to build and execute the package. The new necessary PATHS are sumed up to the default $PATH.

Installing guix python packages

Preliminary remark: guix indexes python- for python3 packages and python2- for python2 packages.

$ guix shell python2-numpy python2@2.7.17

We can launch the python command just after the installation command in running:

$ guix shell python2-numpy python2@2.7.17 -- python

Python 2.7.17 (default, Jan  1 1970, 00:00:01) 
[GCC 7.5.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 

Note the space between – and the command.

Your python environment is immediatly usable. Note: Dependancies are also automtically installed (even there are not python dependancies).

In this environment, the variable PATH and PYTHONPATH are the following:

$ echo $PATH

/gnu/store/2gh3rsdqq9k0plpcnv45bah5lvvqn58f-profile/bin:/gnu/store/3w0fcgs37pq49a8qa87kr9rgv9p4bynl-profile/bin:/nix/var/nix/profiles/default/bin:/applis/site/other/visu_dahu:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/sbin

Here, you can see that the “python2” environment has been sumed to original environment (note the /usr/bin for example).

echo $PYTHONPATH
/gnu/store/2gh3rsdqq9k0plpcnv45bah5lvvqn58f-profile/lib/python2.7/site-packages

To completely isolate your python2 environment, you will use the –pure option. It prevents the use of another previous python installation and related PATH:

$ guix shell --pure  python2-numpy python2@2.7.17

$ (guix shell) echo $PATH
/gnu/store/2gh3rsdqq9k0plpcnv45bah5lvvqn58f-profile/bin

and

$ (guix shell)  echo $PYTHONPATH

/gnu/store/2gh3rsdqq9k0plpcnv45bah5lvvqn58f-profile/lib/python2.7/site-packages

Be careful in this case to install the package coreutils to keep basic unix commands.

Conclusion

You are now able to develop your own python programs in an isolated environement, suitable to make your experiments reproducible.