Guix is a package manager, namely a collection of software tools that automates the process of installing, upgrading, configuring, and removing softwares on the operating system.
References :
Since working on clusters implies that you’ll have to share the resources with many other users, each of them with different needs and contexts of use, using a tool to handle the installation of specific softwares is compulsory.
Guix is just one of the candidates to answer this requirement, among others like Nix, container systems or modules.
Convinced? Let’s try it and see some examples.
Through this tutorial, you will learn Guix basics, good practices and how to setup properly your environments. This might probably be enough for the needs of most of GRICAD users.
If you’re in a hurry you can directly step to Guix cheat sheet.
As examples, let us consider these two use cases:
Through this tutorial, we will detail all the actions required to setup these two ‘working environments’.
You are encouraged to try and run all the commands below.
First of all, any session where guix is required must start with the following command:
$ source /applis/site/guix-start.sh
It brings ‘guix’ and other commands into the current environment and ensures that everything is properly set for Guix usage.
Do not forget to add this line to your oar scripts!
Do not add this line in your bashrc, profile or similar, you will end with some unexpected side effects.
Upon the first call of ‘guix-start.sh’ on a cluster some background actions will occur, mostly to install and update guix on your account. This may take some time, please be patient …
Assuming that we need Python3, with numpy and matplotlib and the emacs text editor let us have a look at what is available and search for some of the required packages, for example:
$ guix search emacs
name: emacs
version: 26.3
outputs: out
systems: x86_64-linux i686-linux
dependencies: acl@2.2.53 alsa-lib@1.2.2 dbus@1.12.16 giflib@5.2.1 gnutls@3.6.12 gtk+@3.24.20 imagemagick@6.9.11-11 libice@1.0.10 libjpeg-turbo@2.0.4
+ libotf@0.9.16 libpng@1.6.37 librsvg@2.40.21 libsm@1.2.3 libtiff@4.1.0 libx11@1.6.9 libxft@2.3.3 libxml2@2.9.10 libxpm@3.5.13 m17n-lib@1.8.0
+ mailutils@3.9 ncurses@6.2 pkg-config@0.29.2 texinfo@6.7 zlib@1.2.11
location: gnu/packages/emacs.scm:76:2
homepage: https://www.gnu.org/software/emacs/
license: GPL 3+
synopsis: The extensible, customizable, self-documenting text editor
description: GNU Emacs is an extensible and highly customizable text editor. It is based on an Emacs Lisp interpreter with extensions for text
+ editing. Emacs has been extended in essentially all areas of computing, giving rise to a vast array of packages supporting, e.g., email, IRC and
+ XMPP messaging, spreadsheets, remote server editing, and much more. Emacs includes extensive documentation on all aspects of the system, from basic
+ editing to writing large Lisp programs. It has full Unicode support for nearly all human languages.
relevance: 33
...
Depending on the searched items, the output might be quite long. Using ‘grep’ or regexp might be a solution but you’ll do better to check directly in the online list of guix packages.
There you can find the following packages : python, python-numpy and python-matplotlib
Once the required packages names are clear, the installation process can start:
$ guix install emacs python@3 python-numpy python-matplotlib
... # MIGHT BE QUITE LONG ...
Note the @3 to specify the required version.
That’s it, python3 is available for you on all nodes, you can try it on any node of the cluster:
$ python3
Python 3.8.2 (default, Jan 1 1970, 00:00:01)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>import numpy as np
$ which python3
/home/perignon/.guix-profile//bin/python3
Before anything else, do not forget to source /applis/site/guix-start.sh for each new terminal session!
Else, your guix environment won’t be available.
There, you certainly noticed that python has been installed in $HOME/.guix-profile, as all the other packages installed above.
$HOME/.guix-profile is the default place where guix will install everything, with the standard organisation on unix systems, bin for binaries, lib for libraries and so on.
$ ls $HOME/.guix-profile/
bin etc include lib libexec manifest sbin share
$ ls $HOME/.guix-profile/lib
...
libpython3.so
...
$ ls $HOME/.guix-profile/bin
...
python3 emacs
...
Indeed, your .guix-profile contains only symbolic links to files from /gnu/store.
Either somebody has already installed the very same package on the cluster or you’re the first.
In the first case, Guix just creates the links in your profile path.
In the latter case, Guix first installs thing into /gnu/store and then creates the links in your profile path.
The following command will list all packages installed into the default profile:
$ guix package -I
glibc-locales 2.29 out /gnu/store/03nvilh2x4z07dxv7h13gh986vvgpnsf-glibc-locales-2.29
hello 2.10 out /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10
emacs 26.3 out /gnu/store/lza7sccqgrv313gbrb9j8f7lgn49irdn-emacs-26.3
python 3.8.2 out /gnu/store/fa1zp10w2sb7gdzk1fwqhr106mpq3lc7-python-3.8.2
python-numpy 1.17.3 out /gnu/store/rr1winibssxgxsf49f6rvdfdigifpviz-python-numpy-1.17.3
python-matplotlib 3.1.2 out /gnu/store/gsjhzr1xy96hz8wvaa8mblvv2niqnkwy-python-matplotlib-3.1.2
Notice glibc-locales and hello, always installed by default, and the others packages you asked for.
Let us step to the second use case, a working environment with gnu compilers and the fftw library.
The point is to install the GNU compilers and the library in a specific place. Compilers are usually available in the package ‘gcc-toolchain’ and guix search will tell you the name of the fftw package.
Let’s install them in a new profile called ‘gcc-fftw’. Dealing with profiles in guix is quite simple and handled with the option ‘-p’ on most of guix commands. This option tells guix to use a specific profile (which is no more than a path where things must be saved) and must be followed by the full (absolute) path to the profile. To simplify this, a specific environment variable is set by default for all users, GUIX_USER_PROFILE_DIR, and must be used:
$ guix install -p $GUIX_USER_PROFILE_DIR/gcc_fftw gcc-toolchain fftw
The following packages will be installed:
gcc-toolchain 10.1.0
fftw 3.3.8
...
The environment variable GUIX_USER_PROFILE_DIR is the place where all guix profiles must be saved. This is compulsory to ensure the availability of softwares on Dahu, Bigfoot and Luke, among other things.
$ echo $GUIX_USER_PROFILE_DIR
/var/guix/profiles/per-user/perignon
$ ls $GUIX_USER_PROFILE_DIR
...
current-guix gcc_fftw
...
Now we can check the list of installed packages in the default profile,
$ guix package -I
glibc-locales 2.29 out /gnu/store/03nvilh2x4z07dxv7h13gh986vvgpnsf-glibc-locales-2.29
hello 2.10 out /gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10
emacs 26.3 out /gnu/store/lza7sccqgrv313gbrb9j8f7lgn49irdn-emacs-26.3
python 3.8.2 out /gnu/store/fa1zp10w2sb7gdzk1fwqhr106mpq3lc7-python-3.8.2
python-numpy 1.17.3 out /gnu/store/rr1winibssxgxsf49f6rvdfdigifpviz-python-numpy-1.17.3
python-matplotlib 3.1.2 out /gnu/store/gsjhzr1xy96hz8wvaa8mblvv2niqnkwy-python-matplotlib-3.1.2
$
and in the new profile, gcc_fftw
$ guix package -p $GUIX_USER_PROFILE_DIR/gcc_fftw -I
gcc-toolchain 10.1.0 out /gnu/store/q9vv9s1jj7hw5igckncysadn6gmccfxm-gcc-toolchain-10.1.0
fftw 3.3.8 out /gnu/store/85b70gpbs9issc1gbdcvnwb8pip6ihlk-fftw-3.3.8
And the default compiler …
$ which gcc
/usr/bin/gcc
$ gcc --version
gcc (Debian 8.3.0-6) 8.3.0
is still the one installed by default on the system, which is not what we want!
The path to the new guix profile (and so gcc) must be added to environment variable. To handle this, we provide a refresh_guix gcc_fftw function to activate or refresh a specific profile:
$ refresh_guix gcc_fftw
Activate profile /var/guix/profiles/per-user/stagiaire11/gcc_fftw
The following packages are currently installed in /var/guix/profiles/per-user/stagiaire11/gcc_fftw:
gcc-toolchain 10.1.0 out /gnu/store/q9vv9s1jj7hw5igckncysadn6gmccfxm-gcc-toolchain-10.1.0
fftw 3.3.8 out /gnu/store/85b70gpbs9issc1gbdcvnwb8pip6ihlk-fftw-3.3.8
$ which gcc
/var/guix/profiles/per-user/stagiaire11/gcc_fftw/bin/gcc/usr/bin/gcc
$ gcc --version
gcc (GCC) 10.1.0
which is what we expect!
Assume now that we need another version of the gnu compilers, but without removing the current one.
Doing this is very easy with guix, in a new profile, e.g. gcc9_fftw
# install and activate the profile
$ guix install -p $GUIX_USER_PROFILE_DIR/gcc9_fftw gcc-toolchain@9 fftw
...
$ refresh_guix gcc9_fftw
Activate profile /var/guix/profiles/per-user/stagiaire11/gcc9_fftw
The following packages are currently installed in /var/guix/profiles/per-user/stagiaire11/gcc9_fftw:
gcc-toolchain 9.3.0 out /gnu/store/4s25r2krb2x5c904gbzv0jkj2fyi5hnx-gcc-toolchain-9.3.0
fftw 3.3.8 out /gnu/store/85b70gpbs9issc1gbdcvnwb8pip6ihlk-fftw-3.3.8
$ gcc --version
gcc (GCC) 9.3.0
# and switch back easily to the other profile:
$ refresh_guix gcc_fftw
Activate profile /var/guix/profiles/per-user/stagiaire11/gcc_fftw
The following packages are currently installed in /var/guix/profiles/per-user/stagiaire11/gcc_fftw:
gcc-toolchain 10.1.0 out /gnu/store/q9vv9s1jj7hw5igckncysadn6gmccfxm-gcc-toolchain-10.1.0
fftw 3.3.8 out /gnu/store/85b70gpbs9issc1gbdcvnwb8pip6ihlk-fftw-3.3.8
$ gcc --version
gcc (GCC) 10.1.0
Finally, you can easily have a list of all your current profiles with
$ guix package --list-profiles
/home/perignon/.config/guix/current
/var/guix/profiles/per-user/perignon/gcc9_fftw
/var/guix/profiles/per-user/perignon/gcc_fftw
/home/perignon/.guix-profile
Notice the current and guix-profile directories, which must not be removed.
current is the place where guix is installed and guix-profile is your default profile.
To remove a package from a profile, just try
$ guix remove -p $GUIX_USER_PROFILE_DIR/gcc9_fftw fftw
# or from default profile
$ guix remove emacs
and to remove a profile, simply remove the concerned directory, e.g.:
$ rm -rf $GUIX_USER_PROFILE_DIR/gcc9_fftw
The tutorial is over. You ought to be able to use guix to create and manage your own working environments.
Here is a summary of the basic things and commands to know.
This should be run at the beginning of any terminal session where Guix is required.
$ source /applis/site/guix-start.sh
guix search <name>
Very verbose … It might be easier to check packages online: https://www.gnu.org/software/guix/packages/
We strongly recommend you to properly define and separate your working environments by using guix profiles.
# In/from default profile
$ guix install <package-name>
$ guix remove <package-name>
# In/from a given profile
$ guix install -p $GUIX_USER_PROFILE_DIR/<profile-name> <package-name>
$ guix remove -p $GUIX_USER_PROFILE_DIR/<profile-name> <package-name>
$ guix package --list-profiles
# Default profile
$ refresh_guix
# A given profile
$ refresh_guix <profile_name>
‘profile_name’ must be an existing directory in $GUIX_USER_PROFILE_DIR.
Guix uses a git repository to handle all the packages. To keep your guix install up-to-date and have access to the last versions of the softwares and libraries, you must run regularly the following command:
$ guix pull
# --> pull from git repository
and if necessary, update all the packages of your profile(s)
guix package -u
# update all the packages of the default profile to their last available version.
# or for a specific profile:
guix package -p $GUIX_USER_PROFILE_DIR/<profile_name> -u
You must be very cautious when running update (-u). Since all packages will be upgraded, you must be sure that you really need the last versions of all libraries.
For some reasons (test, ‘local’ guix recipes …) you may want to add another reference repository (a channel) for guix. There you’ll be able to write ‘recipes’ (scm files) that will be used by guix to install some specific packages.
For instance you can easily create a new project on https://gricad-gitlab.univ-grenoble-alpes.fr to store your recipes. The project must be public.
To have it taken into account on Gricad clusters, do the following:
.config/guix/channels.scm;; Add my personal packages to those Guix provides.
(cons (channel
(name 'some_name)
(url "some_url"))
%default-channels)
some_url being the address of your repo.
guix pull
Détails : https://www.gnu.org/software/guix/manual/en/html_node/Channels.html#Channels
Check for example the ljk-contribs channel
with the channels.scm file:
;; Add my personal packages to those Guix provides.
(cons (channel
(name 'gricad-guix-packages)
(url "https://gricad-gitlab.univ-grenoble-alpes.fr/bouttiep/gricad_guix_packages.git"))
%default-channels)
And then, notice that a pull will update two channels : yours and the standard one.
$ guix pull
Updating channel 'gricad-guix-packages' from Git repository at 'https://gricad-gitlab.univ-grenoble-alpes.fr/bouttiep/gricad_guix_packages.git'...
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Something’s wrong with your guix? Too many profiles? Whatever the reason is, if you want to remove your whole guix configuration and restart from a clean install, just run:
$ guix_remove_user_conf
and then, to re-create proprely your environment, the usual command will do the job:
$ source /applis/site/guix-start.sh