Guix, another package manager

Table of content

What is Guix ?

Guix is a package manager, namely a collection of software tools that automates the process of installing, upgrading, configuring, and removing softwares on the operating system.

References :

Why should you use Guix on GRICAD clusters ?

Well, since working on clusters implies that you’ll have to share the resources with many other users, each of them with different needs and contexts of use, using a tool to handle the installation of specific softwares is compulsory.

Guix is just one of the candidates to answer this requirement, among others like Nix, container systems or modules.

  • Guix will allow you to build your own environments in a very flexible way without the help of administrators of the cluster (i.e. without being root).
  • Guix will allow you to build different working contexts (or “profiles”) and to switch easily between them.
  • Guix will allow you to write your own “recipe” for your software, in order to provide other users with a very simple way to install and use your own library.
  • Thanks to Guix and the concept of profiles, you can install softwares available on all nodes of both Luke and Dahu, in one shot.

Convinced? Let’s try it and see some examples.

Guix short tutorial

Through this tutorial, you will learn Guix basics, good practices and how to setup properly your environments. This might probably be enough for the needs of most of GRICAD users.

If you’re in a hurry you can directly step to Guix cheat sheet.

As examples, let us consider these two use cases:

  • install and use a specific version of Python with some extra packages.
  • install GNU compilers and fftw library

Through this tutorial, we will detail all the actions required to setup these two ‘working environments’. You are encouraged to try and run all the commands below.

Start guix session

First of all, any session where guix is required must start with the following command:

$ source /applis/site/guix-start.sh

It brings ‘guix’ and other commands into the current environment and ensures that everything is properly set for Guix usage.

Do not forget to add this line to your oar scripts!

Do not add this line into your bashrc, profile or alike, you will end with some unexpected side effects.

At first call of ‘guix-start.sh’ on a cluster some background actions will occur, mostly to install and update guix on your account. This may take some time, please be patient …

Look for some packages

Ok, assuming that we need Python3, with numpy and matplotlib and the emacs text editor let us have a look at what is available and search for some of the required packages, for example

$ guix search emacs
name: emacs
version: 26.3
outputs: out
systems: x86_64-linux i686-linux
dependencies: acl@2.2.53 alsa-lib@1.2.2 dbus@1.12.16 giflib@5.2.1 gnutls@3.6.12 gtk+@3.24.20 imagemagick@6.9.11-11 libice@1.0.10 libjpeg-turbo@2.0.4
+ libotf@0.9.16 libpng@1.6.37 librsvg@2.40.21 libsm@1.2.3 libtiff@4.1.0 libx11@1.6.9 libxft@2.3.3 libxml2@2.9.10 libxpm@3.5.13 m17n-lib@1.8.0
+ mailutils@3.9 ncurses@6.2 pkg-config@0.29.2 texinfo@6.7 zlib@1.2.11
location: gnu/packages/emacs.scm:76:2
homepage: https://www.gnu.org/software/emacs/
license: GPL 3+
synopsis: The extensible, customizable, self-documenting text editor
description: GNU Emacs is an extensible and highly customizable text editor.  It is based on an Emacs Lisp interpreter with extensions for text
+ editing.  Emacs has been extended in essentially all areas of computing, giving rise to a vast array of packages supporting, e.g., email, IRC and
+ XMPP messaging, spreadsheets, remote server editing, and much more.  Emacs includes extensive documentation on all aspects of the system, from basic
+ editing to writing large Lisp programs.  It has full Unicode support for nearly all human languages.
relevance: 33
...

Depending on the searched items, the output might be quite long. Using ‘grep’ or regexp might be a solution but you’ll do better to check directly in the online list of guix packages.

There you can find the following packages : python, python-numpy and python-matplotlib

Install things

Once the required packages names are clear, the installation process can start:

$ guix install emacs python@3 python-numpy python-matplotlib
... # MIGHT BE QUITE LONG ...

Note the @3 to specify the required version.

That’s it, python3 is available for you on all nodes, you can try it on any node of the cluster:

$ python3
Python 3.8.2 (default, Jan  1 1970, 00:00:01)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>import numpy as np
$ which python3
/home/perignon/.guix-profile//bin/python3

Before anything else, do not forget to source /applis/site/guix-start.sh for each new terminal session! Else, your guix environment won’t be available.

There, you certainly noticed that python has been installed in $HOME/.guix-profile, as all the other packages installed above. $HOME/.guix-profile is the default place where guix will install everything, with the standard organisation on unix systems, bin for binaries, lib for libraries and so on.

$ ls $HOME/.guix-profile/
bin  etc  include  lib  libexec  manifest  sbin  share
$ ls $HOME/.guix-profile/lib
...
libpython3.so
...

$ ls $HOME/.guix-profile/bin
...
python3 emacs
...

Indeed, your .guix-profile contains only symbolic links to files from /gnu/store.

Either somebody has already installed the very same package on the cluster or you’re the first. In the first case, Guix just creates the links in your profile path. In the latter case, Guix first installs thing into /gnu/store and then creates the links in your profile path.

Check what is currently installed

The following command will list all packages installed into the default profile

$ guix package -I
glibc-locales	2.29	out	/gnu/store/03nvilh2x4z07dxv7h13gh986vvgpnsf-glibc-locales-2.29
hello	2.10	out	/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10
emacs	26.3	out	/gnu/store/lza7sccqgrv313gbrb9j8f7lgn49irdn-emacs-26.3
python	3.8.2	out	/gnu/store/fa1zp10w2sb7gdzk1fwqhr106mpq3lc7-python-3.8.2
python-numpy	1.17.3	out	/gnu/store/rr1winibssxgxsf49f6rvdfdigifpviz-python-numpy-1.17.3
python-matplotlib	3.1.2	out	/gnu/store/gsjhzr1xy96hz8wvaa8mblvv2niqnkwy-python-matplotlib-3.1.2

Notice glibc-locales and hello, always installed by default, and the others packages you asked for.

Let us step to the second use case, a working environment with gnu compilers and the fftw library.

Dealing with specific working environments, profiles

Install packages into a profile

The point is to install the GNU compilers and the library in a specific place. Compilers are usually available in the package ‘gcc-toolchain’ and guix search will tell you the name of the fftw package.

Let’s install them in a new profile called ‘gcc-fftw’. Dealing with profiles in guix is quite simple and handled with the option ‘-p’ on most of guix commands. This option tells guix to use a specific profile (which is no more than a path where things must be saved) and must be followed by the full (absolute) path to the profile. To simplify this, a specific environment variable is set by default for all users, GUIX_USER_PROFILE_DIR, and must be used:

$ guix install -p $GUIX_USER_PROFILE_DIR/gcc_fftw gcc-toolchain fftw

The following packages will be installed:
   gcc-toolchain 10.1.0
   fftw          3.3.8
...

The environment variable GUIX_USER_PROFILE_DIR is the place where all guix profiles must be saved. This is compulsory to ensure the availability of softwares on both Luke and Dahu, among other things.

$ echo $GUIX_USER_PROFILE_DIR
/var/guix/profiles/per-user/perignon

$ ls $GUIX_USER_PROFILE_DIR
...
current-guix gcc_fftw
...

Ok, now we can check the list of installed packages in the default profile,

$ guix package -I
glibc-locales	2.29	out	/gnu/store/03nvilh2x4z07dxv7h13gh986vvgpnsf-glibc-locales-2.29
hello	2.10	out	/gnu/store/a462kby1q51ndvxdv3b6p0rsixxrgx1h-hello-2.10
emacs	26.3	out	/gnu/store/lza7sccqgrv313gbrb9j8f7lgn49irdn-emacs-26.3
python	3.8.2	out	/gnu/store/fa1zp10w2sb7gdzk1fwqhr106mpq3lc7-python-3.8.2
python-numpy	1.17.3	out	/gnu/store/rr1winibssxgxsf49f6rvdfdigifpviz-python-numpy-1.17.3
python-matplotlib	3.1.2	out	/gnu/store/gsjhzr1xy96hz8wvaa8mblvv2niqnkwy-python-matplotlib-3.1.2
$

and in the new profile, gcc_fftw

$ guix package -p $GUIX_USER_PROFILE_DIR/gcc_fftw -I
gcc-toolchain	10.1.0	out	/gnu/store/q9vv9s1jj7hw5igckncysadn6gmccfxm-gcc-toolchain-10.1.0
fftw	3.3.8	out	/gnu/store/85b70gpbs9issc1gbdcvnwb8pip6ihlk-fftw-3.3.8

And the default compiler …

$ which gcc
/usr/bin/gcc
$ gcc --version
gcc (Debian 8.3.0-6) 8.3.0

is still the one installed by default on the system, which is not what we want!

The path to the new guix profile (and so gcc) must be added to environment variable. To handle this, we provide a function to activate or refresh a specific profile:

$ refresh_guix gcc_fftw
Activate profile  /var/guix/profiles/per-user/stagiaire11/gcc_fftw
The following packages are currently installed in /var/guix/profiles/per-user/stagiaire11/gcc_fftw:
gcc-toolchain	10.1.0	out	/gnu/store/q9vv9s1jj7hw5igckncysadn6gmccfxm-gcc-toolchain-10.1.0
fftw	3.3.8	out	/gnu/store/85b70gpbs9issc1gbdcvnwb8pip6ihlk-fftw-3.3.8

$ which gcc
/var/guix/profiles/per-user/stagiaire11/gcc_fftw/bin/gcc/usr/bin/gcc
$ gcc --version
gcc (GCC) 10.1.0

which is what we expect!

Switch between profiles

Assume now that we need another version of the gnu compilers, but without removing the current one.

Doing this is very easy with guix, in a new profile, e.g. gcc9_fftw

# install and activate the profile 
$ guix install -p $GUIX_USER_PROFILE_DIR/gcc9_fftw gcc-toolchain@9 fftw
...
$ refresh_guix gcc9_fftw
Activate profile  /var/guix/profiles/per-user/stagiaire11/gcc9_fftw
The following packages are currently installed in /var/guix/profiles/per-user/stagiaire11/gcc9_fftw:
gcc-toolchain	9.3.0	out	/gnu/store/4s25r2krb2x5c904gbzv0jkj2fyi5hnx-gcc-toolchain-9.3.0
fftw	3.3.8	out	/gnu/store/85b70gpbs9issc1gbdcvnwb8pip6ihlk-fftw-3.3.8

$ gcc --version
gcc (GCC) 9.3.0

# and switch back easily to the other profile:
$ refresh_guix gcc_fftw
Activate profile  /var/guix/profiles/per-user/stagiaire11/gcc_fftw
The following packages are currently installed in /var/guix/profiles/per-user/stagiaire11/gcc_fftw:
gcc-toolchain	10.1.0	out	/gnu/store/q9vv9s1jj7hw5igckncysadn6gmccfxm-gcc-toolchain-10.1.0
fftw	3.3.8	out	/gnu/store/85b70gpbs9issc1gbdcvnwb8pip6ihlk-fftw-3.3.8
$ gcc --version
gcc (GCC) 10.1.0
Listing all your profiles

Finally, you can easily have a list of all your current profiles with

$ guix package --list-profiles
/home/perignon/.config/guix/current
/var/guix/profiles/per-user/perignon/gcc9_fftw
/var/guix/profiles/per-user/perignon/gcc_fftw
/home/perignon/.guix-profile

Notice the current and guix-profile directories, which must not be removed.

current is the place where guix is installed and guix-profile is your default profile.

Removing a package or a profile

To remove a package from a profile, just try

$ guix remove -p  $GUIX_USER_PROFILE_DIR/gcc9_fftw fftw

# or from default profile
$ guix remove emacs

and to remove a profile, simply remove the concerned directory, e.g.:

$ rm -rf $GUIX_USER_PROFILE_DIR/gcc9_fftw

Guix cheat sheet, advices and good practices

The tutorial is over and you must now be able to use guix to create and manage your own working environments.

Here is a summary of the basic things and commands to know.

Start a guix session

This should be run at the beginning of any terminal session where Guix is required.

$ source /applis/site/guix-start.sh

Look for a package

guix search <name>

Very verbose … It might be easier to check packages online : https://www.gnu.org/software/guix/packages/

Installing and removing packages into profiles

We strongly recommend you to properly define and separate your working environments by using guix profiles.

# In/from default profile

$ guix install <package-name>
$ guix remove <package-name>

# In/from a given profile

$ guix install -p $GUIX_USER_PROFILE_DIR/<profile-name> <package-name>

$ guix remove -p $GUIX_USER_PROFILE_DIR/<profile-name> <package-name>


List your existing profiles

$ guix package --list-profiles

Activate a profile and list its content

# Default profile
$ refresh_guix

# A given profile
$ refresh_guix <profile_name>

‘profile_name’ must be an existing directory in $GUIX_USER_PROFILE_DIR.

Maintenance commands

Keep your repository up to date

Guix uses a git repository to handle all the packages. To keep your guix install up-to-date and have access to the last versions of the softwares and libraries, you must run regularly the following command

$ guix pull
# --> pull from git repository

and if necessary, update all the packages of your profile(s)

guix package -u
# update all the packages of the default profile to their last available version.

# or for a specific profile:

guix package -p $GUIX_USER_PROFILE_DIR/<profile_name> -u

You must be very cautious when running update (-u). Since all packages will be upgraded, you must be sure that you really need the last versions of all libraries.

Add a new channel (i.e. a reference repository to pull guix recipes)

For some reasons (test, ‘local’ guix recipes …) you may want to add another reference repository (a channel) for guix. There you’ll be able to write ‘recipes’ (scm files) that will be used by guix to install some specific packages.

For instance you can easily create a new project on https://gricad-gitlab.univ-grenoble-alpes.fr, that must be public.

To take it into account on Gricad clusters, do the following:

  1. on Luke AND Dahu, edit a file .config/guix/channels.scm
;; Add my personal packages to those Guix provides.
(cons (channel
      (name 'some_name) 
      (url "some_url"))
     %default-channels)

some_url being the address of your repo.

  1. Run on both Luke and Dahu
guix pull

Détails : https://www.gnu.org/software/guix/manual/en/html_node/Channels.html#Channels

Check for example the ljk-contribs channel

with the channels.scm file

;; Add my personal packages to those Guix provides.
(cons (channel
      (name 'ljk-guix) 
      (url "https://gricad-gitlab.univ-grenoble-alpes.fr/scalde-contrib/ljk-guix-channel"))
     %default-channels)

And then, notice that a pull will update two channels : yours and the standard one.

$ guix pull
Updating channel 'ljk-guix' from Git repository at 'https://gricad-gitlab.univ-grenoble-alpes.fr/scalde-contrib/ljk-guix-channel.git'...
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...

Restart your guix conf from scratch

Something’s wrong with your guix? Too many profiles? Whatever the reason is, if you want to remove your whole guix configuration and restart from a clean install, just run:

$ guix_remove_user_conf

and then, to re-create proprely your environment, the usual command will do the job:

$ source /applis/site/guix-start.sh