Utiliser le gestionnaire de paquets NIX sur les serveurs de calcul GRICAD

Ressources externes

Introduction

NixOS is a Linux distribution that allows local packages installation and reproducible builds. The packages (NixPkgs) can run on top of any Linux or MacOS distribution and allows a simple user, with no privileges, to install packages of it’s choice. It is specifically interesting in an HPC context where users need a lot of different libraries, sometimes of different versions. Nix also offers a way to develop into an isolated and reproducible environment. Those environment or resulting packages can be easily and efficiently shared between users.

Check this quickstart guide to setup Nix on your own Linux or MacOS computer, and you’ll be able to share the same environment between your personnal computer and GRICAD’s hosts, a very interesting feature for developping!

Nix principles

In a few words, Nix is a package manager using the nix functional language to write nix expressions to build a derivation that generally results in a package instance.

  • Nix is the package manager
  • Nixpkgs is the repository containing all of the nix expressions
  • NixOS is the operating system using nixpkgs (but NixOS is not mandatory and we do not use this OS on the GRICAD computing hosts)

Nix stores packages in the Nix store, usually the directory /nix/store, where each package has its own unique subdirectory containing a hash in its name, such as

/nix/store/b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z-firefox-33.1

All the dependencies of the packages are linked into the binaries (heavy use of the runpath/RPATH). The users may directly use a binary from its path or install the package into it’s nix profile which is pointed by the PATH environement variable. When a package is installed into a user profile, it is actually just linked with a bunch of symbolic links into the home directory.

Warnings

  • Please, don’t use NIX with any other environment (it means, do not load at the same time Nix and /applis/ciment/v2/env.bash). In a complete Nix context, you will not need the “module” command anymore.
  • Packages installations (nix-env operations) may only be made from login nodes (not from computing nodes). Once installed, packages are available from all the nodes of course.
  • You will probably find old docs talking about the “nix-channel” command. Please use the NIX_PATH environment variable instead.

References

The current guide is just a focus on examples and specific configuration of the GRICAD clusters. You’ll find much more about Nix usage on the following links:

Nix activation

source /applis/site/nix.sh

(You can put this line into your ~/.bashrc file or at the beginning of your job scripts.)

This creates a set of variables and configures the PATH variable to point to your nix profile:

  • $PATH : ~/nix-profile/bin is prepended
  • $NIX_PATH : current nix channel (version of the nixpkgs repository)
  • $NIX_USER_PROFILE_DIR : the root path for your profiles. WARNING: creating a profile out of this directory is allowed, but the garbage collector is not aware of the installed packages into this profile, so it may remove them!

On the first run, this also initiates the following files and directories:

  • ~/.nix-profile : a link to your current profile
  • ~/.config/nixpkgs/config.nix : a pre-configured nix environment provided by GRICAD, to ease the customization of some tools like python or R.
  • .config/nix_channel : provided and used by the nix.sh script itself, this stores the current used channel, that will be used to configure the NIX_PATH variable

This script may also be sourced with an option to ask for using another channel beetween a set of recommended channels by GRICAD (this will be persistent):

   --unstable : Activate the latest (but unstable) channel
   --stable: Activate the stable channel                  
   --old: Activate the old obsoleted channel              

Do echo $NIX_PATH to get the current channel of your environment.

Profiles

A profile is simply a directory containing symbolic links to a set of packages.

  • Create (or switch to) a user profile:
$ nix-env --switch-profile $NIX_USER_PROFILE_DIR/test_profile

It’s then possible to install packages into this profile

  • Getting the current profile:
$ ls -l ~/.nix-profile
  • Your path also contains the default nix profile that may provide system wide tools, for example nix commands, the current iRods commands (ils, iput, iget,…)
$ which ils
/nix/var/nix/profiles/default/bin/ils

Packages installation

Basic usage

  • Search for a package (example with the hello package)
$ nix search hello

You can also search for packages from the online web form: https://search.nixos.org/packages

  • Install the hello package
$ nix-env -i -A hello                                          
installing 'hello-2.10'                                                       
building '/nix/store/zzvhm5rvqv5jh4c1zh049027qcbqgjc3-user-environment.drv'...
created 2 symlinks in user environment
bzizou@f-dahu:~$ NIX_PATH=nixpkgs=channel:nixos-17.09 nix-env -i -A ponysay                                                             
unpacking 'https://nixos.org/channels/nixos-17.09/nixexprs.tar.xz'...
installing 'ponysay-3.0.2'
these paths will be fetched (19.93 MiB download, 104.55 MiB unpacked):
  /nix/store/0h1myzfdpl0c4wk2ri02imr588bhl1x3-expat-2.2.4
  /nix/store/1gg3bzzlw7a6m4fqjf5fdv92nzas1n3l-python3-3.6.4
  /nix/store/34vrcmdwar08x75k3khr8q7j8klc70ww-openssl-1.0.2o
  /nix/store/39nrz4ky69qxzciss59h9izx04lz4r13-ponysay-3.0.2
  /nix/store/44pga8rb4ldvijrk853mxf4hqib089wn-sqlite-3.21.0
  /nix/store/4bh7qhnrjk8cgd1bh7nr0glbga0aasdx-coreutils-8.28
  /nix/store/609zdpfi5kpz2c7mbjcqjmpb4sd2y3j4-ncurses-6.0-20170902
  /nix/store/961ykmpzq89jwsg49lc4nw2zjacvrlmr-libffi-3.2.1
  /nix/store/a18nnq9b1vyh9f7f71w5lmip91cqr1px-gdbm-1.13
  /nix/store/ac082jcsg31763mbgiqlirhgsygimn1x-attr-2.4.47
  /nix/store/af9085bk7pgdyqvgkj575ymzm9pajy41-readline-6.3p08
  /nix/store/bqd0pk2ryvw0b47r7k23k13jx8ih2165-bzip2-1.0.6.0.1
  /nix/store/d54amiggq6bw23jw6mdsgamvs6v1g3bh-glibc-2.25-123
  /nix/store/dl66jvc19im9lqslzqszsp2gif2y0y8i-zlib-1.2.11
  /nix/store/jgw8hxx7wzkyhb2dr9hwsd9h2caaasdc-bash-4.4-p12
  /nix/store/mcsl18halr6lx9b4d35ghrx71v7mw2k9-acl-2.2.52
  /nix/store/y4p3gg22jjr223r2r681nmfdfvj245lk-xz-5.2.3
copying path '/nix/store/d54amiggq6bw23jw6mdsgamvs6v1g3bh-glibc-2.25-123' from 'https://cache.nixos.org'...
copying path '/nix/store/ac082jcsg31763mbgiqlirhgsygimn1x-attr-2.4.47' from 'https://cache.nixos.org'...
copying path '/nix/store/jgw8hxx7wzkyhb2dr9hwsd9h2caaasdc-bash-4.4-p12' from 'https://cache.nixos.org'...
copying path '/nix/store/mcsl18halr6lx9b4d35ghrx71v7mw2k9-acl-2.2.52' from 'https://cache.nixos.org'...
copying path '/nix/store/bqd0pk2ryvw0b47r7k23k13jx8ih2165-bzip2-1.0.6.0.1' from 'https://cache.nixos.org'...
copying path '/nix/store/4bh7qhnrjk8cgd1bh7nr0glbga0aasdx-coreutils-8.28' from 'https://cache.nixos.org'...
copying path '/nix/store/0h1myzfdpl0c4wk2ri02imr588bhl1x3-expat-2.2.4' from 'https://cache.nixos.org'...
copying path '/nix/store/a18nnq9b1vyh9f7f71w5lmip91cqr1px-gdbm-1.13' from 'https://cache.nixos.org'...
copying path '/nix/store/961ykmpzq89jwsg49lc4nw2zjacvrlmr-libffi-3.2.1' from 'https://cache.nixos.org'...
copying path '/nix/store/609zdpfi5kpz2c7mbjcqjmpb4sd2y3j4-ncurses-6.0-20170902' from 'https://cache.nixos.org'...
copying path '/nix/store/34vrcmdwar08x75k3khr8q7j8klc70ww-openssl-1.0.2o' from 'https://cache.nixos.org'...
copying path '/nix/store/af9085bk7pgdyqvgkj575ymzm9pajy41-readline-6.3p08' from 'https://cache.nixos.org'...
copying path '/nix/store/44pga8rb4ldvijrk853mxf4hqib089wn-sqlite-3.21.0' from 'https://cache.nixos.org'...
copying path '/nix/store/y4p3gg22jjr223r2r681nmfdfvj245lk-xz-5.2.3' from 'https://cache.nixos.org'...
copying path '/nix/store/dl66jvc19im9lqslzqszsp2gif2y0y8i-zlib-1.2.11' from 'https://cache.nixos.org'...
copying path '/nix/store/1gg3bzzlw7a6m4fqjf5fdv92nzas1n3l-python3-3.6.4' from 'https://cache.nixos.org'...
copying path '/nix/store/39nrz4ky69qxzciss59h9izx04lz4r13-ponysay-3.0.2' from 'https://cache.nixos.org'...
building '/nix/store/4jdz5ybryp0yp2iy1xynagx465pdf9pi-user-environment.drv'...
created 20 symlinks in user environment

Note that the channel (all of the available nixpkgs expressions!) is downloaded and unpacked if not already done, into a local cache. Then, if the package and the dependencies are not already available into the nix store, they are downloaded. It may happen that a package is not available as a binary package; if this happens, it will be automatically compiled!

  • Listing installed packages in the current profile
$ nix-env -q 
hello-2.10                  
ponysay-3.0.2
  • Comparing versions beetween installed and available on the channel pointed by $NIX_PATH
echo $NIX_PATH
nixpkgs=channel:nixos-21.05
$ nix-env -qc 
hello-2.10     = 2.10        
ponysay-3.0.2  < 3.0.3       
  • Upgrade all packages of the current profile
$ nix-env --upgrade
upgrading 'ponysay-3.0.2' to 'ponysay-3.0.3'
these paths will be fetched (0.79 MiB download, 10.66 MiB unpacked):
  /nix/store/kr5z7lqyafnr6nxfv5lw7vjk1n11gdaq-ponysay-3.0.3
copying path '/nix/store/kr5z7lqyafnr6nxfv5lw7vjk1n11gdaq-ponysay-3.0.3' from 'https://cache.nixos.org'...
building '/nix/store/53ymb0vxcvypgqg30yi9i1mw81583m8n-user-environment.drv'...
created 20 symlinks in user environment
  • The upgraded ponysay version does not work as expected? We can revert the upgrade by doing a rollback:
$ nix-env --rollback 
  • Remove a package
$ nix-env -e hello

The Gricad installation of nix makes use of the $NIX_PATH environment variable to fix the release of the packages repository to use, with for example the value NIX_PATH=nixpkgs=channel:nixos-22.05. The nix-env command may behave differently when used from the inside of a non-interactive script and it may not use the NIX_PATH variable by default. In that case, it may be necessary to add the -f '<nixpkgs>' directive to force nix to use the $NIX_PATH variable, for example: nix-env -f '<nixpkgs>' -qc.

Packages from a NUR repository

NUR is a meta-repository of users repositories, allowing the users to share custom packages that are out of the Nixpkgs tree. GRICAD provides it’s own NUR repository that is pre-configured into your Nix environment. It generally contains pre-release of packages that are currently tested before official integration into Nixpkgs, or packages with specific local customizations. It’s also the place where we provide packages that are not distrubable such as Intel compilers. For this kind of packages, the sources are available only localy and the build/installation of the package is not possible outside of GRICAD.

List the available NUR/GRICAD packages:

$ nix-env -qaP -A nur.repos.gricad

Install a NUR/GRICAD package:

$ nix-env -iA nur.repos.obitools3

Contributing to GRICAD’s NUR packages is possible. Simply do pull requests here: https://github.com/Gricad/nur-packages

Applications vs Libraries

It’s important to note that a nix package may be an application (a set of runtime binaries) or some libraries. But inside a profile, you should only install end userapplications. For example, it’s generally not a good idea to install a compiler or a development library. If you need to compile a programm, you’ll probably need to create a new package or at least use nix-shell (see later).

  • For packages of interpreters like python or R, please go to section “Specific packages”

  • If you need to compile software, don’t try to install compilers at this stage. Please, check the “Compiling…” sections

About the versions of the packages

This section may be skipped as it is not mandatory to understand for a basic usage of nix on the GRICAD clusters.

As seen earlier, channels are stable (excepted for the unstable channel) versions of the nixpkgs repository. It allows to easily get an older version of a package. Once installed, a package has no link to the original channel, but it stays into the store unless it is not used by any profile. It means that you can install several versions of the same package into different profiles and you can switch to any version, even if the package does not come from a stable channel. As an example, it is even possible to install a package by calling directly it’s path in the store. This might be something that you saved into a text file or given by another user that wants you to test the exactly same version of binaries:

$ readlink $(which ponysay)
/nix/store/39nrz4ky69qxzciss59h9izx04lz4r13-ponysay-3.0.2/bin/ponysay

## Another user or you in another profile:
$ nix-env --switch-profile $NIX_USER_PROFILE_DIR/ponysay_ok
$ nix-env -i /nix/store/39nrz4ky69qxzciss59h9izx04lz4r13-ponysay-3.0.2

Or you can directly call the binary of a specific version

$ /nix/store/39nrz4ky69qxzciss59h9izx04lz4r13-ponysay-3.0.2/bin/ponysay --help

Nix flakes (advanced usage)

Flakes are an up-coming feature, still experimental, but that you can try. It aims at reproducibility, composability and usability in the Nix ecosystem.

To enable it:

$ nix-env -iA nixFlakes
$ echo "experimental-features = nix-command flakes" > ~/.config/nix/nix.conf 
$ export PATH=~/.nix-profile/bin:$PATH

A good introduction to flakes: https://www.tweag.io/blog/2020-05-25-flakes

Python packages

Nix provides Python and a lot of Python modules as packages. But due to dependencies, a python module can’t be installed standalone, and have to be installed into an environned Python setup. The GRICAD Nix environment comes with a pre-configured file to ease that.

First, you have to edit the nixpkgs config file to customize your python packages list:

$ vi ~/.config/nixpkgs/config.nix

Find, in this file, the section starting with "# Python environment" and customize the list of packages:

    # Python environment
    pythonEnv = pkgs.python3.withPackages (ps: with ps; [

          #################################################
          # You can list your python packages below and
          # install a python environement with:
          #      nix-env -f "<nixpkgs>" -iA pythonEnv
          #################################################
          numpy ipython virtualenv pip notebook

Then, do the installation of the environed python into your profile with the suggested cmd:

$ nix-env -f "<nixpkgs>" -iA pythonEnv

You should see python-env as a package into your profile:

$ nix-env -q
python3-3.8.11-env

If you need a specific python version, you can change the pythonEnv = line, for example:

## Latest Python 2:
    pythonEnv = pkgs.python2.withPackages (ps: with ps; [
## Python 3.7:
    pythonEnv = pkgs.python37.withPackages (ps: with ps; [

Installing python libraries not available through Nix

If you need extra python packages that are not available in Nix, you may have to create a virtualenv and use pip inside. Check that you have installed a python-env (section above) with pip in the packages list. Then:

  • Create a directory for your virtual python environments:
mkdir virtualenv
cd virtualenv
  • Create a ‘test’ virtualenv
$ python -m venv test
  • Load this virtualenv (do this also for runtime)
$ source test/bin/activate
  • Then install a package with pip
pip install <some_package>
  • If you have problems at runtime with some dynamics libraries not found, you can try to export the cc library environment before running your python application:
$ export LD_LIBRARY_PATH=$(nix eval --raw nixpkgs.stdenv.cc.cc.lib)/lib

Or with nix >= 2.8 :

$ export LD_LIBRARY_PATH=$(nix --extra-experimental-features nix-command eval --impure --raw --expr 'with import <nixpkgs> {}; stdenv.cc.cc.lib')/lib
  • You may also have to provide some build dependencies. Here’s an example with HDF5 and the colmet package:
$ python -m venv colmet
$ source colmet/bin/activate
$ nix-env -iA hdf5
$ export HDF5_DIR=~.nix-profile
$ pip install colmet
## If your python module does not work with an environment variable, you can specify the external dependency installation PATH through a pip option as follows:
$ pip install --global-option=build_ext --global-option="-I/$HOME/.nix-profile/include" --global-option="-L/$HOME/.nix-profile/lib" colmet

R packages

The official documentation contains an R section: https://nixos.org/manual/nixpkgs/stable/#r

CRAN modules

All the CRAN modules are available into the Nixpkgs distribution as packages. You can check the list of available R libraries with the following command:

$ nix-env -qaP -A rPackages

Don’t install directly R modules as R packages into your Nix environment. They’re not usable this way, but must be installed wrapped to be able to find each other. Your GRICAD’s environment is pre-configured with a sample environned R override. You can customize it before installing rEnv:

## Edit .config/nixpkgs/config.nix and got to the "# R environnement" section
$ vi .config/nixpkgs/config.nix
## Add the list of needed modules, with the name provided in the "-qaP -A rPackages" query
## For example:
   packageOverrides = pkgs: {                                
                                                             
     # R environnement                                       
     rEnv = pkgs.rWrapper.override {                         
       packages = with pkgs.rPackages; [                     
                                                             
           ################################################# 
           # You can list your R packages below and intall   
           # an R environement with:                         
           #      nix-env -f "<nixpkgs>" -iA rEnv            
           ################################################# 
           optimx                                            
           snow                                              
           phylobase                                         
           devtools                                          
       ];                                                    
     };                                                      


## Then install your environned R interpreter:
$ nix-env -iA rEnv

R modules from github

In the above section example, you can add a module from github with buildRPackage (example with lwgeom):

   packageOverrides = pkgs: {                                
                                                             
     # R environnement                                       
     rEnv = pkgs.rWrapper.override {                         
       packages = with pkgs.rPackages; [                     
                                                             
           ################################################# 
           # You can list your R packages below and intall   
           # an R environement with:                         
           #      nix-env -f "<nixpkgs>" -iA rEnv            
           ################################################# 
           optimx                                            
           snow                                              
           phylobase                                         
           devtools
            (buildRPackage {
               name = "lwgeom";
               src = pkgs.fetchFromGitHub {
                 owner = "r-spatial";
                 repo = "lwgeom";
                 rev = "be10a443b17f361b73eae07ec11bbf31aa49fc87";
                 sha256 = "0nx8l8iqnjgv97rh9qd6z8zwh1b2l5w3fjk9xl7h6j1130pci0x0";
               };
            propagatedBuildInputs = with pkgs ; [ sqlite proj Rcpp units sf pkgconfig geos ];
            })
       ];
     };

## Then install your environned R interpreter:
$ nix-env -iA rEnv

The needed R library will be automatically built!

Installing a R module locally

  • Create a directory into your home, where you will store the local modules, for example:
$ mkdir ~/my_r_libraries
  • Install the module:
$ R CMD INSTALL --library=~/my_r_libraries ~/tmp/my_module.tgz

or with devtools:

R
> library(devtools)
> devtools::install_github(repo="nmatzke/BioGeoBEARS", INSTALL_opts="--byte-compile", dependencies=FALSE, lib="~/my_r_libraries")
  • Then, to load the module (runtime):
R
> .libPaths( c( .libPaths(), "~/my_r_libraries/") )
> library(my_module)

Nix-shell

Basic usage

Nix-shell allows to enter into the environment of one or several packages. It is generally only available from cluster frontends but some hosts may have a re-routing configuration allowing it’s use also from computing nodes (it’s the case on the Dahu cluster)

There’s no need to install a package into a nix profile to use nix-shell. That makes it usefull to run applications directly from Nix, withtout having to install it.

Examples:

## Enter into the shell of the package 'cowsay'
$ nix-shell -p cowsay
[nix-shell:~]$ echo $buildInputs
/nix/store/cr23xzsbbvjgxaf1mchn2drhh6yvsq0q-cowsay-3.03+dfsg2

## Launch cowsay from a nix-shell, without installing the package
$ nix-shell -p cowsay --run "cowsay hello"

Empowering your environement using nix-shell

Nix-shell is actually a very efficient way to create a reproducible environment. You can create a default.nix file containing the complete description of what to install into your environment, with for example a pinned nixpkgs release, customized python modules, development libraries,…

Here’s a commented example of a default.nix file. This file can be placed into a directory of your project, and you just have to type nix-shell <ENTER> to build and enter into this environment:

# `default.nix` file for a reproducible environment containing
# python 3.8 and the opencv4 module with gtk support
#
# Pin the nixpkgs version
# 
let
  hostPkgs = import <nixpkgs> {};
  pinnedPkgs = hostPkgs.fetchFromGitHub {
    owner = "NixOS";
    repo = "nixpkgs";
    rev = "22.05";
    sha256 = "1ckzhh24mgz6jd1xhfgx0i9mijk6xjqxwsshnvq789xsavrmsc36";
  };
in
# Add some customizations
let
  # Allow unfree packages
  pkgs = (import pinnedPkgs) {
  config = rec {
    allowUnfree = true;
    };
  };
  # Opencv is provided with no Gtk support by default, into nixpkgs
  # so, we create here an override to enable Gtk into our opencv environment
  opencvGtk = pkgs.python3.pkgs.opencv4.override (old : { enableGtk2 = true; });

# Create the shell
in with pkgs;
let
in mkShell rec {
  name = "python3shell";
  libPath = with pkgs ; lib.makeLibraryPath [
                          stdenv.cc.cc
                          zlib
                        ];
  # Here, we list some needed python modules, especialy our
  # overrided opencv one (opencvGtk)
  buildInputs = with python3.pkgs; [
    pip
    scipy
    matplotlib
    h5py
    opencvGtk
  ] ++ [ lsb-release ];

  # Shell hooks are here to help you customize your environment variables, alises,...
  # We even provide a way to install pip modules into the project directory
  shellHook = ''
    export LD_LIBRARY_PATH=${libPath}:$LD_LIBRARY_PATH
    alias pip="PIP_PREFIX='$(pwd)/_build/pip_packages' TMPDIR='$HOME' \pip"
    export PYTHONPATH="$(pwd)/_build/pip_packages/lib/python3.8/site-packages:$PYTHONPATH"
    export PATH="$(pwd)/_build/pip_packages/bin:$PATH"
    unset SOURCE_DATE_EPOCH
  '';
}

You can find more custom nix-shell envs here: https://github.com/bzizou/sysadmin/tree/master/nix_environments

For more about nix-shell, check the documentation

Compiling and using Openmpi programs (with GCC)

Compilation

The best way to compile an openmpi programm that is not already packaged is by making a new package. But in some cases, you can also try to do it in a more direct (but no so reproducible) way by using nix-shell:

$ nix-shell -p openmpi -p gcc
[nix-shell:~]$ mpicc -o hello_mpi hello_mpi.c
[nix-shell:~]$ exit

Runtime

For runtime, you can install openmpi into your nix profile:

$ nix-env -i -A openmpi

Note that you have to do the nix-shell for compiling and the nix-env for runtime installation of openmpi in a same session (same $NIX_PATH) to prevent from having a different version of openmpi at compilation time and run time.

Here’s a sample OAR job script for Dahu, to run your resulting program:

##!/bin/bash            
##OAR -l /nodes=2/core=2
##OAR -t devel
##OAR --project myproject

## Ensure Nix is loaded. The following line can be into your ~/.bashrc file.
source /applis/site/nix.sh

## Run the program
mpirun -np `cat $OAR_FILE_NODES|wc -l` --machinefile $OAR_NODE_FILE -mca plm_rsh_agent "oarsh" --prefix $HOME/.nix-profile $OAR_WORKING_DIRECTORY/hello_mpi

Using Intel compilers and libraries

Intel compilers and libraries are proprietary software that may not be distributed normally as free packages. So we made a specific packaging for CIMENT, in the GRICAD/Nur repository.

Quick usage

To ease the use of the Intel compilers, we have created an environment variable that directly points to a working copy of them. No specific NIX knowledge is necessary to use it. Simply use the $INTEL_ONEAPI variable as the root directory of the Intel compilers. Here is an example for compiling an mpi fortran program using fftw:

source /appli/site/nix.sh
source $INTEL_ONEAPI/setvars.sh
mpiifort -o main_FFTW main_FFTW.f90 -I$MKLROOT/include/fftw -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread -lm

or for the 2024 version:

source /appli/site/nix.sh
source $INTEL_ONEAPI_2024/setvars.sh
export LD_LIBRARY_PATH=$INTEL_ONEAPI_2024_GCC_DIR

You can do the same thing for the runtime, ie “source /appli/site/nix.sh && source $INTEL_ONEAPI/setvars.sh”.

Installation

For a more advanced usage, you might have to install the Intel Oneapi into your nix profile.

This can be made by either install the fixed version:

$ nix-env -i -A $INTEL_ONEAPI_2024

or the latest from the NUR repository of Gricad:

$ nix-env -i -A nur.repos.gricad.intel-oneapi

Compile time

  • Load specific environment variables for the Intel compilers:
source ~/.nix-profile/setvars.sh

The C compiler icc and the Fortran compiler ifort should now be available. The path of libraries is given by the LIBRARY_PATH variable that you might have to pass to your compilation scripts. The LD_LIBRARY_PATH variable also contains the path that may be needed at the running time.

The mpiicc intel mpi command should also be available.

  • Compiling
mpiicc -o hello_mpi hello_mpi.c

(note the double ii into mpiicc )

NOTE: check that your binaries are well linked against the good libraries by issuing the ldd <binary_path> command: most of the reported libraries should be located into a path starting with /nix.

NOTE: if you need a quick path for the libraries, they should all be found into your Nix profile: ~/.nix-profile. For example, blacs is found into ~/.nix-profile/mkl/latest/lib/intel64/

Runtime

source ~/.nix-profile/setvars.sh
mpirun -n `cat $OAR_FILE_NODES|wc -l` -f $OAR_NODE_FILE -bootstrap-exec oarsh ./hello_mpi

Sample package modified to use the Intel compilers: Lammps

You’ll find here: https://github.com/Gricad/nur-packages/tree/master/pkgs/lammps an example of a nixpkgs package that has been modified to be compiled with the Intel compilers and Intel MPI.

Package modification and recompilation, using the NUR GRICAD ’s repository

You may need to recompile a package with some local modifications. Here is an example with the gdal package that we want with the exact version 3.1.1 in place of the lastest.

WARNING: This is actually not what you’ll do if you want to contribute to the Nix project with a fix (you’ll better directly fix the package without changing it’s name, and submit your modifications to the nix team, see https://nixos.org/wiki/Create_and_debug_nix_packages for more informations). But this way, you can test your package modifications and start directly to use it before submitting to the upstream repository.

For this part you’ll need a Github account

  • First of all, get a local copy of the Nix packages definitions (called expressions).
git clone https://github.com/NixOS/nixpkgs
git clone git@github.com:<your_user_name>/nur-packages.git
  • Enter into the nur-packages directory and create a new branch:
cd nur-packages
git checkout -b gdal-3.1.1

Note that you could change the origin branch before cloning it, if you don’t want to work with the unstable channel.

  • Create a new directory if not already present and copy the source of the gdal package:
mkdir -p pkgs/gdal
cp ../nixpkgs/pkgs/development/libraries/gdal/default.nix pkgs/gdal/3.1.1.nix
  • Edit the pkgs/gdal/3.1.1.nix file and replace the version number in the top of the expression:
[...]
stdenv.mkDerivation rec {
  pname = "gdal";
  version = "3.1.1";

[...]
  • We’ll need the sha256 hash of this version. For that, use the nix-prefetch-github tool with the values of the expression:
nix-shell -p nix-prefetch-github --run "nix-prefetch-github OSGeo gdal --rev v3.1.1"
## Then edit again pkgs/gdal/3.1.1.nix to replace the sha256 hash

IMPORTANT: replace the sha256 line of pkgs/gdal/3.1.1.nix with the one you got with this prefetch. If you dont', you will still build the version from the package as it is this hash that is actually used to get the right version from the github repository!

  • Edit the default.nix file that contains the definition of the packages of this nur repository. You have to add an entry for your customized package, for example gdal_311 :
  # gdal 3.1.1 version
  gdal_311 = pkgs.callPackage ./pkgs/gdal/3.1.1.nix { };
  • Build the package
## Not mandatory, but you might have to set your NIX_PATH to the unstable 
## channel as we forked from the unstable channel (or switch to a stable 
## channel before checking out the new git branch):

export NIX_PATH="nixpkgs=channel:nixpkgs-unstable"

## Start the build:

nix-build . -A gdal_311

Note: if it fails with config.rpath' not found, simply add a line preAutoreconf = "touch config.rpath"; into the expression and retry.

  • Run

The result ot the binary package should be linked into ./result:

$ ./result/bin/gdalinfo --version
GDAL 3.1.1, released 2020/06/22
  • Note that other users of the same host could install your package into their profile using the destination of the link:
$ ls -l result
lrwxrwxrwx 1 bzizou users 54 Nov 16 17:43 result -> /nix/store/dz1spldnwwfwp7m4177gwwd20py4gj0z-gdal-3.1.1
$ nix-env -i /nix/store/dz1spldnwwfwp7m4177gwwd20py4gj0z-gdal-3.1.1
  • If you think that your customized package should be useful to other GRICAD users, don’t hesitate to commit, push and pull-request into the nur-gricad respository!

Development environments (advanced usage)

It’s possible to create isolated development environments to be able to:

  • not interfere with other projects
  • install custom libraries, for example in python
  • have a complete reproducibility

Here is an example to create a Python 3 environment with netCDF4 module.

  • Create a default.nix file with this content:
with import <nixpkgs> {}; {
    pyEnv = stdenv.mkDerivation {
      name = "py";
      buildInputs = [ stdenv gcc zlib hdf5 python38Full netcdf python38Packages.numpy python38Packages.virtualenv ];
    };
}

The buildInputs defines a set of Nix packages that are needed dependencies.

  • Now, start a nix-shell to enter this isolated environment. Things may be built at this stage, if binaries are not already into the nix store.
nix-shell

This drops you into a shell.

  • Then, you can create a python virtualenv and load it:
python -m venv netcdf
source netcdf/bin/activate
  • Use pip to install the netCDF4 module:
pip install netCDF4 
  • You can now exit from the shell:
exit
  • Make a backup of this environement:
nix-instantiate --indirect --add-root $PWD/shell.drv
  • This allows you to get into the same environement:
nix-shell shell.drv
  • You can use it from a bash script:
##!/usr/bin/env nix-shell                                                    
##!nix-shell -I shell.drv -i bash 
                                                                           
source netcdf/bin/activate                      
                                                                            
export LD_LIBRARY_PATH="${NIX_LDFLAGS_FOR_TARGET// -L/:}"

which python                                                                
python -c "import netCDF4"