For all large data volume transfer operations on the Dahu, Bigfoot and Luke clusters it is important to use the Cargo machine and never launch them on cluster head nodes.
The machine cargo.univ-grenoble-alpes.fr is a dedicated server for all large data volume transfer operations on the Luke and Dahu clusters.
Cargo is only accessible via ssh key authentication. It is therefore necessary to have copied your ssh key to Dahu in order to access it. You can’t access it by password. You can’t copy your ssh key directly to Cargo either, you have to copy it to the associated cluter first. If you have already set up the transparent connection to the clusters with the alias “proxycommand” .ciment, you can copy your ssh key to Dahu with the command ssh-copy-id dahu.ciment.
Do not try to connect to cargo if you have not already copied your SSH public key on the Dahu cluster.
If you try to connect without having beforehand copied your SSH public key on Dahu, you will get a "Permission denied (publickey)" error.
It provides access to:
It is accessible via the ssh and ftp protocols from any IP address. It therefore allows to run data transfer via rsync, scp or ftp between storage spaces on this server and any distant machine.
It is very important to use this server for all large data transfer operations, as it is a machine dedicated for this function, rather than cluster head nodes. Any large transfer operation launched on cluster head nodes is susceptible to be killed without warning.
As Cargo is accessible via ssh from any IP address. You can connect directly, via the command ssh login@cargo.univ-grenoble-alpes.fr, without having to pass through the bastions or cluster head nodes and run any file transfer between the different storage spaces from there.
An example of call allowing to synchronise a directory in your home space to Bettik is :
my-perseus-login@dahu|luke:~$ rsync -avxH /home/{mon-login}/{mon-repertoire}/ /bettik/{mon-login}/{mon-repertoire}/
NB: the / at the end of the paths are important.
Screen and tmux are available on Cargo, allowing you to create a command line terminal in which you can run your file transfer operations without having to stop them when you must shut down your connexion.
You can also transfer data between your local machine and Bettik using the cargo server :
my-local-login@ma-machine:~: rsync -avxH /{local-storage-path}/ my-perseus-login@cargo.univ-grenoble-alpes.fr:/bettik/{path-to-my-data}/
Do not add a line loading an environment such as guix, nix, module or other (e.g. source /applis/site/guix-start.sh) via the .bashrc or .profile file, you will probably get unexpected side-effects which usually prevent you from doing data transfers.