Mantis cloud storage infrastructure

<TL;DR>

To use Mantis, you have to run once:

source /applis/site/nix.sh
/applis/site/mantis_init.sh

and, to keep your authentication token active, do at least once every 2 weeks:

source /applis/site/nix.sh
iinit

Access from the cluster via command line is done through iCommands (ils, iput, iget…) which are packaged in Nix:

source /applis/site/nix.sh

</TL;DR>

Mantis

Mantis is an iRODS distributed storage cloud available from every computing nodes of the GRICAD HPC infrastructure. It is an efficient way to store input data or results of computing jobs (stage-in/stage-out). It is powerful (as storage is distributed among several nodes), capacitive and extensible (laboratories can easily contribute to extend the storage capacity by adding new 250TB nodes) and provides a programmable rules engine.

Access and data throughput is not as fast as on other infrastructures such as Bettik but it offers a point where to load data prior to launching a computational job and to store the final output.

Most storage infrastructures, in particular cloud storage, do not handle large collections of small files very well and this is especially the case with iRODS. It is very much preferable to make a large bundle of files with a tool such as tar before uploading them to Mantis and splitting the bundle on the computing node after it has been downloaded rather than uploading a large collection of small files. Each file transaction (read, write, copy, move) generates many associated database transactions in iRODS, the overhead on the actual file operation can rapidly become very large and negatively affect the walltime of your computational jobs.

Mantis is currently entirely located in the IMAG datacenter and does not offer data storage pool multi-locality.

Access

Currently, access to the Mantis storage is provided from the Dahu, Bigfoot, Luke, Froggy and Ceciccluster clusters. All users who have a Perseus account automatically have a home space in Mantis.

Usage of this infrastructure is submited to acceptation of the charter.

Initialisation

Before using the Mantis storage for the first time, you need to initialize a configuration directory into your home on each computing cluster. To do so, please launch:

/applis/site/mantis_init.sh

This will create a .irods directory with a default configuration and ask for your perseus password to create an authentication token. This is a one-time process.

If you previously used Mantis 1, this will copy your old irods configuration and adapt it to allow further re-use.

Authentication

If your authentication token is expired, you may have the following error:

failed with error -826000 CAT_INVALID_AUTHENTICATION

The solution is to re-authenticate with the iinit command and provide your GRICAD perseus password:

mylogin@mantis-cargo:~$ iinit
Enter your current PAM password:
mylogin@mantis-cargo:~$

You should normally be able to set up a TTL of 1 year (iinit –ttl 8640) but there’s currently a bug that prevents this ttl option to work with our authentication scheme.

So, for now, the password TTL is 2 weeks. That means that you’ll have to use iinit at least once every 2 weeks.

Quotas

By default, a 1TB quota is set up for every user. If you need more space, please open a ticket on the SOS-GRICAD helpdesk or by writing to sos-calcul-gricad.

Clients

iCommands

To access Mantis via the command-line, you need to use the iCommands clients. They are available as a Nix package which is installed by default when you source the nix environment:

source /applis/site/nix.sh

Common icommands are : ils (to list files), iput (to upload files), iget (to download files). Check the irods documentation for more informations.

Note: if you do not whish to load the nix environment, be sure to add /nix/var/nix/profiles/default/bin to your PATH environment variable (export PATH=/nix/var/nix/profiles/default/bin:$PATH)

Web interface

The Metalnx web interface allows you to browse and manage your files from a web browser. The Mantis Matalnx interface is accessible here: Metalnx

Webdav protocol

The webdav protocol may be useful as a gateway to world outside of the Mantis cloud storage. All you need is a webdav client and connect it to the following URL:

https://mantis.univ-grenoble-alpes.fr/davrods/

The trailing slash is important

Example session with the Linux command line client cadaver:

[bzizou@bart:~]$ cadaver https://mantis.univ-grenoble-alpes.fr/davrods/
WARNING: Untrusted server certificate presented for `mantis.univ-grenoble-alpes.fr':
Issued to: Université Grenoble Alpes, 621 avenue Centrale, Saint-Martin-d'Hères, Auvergne-Rhône-Alpes, 38400, FR
Issued by: GEANT Vereniging, NL
Certificate is valid from Tue, 15 Dec 2020 00:00:00 GMT to Wed, 15 Dec 2021 23:59:59 GMT
Do you wish to accept the certificate? (y/n) y
Authentication required for DAV on server `mantis.univ-grenoble-alpes.fr':
Username: bzizou
Password: 
dav:/davrods/> ls
Listing collection `/davrods/': succeeded.
Coll:   povray_results_mantis2                 0  Jun  3  2020
Coll:   system                                 0  Jun 30 16:48
        10GBtestfile                  10485760000  Nov 13 14:26
        test1                          161886424  Aug 26 15:08
dav:/davrods/> get test1
Downloading `/davrods/test1' to test1:
Progress: [=============================>] 100.0% of 161886424 bytes succeeded.
dav:/davrods/> quit
Connection to `mantis.univ-grenoble-alpes.fr' closed.

Cargo

A specific host has been set up for some staging operations to or from Mantis. This is mantis-cargo.u-ga.fr. This host is directly accessible with your Perseus account and provides iCommands clients by default.

This “Cargo” host also provides a big local scratch directory for staging operations (located in /cargo).

Two specific scripts have been set up to allow irods registrations or centralized replication:

~$ cireg.py -h
Usage: cireg.py [options]

Options:
  -h, --help            show this help message and exit
  -c COLLECTION, --collection=COLLECTION
                        Full path of the directory to register as an irods
                        collection
  -y, --yes             Answer yes to all questions
  -K, --checksum        Calculate a checksum on the iRODS server and store
                        with the file details
  -v, --verbose         Be verbose

~$ cipull.py -h
Usage: cipull.py [options]

Options:
  -h, --help            show this help message and exit
  -c COLLECTION, --collection=COLLECTION
                        Full path of the collection to centralize
  -y, --yes             Answer yes to all questions
  -v, --verbose         Be verbose

Remote direct access

Remote direct access from machines within research laboratories is possible, in order to allow users more flexibility in their computational data management (initial data generation directly in the infrastructure, upload of expiremental data output by an equipment, computational output download, etc). In order to set up and optimise these accesses, a case by case evaluation must be requested. To do so, please contact us, either by email at sos-calcul-gricad or via the SOS-GRICAD helpdesk.


Mantis charter

Service offered by GRICAD/CIMENT

GRICAD/CIMENT offers its users a distributed storage system accessible from all the main GRICAD/CIMENT computing clusters.

The core [1] of this system is targeted primarily for the “Grid” mode users (mainly CiGri), this is to say users who run computing jobs involving multiple platforms simultaneously. Access outside grid usage are however also possible, in particular when a laboratory has contributed to the extension of the systeme or for archiving in order to reduce the burden on local disk systems.

[1]: the “core” corresponds to the initial GRICAD/CIMENT volume financed using internal funds for approximately 750Tb.

Quotas

All users with a Perseus account have a default storage space of 1Tb on Mantis in order to simplify access and allow to test the solution.

For requests to extend this quota, please contact us, either by email at sos-calcul-gricad or via the SOS-GRICAD helpdesk.

Participating in capacity increase

As users, you can participate to increase the capacity of the infrastructure. You can contribute, through your laboratory and projects, to purchase hardware which will be added to the existing pool.

The crontibuted hardware will be added to the existing capacity without any distinction in order to increase the global capacity. However, the users or groups who have contributed will see their quota increased by the amount of their net contribution.

Users and groups who have contributed will be able to use the infrastructure without necessarily using the grid mode.

Caveats

No data backup is operated by GRICAD. The infrastructure is built to be resilient to minor hardware failures however we cannot be held responsible in case of loss of data stored on the infrastructure in case of major hardware failure.

We also remind you that the infrastructure’s resilience does not make it a backup. Contrary to a backup system, it does not prevent loss of data due to a manipulation error or software caused data corruption.

The data security is ensured within the limits of the middleware’s capacity. As a user, you have the control over the access rules for data stored on the infrastructure. As such, GRICAD cannot be held responsible for loss or theft of data through misuse of the data access rules mechanisms or in case of a system malfunction.

User engagement

  • Users are bound by the UGA Use of IT Facilities charter.

  • Only data stronly related to the research project under which they have access to the infrastructure will be stored on it.

  • Users must delete all obsolete data in order to free up space globally regardless of their quota usage.

  • Users will report any suspected malfunction to the support staff by mail to sos-calcul-gricad or via the SOS-GRICAD helpdesk.

  • Users with sensitive data requiring particular protection measures must make this known to the support staff by mail to sos-calcul-gricad or via the SOS-GRICAD helpdesk so that appropriate solutions may be envisaged and to be properly informed of the protection level granted.