CiGri is a lightweight grid that relies on a set of underlying computing clusters.
It is particularly well suited for processing bag-of-tasks applications or embarrassingly parallel jobs (this type of processing can result in a very large number of small jobs, such as multi-parametric applications, where you need to run the same program on a large set of parameters and get one result per parameter).
However, it is also capable of handling any type of job that needs to be run on computing clusters.
Its particularity is to run on the unused cpu cycles of a set of supercomputers in a computing center, in the most efficient way possible. But it can also be used to manage a large number of tasks to be executed on a particular cluster, even without effort (a large number of tasks usually overloads the local scheduler of a cluster; the grid is designed to deal with this problem).
In particular, it offers efficient access to an efficient submission mode, besteffort.
Best-effort jobs are zero priority jobs that may be killed without notice if a normal job needs the resources.
It’s a way to compute transparently on clusters or rerrounter that are not yours, taking advantage of moments when processing resources are idle.
CiGri is a useful tool to manage such jobs because you may have to submit again and again the same job until it is not killed. We call this re-submission and CiGri does this automatically and transparently for you.
An in-depth tutorial, using Guix and Mantis, detailing data processing on different clusters, software environments and how to properly test jobs on each target cluster, is available here.