The goal of this tutorial is to explain how to install a local server of Ollama. Once the server is installed and working, we will look at the steps that need to be taken in order to download and start an LLM on the clusters.
Before installing the server, let us take a look at what is Ollama, and what this server will allow us to do. Ollama is a tool designed to encapsulate and execute Large Language Models locally. When this tool is installed, it will run like a local instance of a server that connects to a remote server which contains a large library of models that can be downloaded and ran locally. In a nutshell, this server will allow us to select as many LLM as we want, and chat with them.
If you cannot fulfill these two prerequisites, you can access the documentation regarding the creation of a Perseus account and connection to the clusters. In this tutorial, we will need GPUs, so we will work on the Bigfoot cluster.
Even when simply doing the installation part, the steps that we are taking can be ressource heavy. Hence why it is necessary to connect to an interactive node of Bigfoot from the beginning. In order to do so, you can use the following line from the frontend of Bigfoot:
oarsub -l /nodes=1/gpu=1/migdevice=1 --project project_name -t devel -I
Linux bigfoot 5.10.0-35-amd64 #1 SMP Debian 5.10.237-1 (2025-05-19) x86_64
Welcome to Bigfoot cluster!
: :
.' :
_.-" :
_.-" '.
..__...____...-" :
: \_\ :
: .--" :
`.__/ .-" _ :
/ / ," ,- .'
(_)(`,(_,'L_,_____ ____....__ _.'
"' " """"""" """
GPU, GPU, GPU, ... ;-)
Type 'chandler' to get cluster status
Type 'recap.py' to get cluster properties
Please, read the doc:
https://gricad-doc.univ-grenoble-alpes.fr/hpc/joblaunch/
Sample OAR submissions:
# Get a A100 GPU and all associated cpu and memory resources:
oarsub -l /nodes=1/gpu=1 --project test -p "gpumodel='A100'" "nvidia-smi -L"
# Get a MIG partition of an A100 on a devel node, to make some tests
oarsub -l /nodes=1/gpu=1/migdevice=1 --project test -t devel "nvidia-smi -L"
Last login: Fri Jul 25 11:52:44 2025 from 129.88.196.128
login@bigfoot:~$ oarsub -l /nodes=1/gpu=1/migdevice=1 --project project_name -t devel -I
[DEVEL] Adding devel resource constraints
[MIG] Automatically adding the mig constraint
[WARNING] 'gpumodel' not specified!
[WARNING] it is recommended to specify the gpumodel to use, for example:
[WARNING] -p "gpumodel='A100' or gpumodel='V100'"
[WARNING] or:
[WARNING] -p "gpumodel!='T4'"
[ADMISSION RULE] Adding gpubrand=nvidia constraint by default
[ADMISSION RULE] Use -t amd if you want to use AMD GPUs
[ADMISSION RULE] Set default walltime to 7200.
[ADMISSION RULE] Modify resource description with type constraints
OAR_JOB_ID=3461326
Interactive mode: waiting...
[2025-07-25 13:11:41] Start prediction: 2025-07-25 13:11:41 (R=4,W=2:0:0,J=I,P=project_name,T=devel (Karma=0.038,quota_ok))
Starting...
Connect to OAR job 3461326 via the node bigfoot7
login@bigfoot7:~$ cd /bettik/PROJECTS/project_name/login
Now that we know in which root folder we are going to work, and to make sure that we have a clean installation, we are going to create a folder to store all the files we are working with. Just run the following command:
OLLAMA_ROOT=/bettik/PROJECTS/project_name/login/ollama
mkdir -p ${OLLAMA_ROOT}
There exist a number of releases of Ollama, and we have to choose the one that we wish to install. You can directly access the lastest release of Ollama, or you can scroll through all the existing releases. At the time of writing, the latest release was 0.9.6. You can run the following command while replacing OLLAMA_VERSION by the one you will be using:
OLLAMA_VERSION=v0.9.6
OLLAMA_VERSION_URL=https://github.com/ollama/ollama/releases/download/${OLLAMA_VERSION}/ollama-linux-amd64.tgz
mkdir -p ${OLLAMA_ROOT}/${OLLAMA_VERSION} && wget -O - ${OLLAMA_VERSION_URL} 2>wget.err | tar xzvf - -C ${OLLAMA_ROOT}/${OLLAMA_VERSION}
Some of the available models will depend on the version of Ollama that you choose to install, which is the reason why we will create a storage space in the .ollama directory to be able install multiple versions. To do so, we can use the environment variable of Ollama that we will define with:
export OLLAMA_MODELS=${OLLAMA_ROOT}/${OLLAMA_VERSION}/.ollama
Afterward, and thanks to the already downloaded files, we will add all the Ollama commands to our environment by using a Shell script. The goal is to create this file, open it, and add all the necessary content. We are going to start by creating this file, while being located inside the previously created ollama folder :
login@bigfoot7:/bettik/PROJECTS/project_name/login/ollama$ touch env_ollama-${OLLAMA_VERSION}.sh
Then, we are going to open this file in vim with the command vim env_ollama-${OLLAMA_VERSION}.sh. When in vim, simply press the i key on your keyboard, in order to insert content, and paste the text below, while modifying it to suit your needs.
OLLAMA_VERSION=your_ollama_version
OLLAMA_ROOT=/bettik/PROJECTS/project_name/login/ollama
# Ollama environment
export LD_LIBRARY_PATH=${OLLAMA_ROOT}/${OLLAMA_VERSION}/lib:${LD_LIBRARY_PATH}
export PATH=${OLLAMA_ROOT}/${OLLAMA_VERSION}/bin:${PATH}
export OLLAMA_MODELS=${OLLAMA_ROOT}/${OLLAMA_VERSION}/.ollama
Once the content has been added to your file, press the Escape key, and write :wq before pressing Enter. You will be in your terminal once again, and you can verify the content of the file created with cat env_ollama-${OLLAMA_VERSION}.sh.
Lastly, we will need to make this file executable with chmod +x env_ollama-${OLLAMA_VERSION}.sh.
At this point, we have all the necessary files to start our server. All the previous steps need to be done only once by version of Ollama. Because we are now going to start a server and chat with an LLM, it is imperative that you are working on a development node of Bigfoot, simply refer to what was previously mentionned on this page regarding Bigfoot nodes. In order to have access to the commands given by ollama, we first need to source the Shell file that we have created earlier:
login@bigfoot7:/bettik/PROJECTS/project_name/login/ollama$ source env_ollama-your_ollama_version.sh
To start the server as a background task, you can use the following command:
ollama serve &>ollama-server.log &
This command will start a server, write its logs in the ollama-server.log file, and run in the background so that you can interact with it from the same terminal.
Now that we can interact with the server, there are multiple commands that are available in order to manage LLMs. The list of every available commands can be accessed by typing ollama help. The most useful ones are:
ollama list to list all the locally available models.ollama pull mistral to download a model, for example mistral.ollama run mistral to run a chat with a given LLM, for example mistral.Whenever a chat is started, it is then possible to exchange messages with an LLM. Whatever you write is stored in the context memory of the LLM while your session is active. To leave the chat, simply type \bye. Lastly, the server is automatically closed when you log out of the node that you are currently located on.