Skip to main content

​
Operating System

Ubuntu 20.04 LTS or later
Ubuntu 22.04 LTS (recommended)
Debian 11 or later

​
Required Software

1

Docker

Docker Engine 20.10 or later
docker --version
Expected output: Docker version 20.10.0 or higher
2

Docker Compose

Docker Compose 2.0 or later
docker compose version
Expected output: Docker Compose version v2.0.0 or higher
3

NVIDIA Driver

NVIDIA Driver 525+ for newer GPUs (A10, A100, L4)NVIDIA Driver 470+ for older GPUs (T4, V100)
nvidia-smi
Should display GPU information without errors
4

NVIDIA Container Toolkit

Required for GPU access in containers
nvidia-container-cli --version

​
Install Docker

sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
  sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io \
  docker-buildx-plugin docker-compose-plugin

sudo usermod -aG docker $USER
newgrp docker

​
Install NVIDIA Driver

sudo apt-get update
sudo apt-get install -y ubuntu-drivers-common

sudo ubuntu-drivers autoinstall

sudo reboot
After reboot, verify:
nvidia-smi

​
Install NVIDIA Container Toolkit

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo apt-key add -
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

sudo systemctl restart docker

​
Verify GPU Access

Test that Docker can access the GPU: 
docker run --rm --gpus all nvidia/cuda:11.8.0-base-ubuntu22.04 nvidia-smi
You should see your GPU information displayed.