Stack Description: Ollama + Open WebUI with Multi-GPU Support (Docker + Portainer + Native WSL2)

Overview

This Docker Compose configuration deploys:

1. ollama – A local LLM inference engine that uses both available GPUs for optimal performance.
2. open-webui – A web-based frontend to interact with Ollama, isolated to use only GPU 1 (leaving GPU 0 free for other processes).

YAML

version: "3.9"

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    ports:
      - 11434:11434
    volumes:
      - ollama:/mnt/c/Users/Docker/ollama
    tty: true
    restart: unless-stopped
    runtime: nvidia
    environment:
      - NVIDIA_VISIBLE_DEVICES=0,1  # Use both GPUs
    deploy: {}

  open-webui:
    image: ghcr.io/open-webui/open-webui:cuda
    container_name: open-webui
    ports:
      - 3000:8080
    volumes:
      - open-webui:/app/backend/data
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_SECRET_KEY=
      - NVIDIA_VISIBLE_DEVICES=1  # Use only GPU 1
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: unless-stopped
    runtime: nvidia
    deploy: {}

volumes:
  ollama: {}
  open-webui: {}

Key Features

• Multi-GPU support via NVIDIA Container Runtime
• GPU isolation between containers using NVIDIA_VISIBLE_DEVICES
• Portainer-compatible stack (no Swarm mode required)
• No device mounts required (/dev/nvidia*) — handled by NVIDIA runtime
• Optimized for native Docker inside WSL2, without Docker Desktop

GPU Configuration Summary

Container	GPUs Used	Purpose
ollama	GPU 0 + 1	Heavy inference (max power)
open-webui	GPU 1	Lightweight UI (non-blocking)

Configuration is handled via:

runtime: nvidia
environment:
  - NVIDIA_VISIBLE_DEVICES=0,1  # For ollama
  - NVIDIA_VISIBLE_DEVICES=1    # For open-webui

Requirements

This setup assumes the following system configuration:

Component	Requirement
Host OS	Windows 10/11
WSL2 Distro	Ubuntu or Debian-based (where Docker is installed)
Docker Engine	Installed natively inside WSL2 (not via Docker Desktop)
NVIDIA Drivers (Windows)	Installed on the host (supporting WSL2 + CUDA)
CUDA Support in WSL2	Automatically provided if GPU drivers are correct
NVIDIA Container Toolkit	Installed inside WSL2 (see below)

How to Enable GPU Support in WSL2 Docker

Run these steps inside your WSL2 distro:

# Install NVIDIA Container Toolkit
sudo apt update
sudo apt install -y nvidia-container-toolkit

# Configure Docker to use the NVIDIA runtime
sudo nvidia-ctk runtime configure --runtime=docker

# Restart Docker
sudo systemctl restart docker  # or use sudo service docker restart

Verify that Docker can access the GPU:

docker run --rm --gpus all nvidia/cuda:12.2.0-base nvidia-smi

You should see both GPUs listed.

Notes

• You do not need Docker Desktop for this setup — it runs entirely within native WSL2.
• Device files like /dev/nvidia1 exist inside WSL2 and are managed by the NVIDIA runtime — no need to mount them manually in Compose.
• Portainer must also be running inside WSL2 with GPU access, or be configured to talk to the WSL2 Docker daemon directly.

Validation Tips

To confirm it’s working correctly:

# Inside each container:
docker exec -it ollama nvidia-smi
docker exec -it open-webui nvidia-smi

Expected:

• ollama sees both GPUs (0 and 1)
• open-webui sees only GPU 1

Monitor usage with:

watch -n 1 nvidia-smi