Jetson Nano Orin config for Whisper and Piper

Hardware
1

I struggled to find a working Docker compose to run a version of Whisper that made full use of the Jetson GPU. This works really well (finally!). Hope this is useful

# ==============================================================================
# Wyoming Voice Stack (Whisper TRT + Piper TTS)
# Platform: NVIDIA Jetson Orin Nano (JetPack 6 / iGPU)
# Path: ~/voice/docker-compose.yml
# ==============================================================================

services:
  wyoming-whisper-trt:
    container_name: wyoming-whisper-trt
    image: captnspdr/wyoming-whisper-trt:latest-igpu
    restart: unless-stopped
    
    # Enable NVIDIA Container Runtime for Tegra iGPU
    runtime: nvidia

    # Shared memory allocation needed for CUDA context
    ipc: host
    shm_size: "2gb"

    ports:
      - "10300:10300"

    devices:
      - /dev/dri:/dev/dri

    environment:
      # Expose NVIDIA Compute and Video hardware
      NVIDIA_VISIBLE_DEVICES: "all"
      NVIDIA_DRIVER_CAPABILITIES: "compute,utility,video"

      # Prevent PyTorch memory fragmentation on unified memory
      PYTORCH_CUDA_ALLOC_CONF: "expandable_segments:True"

      # Whisper Model Parameters
      MODEL: "base.en"
      LANGUAGE: "en"
      URI: "tcp://0.0.0.0:10300"
      COMPUTE_TYPE: "float16"
      DEVICE: "cuda"
      BEAM_SIZE: "1"
      TEMPERATURE: "0.0"
      DECODER_MODE: "simple"
      VAD_FILTER: "true"
      VAD_THRESHOLD: "0.5"

      # Persistence paths for model assets
      DATA_DIR: "/data"
      DOWNLOAD_DIR: "/data"
      HF_HOME: "/data/hf_cache"
      TORCH_HOME: "/data/torch_cache"

    volumes:
      # Stores compiled TensorRT engine files and HuggingFace models on host
      - ./data/whisper:/data
      # Preserves PyTorch/TRT internal build timing caches on host
      - ./data/whisper_cache:/root/.cache
      # NAMED VOLUME: Mounts full app folder so Docker populates source code first, then saves .venv
      - whisper_app_state:/usr/src/wyoming-whisper-trt

  wyoming-piper:
    container_name: wyoming-piper
    image: rhasspy/wyoming-piper:latest
    restart: unless-stopped
    ports:
      - "10200:10200"
    environment:
      - TZ=Europe/London
    command:
      - "--voice"
      - "en_GB-alan-medium"
      - "--length-scale"
      - "0.90"
      - "--uri"
      - "tcp://0.0.0.0:10200"
      - "--data-dir"
      - "/data"
      - "--download-dir"
      - "/data"
      - "--speaker"
      - "0"
    volumes:
      # Saves downloaded voice models (.onnx and .json metadata) on host
      - ./data/piper:/data

# Declare the named volume managed by Docker
volumes:
  whisper_app_state:

It will take about 10 mins to build the environment; be patient! . You can check progress by running
docker compose logs

Once built, it preserves the Python venvs; otherwise, it takes 10 mins to rebuild every time it restarts. Be aware this forces every subsequent reboot to use the initial compiled files. If you need to update the container to a later version, run the following:

Step 1: Stop the stack

docker compose down

Step 2: Delete ONLY the app state named ‘volume’ holding the old .venv

docker volume rm voice_whisper_app_state

(Note: voice_ is the folder where the container is stored. Change as required)

Step 3: Pull the latest image and start the stack (Docker will recreate the volume fresh!).

docker compose pull && docker compose up -d

Step 4: Tail the logs to confirm startup

docker compose logs

2

Have changed out the piper for pocket TTS. This has much nicer voices and handles years and dates better.

# ==============================================================================
# Wyoming Voice Stack (Whisper TRT + Pocket TTS)
# Platform: NVIDIA Jetson Orin Nano (JetPack 6 / iGPU)
# Path: ~/voice/docker-compose.yml
# ==============================================================================

services:
  wyoming-whisper-trt:
    container_name: wyoming-whisper-trt
    image: captnspdr/wyoming-whisper-trt:latest-igpu
    restart: unless-stopped

    # Enable NVIDIA Container Runtime for Tegra iGPU
    runtime: nvidia

    # Shared memory allocation needed for CUDA context
    ipc: host
    shm_size: "2gb"

    ports:
      - "10300:10300"

    devices:
      - /dev/dri:/dev/dri

    environment:
      # Expose NVIDIA Compute and Video hardware
      NVIDIA_VISIBLE_DEVICES: "all"
      NVIDIA_DRIVER_CAPABILITIES: "compute,utility,video"

      # Prevent PyTorch memory fragmentation on unified memory
      PYTORCH_CUDA_ALLOC_CONF: "expandable_segments:True"

      # Whisper Model Parameters
      MODEL: "base.en"
      LANGUAGE: "en"
      URI: "tcp://0.0.0.0:10300"
      COMPUTE_TYPE: "float16"
      DEVICE: "cuda"
      BEAM_SIZE: "1"
      TEMPERATURE: "0.0"
      DECODER_MODE: "simple"
      VAD_FILTER: "true"
      VAD_THRESHOLD: "0.5"

      # Persistence paths for model assets
      DATA_DIR: "/data"
      DOWNLOAD_DIR: "/data"
      HF_HOME: "/data/hf_cache"
      TORCH_HOME: "/data/torch_cache"

    volumes:
      # Stores compiled TensorRT engine files and HuggingFace models on host
      - ./data/whisper:/data
      # Preserves PyTorch/TRT internal build timing caches on host
      - ./data/whisper_cache:/root/.cache
      # NAMED VOLUME: Mounts full app folder so Docker populates source code first, then saves .venv
      - whisper_app_state:/usr/src/wyoming-whisper-trt

  wyoming-pocket-tts:
    container_name: wyoming-pocket-tts
    image: ghcr.io/araa47/wyoming_pocket_tts:latest
    restart: unless-stopped
    ports:
      # Host:Container mapping
      - "10200:10200"
    environment:
      - TZ=Europe/London
      # Redirect Hugging Face cache into mounted host volume
      - HF_HOME=/data/hf_cache
      # Enforce strict offline operation using local cache
      - HF_HUB_OFFLINE=1
      - TRANSFORMERS_OFFLINE=1
      # Disable HTTP/HTTPS proxy lookups in Python to eliminate connection timeouts
      - NO_PROXY=*
      - no_proxy=*
      # Prevent PyTorch OpenMP thread allocation stalls
      - OMP_NUM_THREADS=1
    # Tell Docker's resolver to immediately resolve/fail IPv6 DNS queries
    dns_opt:
      - single-request-reopen
      - use-vc
    command:
      - python3
      - -m
      - wyoming_pocket_tts
      - --host
      - "0.0.0.0"
      - --port
      - "10200"
      - --language
      - en
    volumes:
      # Local directory for caching pocket-tts data
      - ./data/pocket_tts:/data
      # Named volume for custom/downloaded voice clips
      - pocket-tts-voices:/share/tts-voices

# Declare named volumes managed by Docker
volumes:
  whisper_app_state:
  pocket-tts-voices: