This section provides a detailed guide on the configuration options for the development environment of the docker/genai-stack. The focus is on the docker-compose.yml file, which serves as the main configuration for the Docker-based architecture.

Services Configuration

The primary services defined in the docker-compose.yml include:

• llm
• llm-gpu
• pull-model
• database
• loader
• bot
• pdf_bot
• api
• front-end

1. Orchestrating the LLM Services

The language model service (llm) is configured with a Docker image and designated profiles. The GPU variant llm-gpu is set up to leverage NVIDIA’s GPU capabilities.

services:

  llm: &llm
    image: ollama/ollama:latest
    profiles: ["linux"]

  llm-gpu:
    <<: *llm
    profiles: ["linux-gpu"]
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

2. Pull Model Service

The pull-model service retrieves models on startup. Its configuration involves setting environmental variables using placeholder syntax:

  pull-model:
    image: genai-stack/pull-model:latest
    build:
      context: .
      dockerfile: pull_model.Dockerfile
    environment:
      - OLLAMA_BASE_URL=${OLLAMA_BASE_URL-http://host.docker.internal:11434}
      - LLM=${LLM-llama2}

This allows customization of the model being pulled with the option to override the defaults using environmental variables.

3. Database Configuration

The database service utilizes Neo4j. The following settings ensure correct configuration regarding authentication and data storage:

  database:
    user: neo4j:neo4j
    image: neo4j:5.11
    ports:
      - 7687:7687
      - 7474:7474
    volumes:
      - $PWD/data:/data
    environment:
      - NEO4J_AUTH=${NEO4J_USERNAME-neo4j}/${NEO4J_PASSWORD-password}
      - NEO4J_PLUGINS=["apoc"]

The health check is also specified to ensure that the service is running correctly:

    healthcheck:
      test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider localhost:7474 || exit 1"]
      interval: 15s
      timeout: 30s
      retries: 10

4. Loader and Bot Configuration

Both the loader and bot services share similar configurations, particularly in their environmental variables, which primarily relate to Neo4j and APIs. Example configuration for the loader service is shown below:

  loader:
    build:
      context: .
      dockerfile: loader.Dockerfile
    environment:
      - NEO4J_URI=${NEO4J_URI-neo4j://database:7687}
      - OPENAI_API_KEY=${OPENAI_API_KEY-}

5. PDF Bot Configuration

Similarly, the pdf_bot configuration mirrors that of the bot, but it highlights the uniqueness required for processing PDF-related tasks:

  pdf_bot:
    build:
      context: .
      dockerfile: pdf_bot.Dockerfile
    environment:
      - NEO4J_URI=${NEO4J_URI-neo4j://database:7687}
      - OPENAI_API_KEY=${OPENAI_API_KEY-}

6. API Configuration

The api service integrates various environmental variables, alongside robust health checks, ensuring it is functioning correctly post-initialization:

  api:
    build:
      context: .
      dockerfile: api.Dockerfile
    healthcheck:
      test: ["CMD-SHELL", "wget --no-verbose --tries=1 http://localhost:8504/ || exit 1"]
      interval: 5s
      timeout: 3s
      retries: 5

7. Front-End Configuration

The front-end service is designed to sync with the application, listening for changes and rebuilding as necessary. This includes dependency management for the front-end directory:

  front-end:
    build:
      context: .
      dockerfile: front-end.Dockerfile
    x-develop:
      watch:
        - action: sync
          path: ./front-end
          target: /app
          ignore:
            - ./front-end/node_modules/

Summary

The docker-compose.yml configuration provides a solid foundational setup for the development environment of the docker/genai-stack, focusing on microservices architecture, proper resource allocation, and handling custom environmental configurations. Each service’s environmental variables and networking settings are crucial in ensuring proper connectivity and functionality within the stack.

Source: Provided Docker compose configuration details.