Production Scaling in pingcap/autoflow

Scaling the pingcap/autoflow project for production involves multiple steps that ensure high availability, efficient resource utilization, and improved performance. Below are detailed instructions and code examples to guide the scaling process.

Docker Compose Configuration

Utilize Docker Compose to orchestrate your application services. Paid attention to the docker-compose.yml file, which defines the services necessary for the application to run effectively. The setup outlined here supports scalability via containerized services.

Example: docker-compose.yml

version: '3.8'

services:
  redis:
    image: redis:6.0.16
    restart: always
    volumes:
      - ./redis-data:/data
    command: ["redis-server", "--loglevel", "warning"]

  backend:
    image: tidbai/backend:0.2.8
    restart: always
    depends_on:
      - redis
    ports:
      - "8000:80"
    env_file:
      - .env
    volumes:
      - ./data:/shared/data
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "6"

  frontend:
    image: tidbai/frontend:0.2.8
    restart: always
    depends_on:
      - backend
    ports:
      - 3000:3000
    environment:
      BASE_URL: http://backend
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "6"

  background:
    image: tidbai/backend:0.2.8
    restart: always
    depends_on:
      - redis
    ports:
      - "5555:5555"
    env_file:
      - .env
    volumes:
      - ./data:/shared/data
    command: /usr/bin/supervisord
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "6"

  local-embedding-reranker:
    image: tidbai/local-embedding-reranker:v3-with-cache
    ports:
      - 5001:5001
    environment:
      - PRE_LOAD_DEFAULT_EMBEDDING_MODEL=true
      - PRE_LOAD_DEFAULT_RERANKER_MODEL=false
      - TRANSFORMERS_OFFLINE=1

Code Deployment and Environment Variables

Environment variables play a crucial role in configuring your application for different environments. They allow you to scale deployments by toggling features or settings without altering the codebase.

Example: Environment Variables Configuration

In the Docker Compose file, you can see that the environment variables are specified for each service, which can be overridden using an .env file:

env_file:
  - .env

This respects the twelve-factor app methodology, ensuring configurations are portable.

Dockerfile for Production

The Dockerfile is crucial for building the image that will run your application in a production environment. It includes multiple stages to optimize the build process, ensuring only relevant files are included in the final image.

Example: Dockerfile

FROM node:20-alpine AS base

# Install dependencies
FROM base AS deps
RUN apk add --no-cache libc6-compat

WORKDIR /tidb.ai
COPY frontend/package.json ./frontend
COPY frontend/pnpm-*.yaml ./frontend
COPY frontend/app/package.json ./frontend/app

WORKDIR /tidb.ai/frontend
RUN corepack enable pnpm
RUN pnpm i --frozen-lockfile

# Build
FROM base AS builder
RUN apk add --no-cache git
WORKDIR /tidb.ai
COPY --from=deps /tidb.ai/frontend/node_modules ./frontend/node_modules
COPY . .
WORKDIR /tidb.ai/frontend
ENV NODE_ENV=production
RUN npm run build

# Production image
FROM base AS runner
WORKDIR /tidb.ai
COPY --from=builder /tidb.ai/frontend/app ./
USER nextjs
EXPOSE 3000
CMD ["node", "app/server.js"]

Handling Scalability with Load Balancing

When deploying the frontend and backend components, it’s essential to consider load balancing to enhance scalability. Using reverse proxy solutions like Nginx or cloud-based load balancers can distribute traffic effectively among multiple instances of your services.

Service Replication

Docker Compose allows for easy service scaling by using the scale option. For instance, to scale the frontend service:

docker-compose up --scale frontend=3

This command spins up three instances of the frontend service, helping to handle increased traffic effectively.

Monitoring and Logging

Logs provide vital insights once applications are in production. Configure logging for each Docker service within docker-compose.yml. Each service uses a json-file logging driver, which facilitates the collection and management of logs.

Example: Logging Configuration

logging:
  driver: json-file
  options:
    max-size: "50m"
    max-file: "6"

This setup helps in managing log sizes and reducing storage overhead.

Conclusion

Scaling pingcap/autoflow in production requires thoughtful planning around container orchestration, environment management, and monitoring.

Source

The information used in this document is drawn from the provided code snippets and context.