Overview

When scaling the sourcegraph/zoekt project for production, several considerations must be taken into account. This documentation outlines a step-by-step guide for expert developers on how to effectively scale the project in a production environment, focusing on the Dockerfile, build processes, and dependency management.

1. Building a Production Ready Docker Image

The first step in scaling is to create a robust Docker image that encapsulates all the necessary dependencies and configurations for the zoekt application. Below is an example Dockerfile that illustrates the two-stage build process typically used to create the production image.

FROM golang:1.22.2-alpine3.19 AS builder

RUN apk add --no-cache ca-certificates

ENV CGO_ENABLED=0
WORKDIR /go/src/github.com/sourcegraph/zoekt

# Cache dependencies
COPY go.mod go.sum ./
RUN go mod download

COPY . ./
ARG VERSION
RUN go install -ldflags "-X github.com/sourcegraph/zoekt.Version=$VERSION" ./cmd/...

FROM rust:alpine3.19 AS rust-builder

RUN apk add --no-cache git wget musl-dev build-base

RUN wget -qO- https://github.com/sourcegraph/sourcegraph/archive/0c8aa18eece45922a2b56dc0f94e21b1bb533e7d.tar.gz | tar xz && mv sourcegraph-* sourcegraph

ARG TARGETARCH

# Compile the syntactic highlighter with Cargo
RUN cd sourcegraph/docker-images/syntax-highlighter && /sourcegraph/cmd/symbols/cargo-config.sh && cd /

RUN cargo install --path sourcegraph/docker-images/syntax-highlighter --root /syntect_server --bin scip-ctags

FROM alpine:3.19 AS zoekt

RUN apk add --no-cache git ca-certificates bind-tools tini jansson wget

COPY install-ctags-alpine.sh .
RUN ./install-ctags-alpine.sh && rm install-ctags-alpine.sh

COPY --from=builder /go/bin/* /usr/local/bin/
COPY --from=rust-builder /syntect_server/bin/scip-ctags /usr/local/bin/scip-ctags

ENTRYPOINT ["/sbin/tini", "--"]

Explanation of Dockerfile Structure

1. Base Image: Start with the Golang base image to build the application.

2. Dependency Caching: Dependencies are cached using Go modules to improve build efficiency.

3. Two-Stage Build:

   • Builder Stage: Compile the zoekt application and its required binaries.
   • Rust Builder Stage: Compile the Rust component of the application, specifically the syntax highlighter.

4. Final Image: Use an Alpine base image to keep the final image lightweight, copying necessary binaries from the builder stages.

2. Environment Variables and Build Flags

Utilizing environment variables and build flags within the Dockerfile can greatly enhance the flexibility of the application. The CGO_ENABLED variable is set to 0 to ensure static linking of Go binaries, which reduces dependency on external libraries in running environments.

Example Build Command

When building the application, ensure the module information impacts the output properly:

docker build --build-arg VERSION=1.0.0 -t zoekt:1.0.0 .

This command triggers the Docker build process and allows for versioning with the VERSION variable passed as a build argument.

3. Deploying in a Scalable Architecture

To scale Sourcegraph/Zoekt in a production environment, deploy the built Docker container using orchestrators like Kubernetes or Docker Swarm. This allows for horizontal scaling and load balancing. Below is an example Kubernetes Deployment configuration.

Example Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: zoekt
spec:
  replicas: 3
  selector:
    matchLabels:
      app: zoekt
  template:
    metadata:
      labels:
        app: zoekt
    spec:
      containers:
      - name: zoekt
        image: zoekt:1.0.0
        ports:
        - containerPort: 3070
        env:
        - name: ZOEKT_REPO
          value: "your-repo-url"
        resources:
          requests:
            memory: "512Mi"
            cpu: "0.5"
          limits:
            memory: "1Gi"
            cpu: "1"

Key Points

• Replicas: Setting multiple replicas ensures load balancing and redundancy.

• Resource Management: Proper memory and CPU requests/limits are necessary for effective scaling.

4. Monitoring and Logging

In a production environment, it’s vital to implement monitoring and logging solutions. Integrate tools like Prometheus, Grafana, or Loki to track application performance and troubleshoot issues effectively.

Example Configuration for Logging

Modify your Dockerfile to include logging capabilities, which can be beneficial for tracking errors and application behavior:

RUN apk add --no-cache logrotate

This allows for log rotation in a production setting, essential for maintaining performance and storage efficiency.

Conclusion

Scaling sourcegraph/zoekt in production requires careful planning, robust Docker configuration, and efficient deployment architectures. By following the practices outlined in this guide, expert developers can create scalable, manageable instances of the zoekt application, ensuring reliability and performance in high-demand environments.

Source: Dockerfile information provided in the documentation.