Monitoring a Docker Compose project in production is critical to ensure application health, performance, and reliability. Below is a step-by-step guide detailing how to monitor the project effectively.

Observability Setup

To monitor the application running in Docker Compose, incorporating observability tools is essential. Commonly used tools include Prometheus for metrics gathering, Grafana for visualization, and logs aggregation services like ELK Stack or Fluentd.

Code Integration

Prometheus Configuration

To enable Prometheus monitoring, modify your docker-compose.yml to include a Prometheus service and exporter. The following snippet demonstrates how to set this up:

services:
  frontend:
    image: nginx
    container_name: frontend
    volumes:
      - project-data:/data

  prometheus:
    image: prom/prometheus
    container_name: prometheus
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    networks:
      - monitoring

volumes:
  project-data:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: "${TEST_DIR}"

networks:
  monitoring:
    driver: bridge

Prometheus Configuration File

The prometheus.yml file configures Prometheus to scrape metrics from your service. An example configuration could look like this:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'frontend'
    static_configs:
      - targets: ['frontend:80']  # Adjust accordingly based on your service's metrics endpoint.

Application Metrics

In your Go application, ensure that you are exposing an endpoint that provides metrics in the Prometheus format. Use the prometheus/client_golang library to create him.

Install the package:

go get github.com/prometheus/client_golang/prometheus
go get github.com/prometheus/client_golang/prometheus/promhttp

Example code for exposing metrics:

package main

import (
    "net/http"
    "github.com/prometheus/client_golang/prometheus"
    "github.com/prometheus/client_golang/prometheus/promhttp"
)

var (
    requestCount = prometheus.NewCounterVec(
        prometheus.CounterOpts{
            Name: "http_requests_total",
            Help: "Total number of HTTP requests",
        },
        []string{"path"},
    )
)

func init() {
    prometheus.Register(requestCount)
}

func handler(w http.ResponseWriter, r *http.Request) {
    requestCount.WithLabelValues(r.URL.Path).Inc()
    // Handle request
}

func main() {
    http.Handle("/metrics", promhttp.Handler())
    http.HandleFunc("/", handler)
    http.ListenAndServe(":8080", nil)
}

Log Monitoring

For log aggregation, consider integrating the ELK Stack (Elasticsearch, Logstash, and Kibana) or a simpler logging framework. The following demonstrates how to set up a basic logging structure using Docker Compose.

Adding ELK Stack

Extend your docker-compose.yml:

services:
  elasticsearch:
    image: elasticsearch:7.9.2
    environment:
      - discovery.type=single-node
    ports:
      - "9200:9200"

  logstash:
    image: logstash:7.9.2
    ports:
      - "5044:5044"
    volumes:
      - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf

  kibana:
    image: kibana:7.9.2
    ports:
      - "5601:5601"

Logstash Configuration

Create a logstash.conf to configure the input and output of logs:

input {
  docker {
    port => "5044"
    type => "docker-logs"
  }
}

output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
  }
}

Conclusion

By integrating monitoring and logging tools into the Docker Compose setup, you ensure a robust production environment where you can easily observe and respond to application behavior and performance metrics.

For detailed examples and more configurations, refer to the Prometheus and ELK Stack documentation.

Sources: Prometheus Documentation, ELK Stack Documentation.