Production monitoring for a Dockerized application involves a series of steps to ensure that the services are efficiently running, logs are captured, and metrics are collected. This section will guide the user through the process of monitoring the Docker containers in production for the application defined in the provided docker-compose.yml and Dockerfile.

1. Set Up Monitoring Tools

To effectively monitor Docker containers in a production environment, select appropriate tools such as Prometheus for metrics collection and Grafana for visualization, or use built-in Docker monitoring tools like Docker stats. Below is a setup demonstrating how to integrate Prometheus and Grafana with the Docker setup.

Docker Compose Configuration

Modify the docker-compose.yml file to add Prometheus and Grafana services:

version: "2"
services:
  # Existing services...

  prometheus:
    image: prom/prometheus
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"
  
  grafana:
    image: grafana/grafana
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin

Prometheus Configuration

Create a prometheus.yml file to scrape metrics from your services:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'docker-services'
    static_configs:
      - targets: ['server:8080', 'signer:8080', 'mysql:3306']

Running the Containers

After modification, run your containers:

docker-compose up -d

2. Log Management

Centralized logging is crucial for monitoring. Log entries should be directed to a log management system such as ELK Stack (Elasticsearch, Logstash, Kibana) or a simpler approach using Fluentd.

Example Logging with Fluentd

Add Fluentd to your docker-compose.yml:

  fluentd:
    image: fluent/fluentd
    ports:
      - "24224:24224"
    volumes:
      - ./fluent.conf:/fluentd/etc/fluent.conf

Fluentd Configuration File

Create a fluent.conf file to configure the input and output sources:

<source>
  @type forward
  port 24224
</source>

<match **>
  @type elasticsearch
  host elasticsearch
  port 9200
</match>

3. Health Checks

Implement health checks in the docker-compose.yml to automate monitoring of the application state:

services:
  server:
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  signer:
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3

4. Command-Line Monitoring

Docker CLI allows for real-time monitoring of running containers. Use the following commands to monitor container resource usage:

docker stats

This command provides information regarding CPU and memory usage of all running containers.

5. Custom Metrics Collection

For advanced monitoring scenarios, consider implementing custom metrics within the application using Go code, leveraging libraries such as Prometheus Go client.

Example of Custom Metrics in Go

Add the dependency to your Go module:

go get github.com/prometheus/client_golang/prometheus

Implement custom metrics in your service code:

package main

import (
    "net/http"
    "github.com/prometheus/client_golang/prometheus"
    "github.com/prometheus/client_golang/prometheus/promhttp"
)

var (
    requestCount = prometheus.NewCounterVec(
        prometheus.CounterOpts{
            Name: "http_requests_total",
            Help: "Total number of HTTP requests",
        },
        []string{"method"},
    )
)

func init() {
    prometheus.MustRegister(requestCount)
}

func handler(w http.ResponseWriter, r *http.Request) {
    requestCount.WithLabelValues(r.Method).Inc()
    // Your handler logic
}

func main() {
    http.Handle("/metrics", promhttp.Handler())
    http.ListenAndServe(":8080", nil)
}

With the implementation above, the application will expose metrics at /metrics, which can then be scraped by Prometheus.

6. Monitoring Alerts

Finally, set up alerts for failure conditions or performance issues using Prometheus Alertmanager. Define alert rules in prometheus.yml:

groups:
- name: example_alert
  rules:
  - alert: HighRequestRate
    expr: http_requests_total > 100
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "High request rate detected"

Reference Code

All configurations and Go code above incorporate the practices suitable for monitoring a Docker environment based on the provided code structure and intended application architecture.

Sources: The configurations and code samples above are derived from the provided docker-compose.yml, Dockerfile, and standard practices in Docker production environments.