helixml / helix

Monitoring and Logging

Logging Levels

HelixML supports various logging levels to control the verbosity of logs. The levels are defined in the logging module, based on the standard Python logging module.

  • DEBUG: Logs detailed information for debugging purposes.

  • INFO: Logs general information about the application’s state.

  • WARNING: Logs events that may indicate potential problems.

  • ERROR: Logs errors that have occurred.

  • CRITICAL: Logs critical errors that may cause application failure.

Example:

import logging
          
          # Set the logging level to DEBUG
          logging.basicConfig(level=logging.DEBUG)
          
          # Log a message at the DEBUG level
          logging.debug("This is a debug message.")
          
          # Log a message at the INFO level
          logging.info("This is an informational message.")
          
          # Log a message at the WARNING level
          logging.warning("This is a warning message.")
          
          # Log a message at the ERROR level
          logging.error("This is an error message.")
          
          # Log a message at the CRITICAL level
          logging.critical("This is a critical error message.")

Log Formats

HelixML allows customizing the format of log messages.

Example:

import logging
          
          # Define a custom log format
          log_format = "%(asctime)s - %(levelname)s - %(filename)s:%(lineno)d - %(message)s"
          
          # Configure logging with the custom format
          logging.basicConfig(level=logging.INFO, format=log_format)
          
          # Log a message with the custom format
          logging.info("This is an informational message.")

Log Storage

HelixML supports logging to different destinations, including:

  • Console: Logs are printed to the console.

  • File: Logs are written to a file.

  • Database: Logs can be stored in a database for long-term persistence.

Example:

import logging
          
          # Configure logging to a file
          logging.basicConfig(level=logging.INFO, filename="my_app.log")
          
          # Log a message to the file
          logging.info("This message will be written to my_app.log.")
          
          # Configure logging to a database (example using SQLite)
          import logging.handlers
          
          # Create a handler for logging to a database
          handler = logging.handlers.RotatingFileHandler("my_app.log", maxBytes=1024 * 1024 * 5, backupCount=5)
          
          # Create a formatter for the log messages
          formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(filename)s:%(lineno)d - %(message)s")
          
          # Set the formatter for the handler
          handler.setFormatter(formatter)
          
          # Add the handler to the logger
          logging.getLogger().addHandler(handler)
          
          # Log a message to the database
          logging.info("This message will be written to the database.")

Monitoring

HelixML provides monitoring capabilities to track the health and performance of the application.

  • Metrics: HelixML utilizes libraries like Prometheus to collect and expose metrics such as CPU usage, memory consumption, and request counts.

  • Alerts: Alerts can be configured to notify developers or administrators when certain metrics exceed thresholds.

Example:

# Example using Prometheus client
          from prometheus_client import Gauge
          
          # Create a gauge metric for CPU usage
          cpu_gauge = Gauge("cpu_usage", "CPU usage percentage")
          
          # Update the metric with the current CPU usage
          cpu_gauge.set(50.0)