HelixML Architecture
The Helix project (https://github.com/helixml/helix/) utilizes several key technologies and dependencies, including Go, Docker, Git, PostgreSQL, and others. This document will explain the architecture of HelixML, focusing on its main components and their interactions.
The Big Picture
HelixML is designed as a microservices-based architecture, with each service having a specific role and communicating with others through APIs. The main services in HelixML are:
- CurveBS: A scalable and reliable block storage service built on top of PostgreSQL. It provides data storage and caching functionality. More details can be found in the Curve documentation (https://github.com/opencurve/curve).
- CurveAdm: A tool for deploying and managing Curve clusters. It supports the deployment of CurveBS and CurveFS, with scaling, upgrading, and other functions under development. Refer to the CurveAdm documentation (https://curvebs.readthedocs.io/en/latest/) for more information.
- API Gateway: An entry point for external services to interact with HelixML. It uses Gorilla Router for routing requests and JWT for authentication.
- Go Mocking Framework: A library for generating mock objects during testing.
- Gorilla Websocket: A WebSocket library for real-time communication between services.
- Material UI: A UI library for building web interfaces.
Design Philosophy
HelixML follows a few design principles:
- Modularity: Each service is designed to be independent and have minimal dependencies on other services. This allows for better scalability and maintainability.
- Automation: HelixML uses tools like Docker and Kubernetes to automate deployment, scaling, and management of services.
- Security: Services communicate through secure APIs, using JWT for authentication and TLS for encryption.
Programming Languages
HelixML primarily uses Go as its programming language, taking advantage of its simplicity, performance, and strong support for concurrency. The project also uses HCL2 for Packer templates and SQL for database interactions.
Go Programming Language
Go, also known as Golang, is a statically typed, compiled language developed by Google. It is designed for simplicity, concurrency, and productivity. HelixML uses Go for its main services, such as CurveBS and CurveAdm.
Docker
Docker is a platform for developing, shipping, and running applications using container technology. HelixML uses Docker to package and distribute its services, ensuring consistent execution across different environments.
Git
Git is a distributed version control system that HelixML uses for managing its source code. The project’s main repository is hosted on GitHub (https://github.com/helixml/helix/).
PostgreSQL
PostgreSQL is a powerful, open-source object-relational database system. HelixML uses PostgreSQL as the primary data storage for CurveBS, providing data storage and caching functionality.
Go Mocking Framework
The Go Mocking Framework is a library for generating mock objects during testing. It allows developers to create test doubles for interfaces, making it easier to test code in isolation.
Gorilla Router
Gorilla Router is a HTTP request router and dispatcher for building Go web servers. HelixML uses Gorilla Router to route incoming requests to the appropriate handler functions.
Gorilla Websocket
Gorilla Websocket is a WebSocket library for building real-time, bidirectional communication between web clients and servers. HelixML uses Gorilla Websocket for real-time data transfer between services.
Material UI
Material UI is a popular React UI framework that follows Google’s Material Design guidelines. HelixML uses Material UI to build its web interfaces, ensuring a consistent and modern look and feel.
JWT
JSON Web Tokens (JWT) are a compact, URL-safe means of representing claims to be transferred between two parties. HelixML uses JWT for authentication, allowing services to securely identify clients and grant access to protected resources.
Go CMP
Go CMP (https://github.com/helixml/cmp) is a custom HelixML component for managing configuration and metadata. It is used by CurveAdm to manage Curve cluster configurations.
Examples
CurveBS Deployment
To deploy CurveBS using CurveAdm, follow these steps:
- Install CurveAdm:
curl -L https://raw.githubusercontent.com/curvebs/curveadm/master/install.sh | sh
- Create a cluster topology file for standalone deployment:
# curvebs_standalone.yaml
apiVersion: curveadm.io/v1alpha1
kind: ClusterTopology
metadata:
name: curvebs-standalone
spec:
replicas: 1
zone:
- name: zone1
replicas: 1
meta:
region: us-west-1
- Deploy CurveBS:
curveadm deploy -f curvebs_standalone.yaml
Packer HCL2 Template
Here’s an example of a Packer HCL2 template that builds a Docker image:
packer {
required_plugins {
docker = {
version = ">= 1.0.0"
}
}
}
source "docker" "my-docker-image" {
image = "my-base-image:latest"
commit = true
provisioner "shell" {
inline = [
"echo 'Hello, World!'"
]
}
}
build {
sources = [
"source.docker.my-docker-image"
]
}
Conclusion
HelixML is a microservices-based architecture that utilizes various technologies and dependencies to provide a scalable and reliable block storage service. By following design principles such as modularity, automation, and security, HelixML ensures a robust and maintainable system.