Platform¶
Blackrack Labs runs on a hybrid Kubernetes cluster: a managed cloud control plane plus workers in AWS, and a Proxmox virtual machine in Bogotá used as the low-cost self-hosted tier. Everything — agents, gateways, apps, databases, observability — runs on this cluster as code.
The five pillars¶
- LLM gateway — LiteLLM exposes every model behind one OpenAI-compatible endpoint; the routing policy turns provider selection into fixed workload tiers.
- Provider Broker — the scoped capability gateway through which every agent reaches search, voice, media, and browser tools without holding provider credentials.
- Identity — Authentik is the single sign-on for every human-facing service.
- Secrets — Infisical is the source of truth for credentials, projected into Kubernetes and rotated without touching agents.
- Storage & data — managed PostgreSQL (RDS) for stateful services, Longhorn for cluster-local volumes, S3 for backups.
Design rules that shape everything¶
- Agents never hold upstream provider keys. Capabilities arrive through the broker with a scoped token; model calls arrive through LiteLLM with a per-tenant virtual key. (Security model)
- Providers are interchangeable. A model alias is a policy, not a dependency. Every tier has an explicit, audited fallback chain.
- Live state must be reproducible from git. A change is not done until it is in the manifests, the sealed secrets, and the changelog.
- Fail closed. Missing config, missing keys, or unknown hosts mean the request is denied — never passed through.
Cluster at a glance¶
| Layer | Technology | Role |
|---|---|---|
| Control plane | k3s on AWS EC2 (arm64) | Kubernetes control plane only (no storage) |
| Worker (cloud) | AWS EC2 (x86) | Infra services and the Longhorn data node |
| Workers (self-hosted) | Proxmox VM (Bogotá) + browser worker | The whole agent gateway fleet; the self-hosted browser |
| Networking | Caddy ingress, Tailscale mesh | Public entry, zero-trust admin access |
| Storage | local-path (gateways), Longhorn (AWS volumes), RDS PostgreSQL, S3 | Agent state, platform volumes, relational data, backups |
| Provisioning | Terraform + GitHub Actions | Infrastructure as code, deploy pipelines |
The cluster runs services in dedicated namespaces: the agent platform, the self-hosted app suite, observability, and the CI/CD control plane.