Use this path once per-queue autoscaling and zero-downtime rolling
deploys are worth the operational cost of running a cluster. For a
pilot or a small client base, Docker Compose on a VPS
is less to operate and functionally equivalent.
Chart structure
helm/seo-ai-os/ deploys:
- API deployment (
maxUnavailable: 0, preStop hook, tuned grace period — zero-downtime is a tested property, not an assumption) - Celery worker deployments, one per queue, each with its own HPA tuned to
that queue’s profile (
orchestration,agent_high,agent_low,integrations,system) - PostgreSQL + PgBouncer (primary/replica topology; replica promotion is currently manual by design)
- PodDisruptionBudgets (the worker PDB deliberately avoids the singleton-replica anti-pattern)
- NetworkPolicies
- A nightly Postgres backup CronJob to S3-compatible storage, 14-day retention
GitOps: staging auto, production manual
Staging syncs automatically on every merge tomain. Production is a
deliberate two-human action: scripts/promote-to-production.sh <git-sha> rewrites the production image tag on a new branch and prints
the push/PR commands — it does not push or merge for you. A reviewer
merges the PR; ArgoCD then shows production as out-of-sync until someone
explicitly runs argocd app sync.
Vault: production Raft cluster
Unlike local dev’s single dev-mode Vault container, production runs a 3-node Raft cluster with KMS auto-unseal.scripts/vault-bootstrap.sh
is the one-time setup — see CLI → Vault Bootstrap. Vault
Agent Injector sidecars render secrets to
/vault/secrets/config, which docker/entrypoint-api.sh sources into the
process environment before execing the real command.
Managed Redis is required, not optional
The chart provisions no Redis — a managed instance is assumed, and productionSettings validation refuses to boot if any Redis-backed URL
(redis_url, celery_broker_url, celery_result_backend) resolves to
localhost. This is deliberate: Redis backs the Celery broker, the event
bus, the cache, and the rate limiter — a silent localhost fallback would
mean a pod that boots and passes readiness, then fails every real request.
First deploy: run the checklist, don’t skip it
STAGING_CHECKLIST.md in the repository is the complete, sequenced
first-deploy procedure — pre-flight infra checks, first CI run, database
and RLS verification, the Redis/events/Celery soak (including a real
kubectl rollout restart mid-workflow to prove PEL reclaim works — see
Events), LLM provider
failover, the evaluator framework and circuit breaker’s first live
verification, observability, backup/restore, and a 7-day soak before
anything production-facing follows. It exists precisely because several of
these properties (Redis wiring, PEL reclaim across a real rolling deploy)
are the class of defect only a live cluster surfaces — a Helm template
lint and a code review cannot substitute for actually running it once.