The stack
These ports are for local development. On a shared or public deployment,
bind them to 127.0.0.1 or put them behind your reverse proxy with
authentication — none of them should be directly internet-reachable with
default credentials.
Metrics: 36 series across 7 subsystems
core/observability/metrics.py defines the full catalog — API, workflow,
agent, event, celery, database, and cache metrics — on one Prometheus
registry, namespaced seo_*.
Worker metrics need multiprocess collection
The API exposes /metrics via a single ASGI mount — one process, one
registry. Celery workers are not one process: the default prefork pool
forks multiple children per worker container, each with independent
memory. core/observability/worker_metrics.py wires
prometheus_client’s multiprocess pattern
onto Celery’s worker_init/worker_shutdown signals — set
PROMETHEUS_MULTIPROC_DIR and one aggregating HTTP endpoint opens in the
worker’s parent process (never a forked child, which would collide on
the port). Without this env var set, worker-side metrics (most agent, LLM,
and Celery metrics) simply aren’t exported — this is opt-in specifically
so local make worker-* and the test suite are unaffected.
27 alert rules, cross-checked in CI
docker/observability/alerts/seo-alerts.yml defines every alert;
alerts/README.md maintains a coverage-map table — every metric in
metrics.py is either alerted, or explicitly marked not-alerted with a
stated reason. scripts/verify/check_alert_coverage.py and
check_dashboard_metrics.py enforce this stays true on every push (the
verify-observability CI job) — see
CLI → Verification Scripts.
Redis: noeviction, deliberately
The same Redis instance backs the Celery broker, the event bus, the
registry cache, and the rate limiter. allkeys-lru could silently evict
an undelivered broker or stream key under memory pressure — real task or
event loss with no error anywhere. noeviction instead fails new writes
loudly once the memory ceiling is hit:
