Skip to content

Audit Log

DeltaLLM writes audit events to Postgres for admin actions and selected runtime operations so operators can investigate changes, access, and failures later.

Quick Path

Audit logging is enabled by default. To confirm it is working:

  1. Sign in to the Admin UI or use the master key
  2. Perform an action such as creating a key or running a batch job
  3. Open Audit Logs or call the audit API
  4. Filter by request_id, actor, or action

List recent events:

curl http://localhost:8000/ui/api/audit/events \
  -H "Authorization: Bearer YOUR_MASTER_KEY"

What Gets Audited

DeltaLLM records:

  • control-plane actions from the Admin UI and /ui/api/*
  • selected data-plane actions such as files, batches, rerank, images, audio, and spend access
  • self-service key operations performed by team developers

The audit log is meant for operational review, security investigations, and compliance workflows.

For production, set audit_ingestion_mode: outbox. In the default legacy mode, the system stores required audit events and prompt-render records synchronously. It fails closed when persistence is unavailable. Only best-effort audit events use the bounded, non-durable compatibility queue.

Durable required events keep a server-owned ID and use renewable fenced claims. After all retry attempts fail, these events move to blocked. The system does not delete them. Platform administrators can replay an investigated blocked record through the telemetry-ingestion replay endpoint. See the rollout guide. The replay and its required operator audit commit in the same database transaction.

Best-effort audit records are not authoritative. If queue capacity or an audit dependency is unavailable, the system drops and counts the best-effort record. This does not change a request, MCP tool, notification, or callback result. Required audit still fails closed with a controlled service-unavailable response. Before it does, a required enqueue that hit a lock or connection-pool wait past its bound is retried once after a short random delay. The event ID stays the same, so the retry cannot create a duplicate. Outages, such as a lost database connection, are not retried. Each retry increments deltallm_audit_write_failures_total with a *_contention_retry path. MCP tool attempts require audit events. If the audit service is unavailable, the gateway rejects the attempt before it calls the remote tool transport.

Self-Service Key Audit Actions

When team developers create or manage their own keys, DeltaLLM records distinct audit actions:

Action When
ADMIN_KEY_SELF_CREATE Developer creates a self-service key
ADMIN_KEY_SELF_ROTATE Developer regenerates their own key
ADMIN_KEY_SELF_REVOKE Developer revokes their own key

These events include the actor_id (the developer's account ID) and resource_id (the key token hash), allowing admins to trace all self-service activity.

Who Can Read It

Audit access requires the master key or an authenticated admin session with audit.read.

That includes:

  • platform admins
  • org owners
  • org admins
  • org auditors

Team-only roles do not get audit access by themselves.

Main Endpoints

All audit endpoints are under /ui/api/audit/*.

Endpoint Purpose
GET /ui/api/audit/events List events with filters and pagination
GET /ui/api/audit/events/{event_id} Get one event, including stored payload records
GET /ui/api/audit/events/{event_id}/payloads Get only the stored payload records of one event
GET /ui/api/audit/events/{event_id}/related Events around one event: same request, same resource, recent sign-ins
GET /ui/api/audit/actions Every known action with its category
GET /ui/api/audit/organizations Organizations whose audit log the caller can read
GET /ui/api/audit/timeline Show all events for a request or correlation id
GET /ui/api/audit/export?format=jsonl|csv Export filtered results

Useful filters on GET /ui/api/audit/events include:

  • category (admin, auth, api, mcp, spend, system)
  • action
  • status
  • actor (an email, API key name, key hash, or actor ID) and actor_type
  • organization_id
  • resource_type and resource_id
  • ip
  • request_id
  • correlation_id
  • search (an event ID, request ID, or resource ID)
  • since and until (timestamps), or start_date and end_date (whole days)

Pass count=false to skip the total count and page with cursor. See the Audit API for paging and the enrich and context options.

Event Shape

Common event fields include:

  • event_id
  • occurred_at
  • organization_id
  • actor_type
  • actor_id
  • action
  • resource_type
  • resource_id
  • request_id
  • correlation_id
  • status
  • latency_ms
  • input_tokens
  • output_tokens
  • error_type
  • error_code
  • metadata
  • content_stored

When you fetch one event directly, DeltaLLM also returns any stored payloads.

Payload Storage and Redaction

Audit payloads are handled differently depending on the event type:

  • control-plane payloads are redacted for sensitive values such as passwords, tokens, and secrets
  • data-plane payload content is stored only when the organization has audit_content_storage_enabled = true
  • One PostgreSQL transaction disables content storage and redacts active or blocked outbox envelopes. Legacy and durable writers acquire the same organization policy lock. Writers that only read the policy share that lock, so a worker batch never delays a request's audit event. Changing the policy takes the lock exclusively and waits for those readers to finish. After lock acquisition, they read policy in a fresh statement. Durable workers also remove prohibited content from claimed envelopes before completion.
  • Redis invalidation removes stale policy-cache entries across replicas, but privacy correctness does not depend on Pub/Sub availability
  • when content storage is disabled, the event is still recorded but request and response bodies are omitted or marked redacted

Retention

Audit retention runs in the background.

Global defaults:

  • general_settings.audit_metadata_retention_days
  • general_settings.audit_payload_retention_days

Per-organization overrides:

  • metadata.audit_metadata_retention_days
  • metadata.audit_payload_retention_days

Upgrade Note: Resource History Index

The Audit Logs event panel lists recent events on the same resource. A migration adds the index deltallm_auditevent_resource_occurred_at_idx on (resource_type, resource_id, occurred_at). The migration runs in a transaction and blocks audit writes while it builds. On a large audit table, create the index concurrently before you deploy, so the migration finds it and does nothing:

CREATE INDEX CONCURRENTLY IF NOT EXISTS "deltallm_auditevent_resource_occurred_at_idx"
  ON "deltallm_auditevent"("resource_type", "resource_id", "occurred_at");

If the migration times out because the table is busy, create the index as above, mark only that attempt rolled back, and deploy again:

uv run prisma migrate resolve --rolled-back 20261014120000_audit_resource_occurred_index \
  --schema prisma/schema.prisma
uv run prisma migrate deploy --schema prisma/schema.prisma