Observability

“What did the AI actually do, and what did it cost?” Answered.

Every prompt, every tool call, every state transition and every dollar — attributed to the issue, the agent and the repo, recorded as the work happens. When finance, security or your own engineers ask what the pipeline did, the answer is a query, not a post-mortem.

Cost ledger

Every dollar attributed.

Each prompt, each tool call, each completion is logged with its model, its token count, and its dollar amount. The ledger rolls up to the issue, the agent, the tenant, and the time window.

  • Per-issue rollup with full prompt-level breakdown
  • Per-agent comparison across issues
  • Per-tenant attribution for multi-team workspaces
  • CSV / API export for finance and BI tooling
cost_ledger · by agent last 30 days
Cost by agent, last 30 days
AgentShare$
Inspector44.4%$3,570.64
Builder28.9%$2,320.95
Surveyor20.0%$1,609.33
Chronicler3.9%$311.68
Architect2.4%$189.28
Marshal0.5%$36.31

Colony’s own pipeline, last 30 days: 2,787 merged PRs. $2.06 median per PR, last 30 days. Live figures at /built-by-colony/. Column-level detail on the ledger page.

Audit trail

Every decision replayable.

Open any issue and replay its full history: what each agent saw, what it decided, what it produced, who approved the merge. Not a log file — a record, retained per your policy.

  • Every prompt and every response retained
  • Tool call inputs and outputs preserved
  • Replayable transition timeline per issue
  • Retention configurable per tenant (default: lifetime)
audit.issue(<id>) entry shape
timestampMayortask enqueued
timestampSurveyoranalysis complete, state changed
timestampArchitectplan posted
timestampBuilderPR opened
timestampInspectorreview outcome recorded
timestampMarshalmerged
Your data, your tools

Queryable Postgres, not a vendor dashboard.

The ledger and audit trail are stable, documented views. Wire them into your BI tool, alert on spend per agent, scrape the Prometheus endpoint into the monitoring stack you already run.

About the API reference
// query the ledger for an agent's spend
SELECT agent, SUM(cost_usd)
FROM   cost_ledger
WHERE  tenant = 'colony'
  AND  ts >= now() - '7 days'
GROUP BY agent
ORDER BY 2 DESC;
Operational dashboard

The dashboard exists for exceptions, not monitoring theater.

Blocked, awaiting clarification, awaiting review — each with the action beside it. Everything else is the pipeline doing its job.

dashboard · pipeline needs attention
Review ReadyPR waiting on a human
Blockedoperator must unblock, retry or cancel
Needs Clarificationauthor must answer

See the receipts before you sign anything.

Colony’s own development runs through this same ledger, and it is public.