Metrics Reference

The complete pipeline from raw agent usage to the numbers on your dashboard — nothing hidden.

0 → 100: How a cost figure gets to your dashboard

Walk through the full pipeline for a single LLM call. Every step is listed; none are skipped.

Step 1: LLM call completes

Your agent calls a provider (e.g. openai.chat.completions.create(model="gpt-4o", ...)). The provider returns a response including token counts in the API response body.

// Provider API response (simplified)
{
  "usage": {
    "prompt_tokens": 512,
    "completion_tokens": 284,
    "total_tokens": 796
  }
}

Step 2: OTel span captured by SpineforgeSpanProcessor

The OpenLLMetry instrumentor wraps the API call as an OTel span. When the span ends, SpineforgeSpanProcessor.on_end() receives it. The processor extracts: provider, model, token counts, duration, run_id, and spine_id.

Step 3: Rate card lookup

The cost engine queries the rate_cards table to find the rate in effect at the moment this action is being written:

SELECT cost_per_1k_prompt_tokens,
       cost_per_1k_completion_tokens,
       id AS rate_card_id
FROM   rate_cards
WHERE  provider        = 'openai'
  AND  model           = 'gpt-4o'
  AND  effective_from <= NOW()
ORDER  BY effective_from DESC
LIMIT  1;

The rate card with the most recent effective_from date that is still in the past is used. If a provider changes pricing, the new rate card takes effect from effective_from forward — old actions are not re-priced.

Step 4: Cost computed once

Cost is computed immediately, in the application layer, before the DB write:

cost_usd = (
    (prompt_tokens    / 1000) * cost_per_1k_prompt_tokens
  + (completion_tokens / 1000) * cost_per_1k_completion_tokens
)

For the example above (512 prompt + 284 completion tokens on gpt-4o at 2025 Q1 rates):

cost_usd = (512/1000 × 0.005) + (284/1000 × 0.015)
         = 0.00256 + 0.00426
         = 0.00682 USD

Step 5: Action row written to DB

The action is written to the actions table with the computed cost and the rate card ID:

INSERT INTO actions (
  action_id, run_id, spine_id, type, name,
  duration_ms, prompt_tokens, completion_tokens,
  cost_usd,          -- NUMERIC(14,8) — stored, not recomputed
  rate_card_id,      -- FK → rate_cards.id — audit trail
  status, started_at, ended_at
) VALUES (..., 0.00682000, 'rc_gpt4o_2025q1', ...)

This is the only time cost is computed for this action. Loading the dashboard tomorrow, next week, or next year reads this stored value — it is never recalculated.

Step 6: Run total denormalized at run-end

When spine.run() exits, Spineforge upserts the run's total cost onto the runs table:

UPDATE runs
SET    total_cost_usd = (
           SELECT SUM(cost_usd)
           FROM   actions
           WHERE  run_id = :run_id
       ),
       ended_at = NOW(),
       status   = 'success'
WHERE  run_id = :run_id;

This denormalized total is stored once. Dashboard cost aggregations read from runs.total_cost_usd, not from SUM(actions.cost_usd) on every load.

Step 7: Audit trail — tracing any cost figure back to its rate

Every action row stores rate_card_id. You can always trace any cost figure:

SELECT a.action_id, a.cost_usd, r.model,
       r.cost_per_1k_prompt_tokens,
       r.cost_per_1k_completion_tokens,
       r.effective_from
FROM   actions   a
JOIN   rate_cards r ON r.id = a.rate_card_id
WHERE  a.action_id = 'act_abc123';

P50 / P95 Latency — computed live (different from cost)

Latency percentiles are the one exception to the computed-once pattern. They are computed on-the-fly by Postgres using window functions:

-- P50 and P95 latency for an agent over the last 30 days
SELECT
  percentile_cont(0.50) WITHIN GROUP (ORDER BY duration_ms) AS p50_ms,
  percentile_cont(0.95) WITHIN GROUP (ORDER BY duration_ms) AS p95_ms
FROM   actions
WHERE  spine_id    = :spine_id
  AND  type        = 'llm_call'
  AND  started_at >= NOW() - INTERVAL '30 days';

Why this is different from cost: Latency percentiles are inherently a property of a time window and a dataset — they don't have a meaningful "computed once" value. Adding a new action today changes the P95. Cost, by contrast, is a fact about a past event — it happened at a specific rate, and that rate should be frozen.

Practical implication: The P95 you see on the dashboard reflects the current dataset as of now. If you add more data, it may shift. Cost figures will not shift.

Metrics summary table

Rate cards

Rate cards are maintained by Spineforge and updated when providers change their pricing. Each card has:

Next steps

• Pricing & Billing — how Spineforge billing tiers map to usage
• Dashboard Guide — reading cost and latency figures in the UI