When you run coding agents in production, your LLM API bill becomes a black box. Two sessions that look identical from the outside can cost wildly different amounts. Frugal Tokens is a Deno-based observability tool that instruments agent sessions to expose what actually drives spend: cache misses, overlapping sessions, context window expiration, and per-model token consumption.
The tool captures model calls, tool I/O, and cache events without breaking the execution loop. It gives you session-level metrics, percentile breakdowns, and a cost comparison feature that models hypothetical pricing across different LLMs and caching strategies.
Why Agent Cost Observability Matters
Coding agents make dozens of model calls per session. Each call can hit or miss the prompt cache. Long sessions can expire context windows and force re-embedding. Multiple overlapping sessions can amplify spend without warning.
Without instrumentation, you see the aggregate bill but not the breakdown:
- Which sessions burned through cache TTLs?
- How much did overlapping sessions cost during peak hours?
- What would the same session cost with a different model or caching strategy?
Frugal Tokens answers these questions by recording every model call, tool invocation, and cache event. The demo at https://demo.frugaltokens.com/ shows scrubbed data from real sessions.
Architecture: Instrumentation Without Blocking
The tool runs as a Deno process that wraps your agent runtime. It intercepts API calls to LLM providers and logs:
- Model name and token counts (input, output, cached)
- Tool calls and their inputs/outputs
- Cache hit or miss status
- Timestamp and session ID
The instrumentation layer sits between your agent orchestrator and the LLM API. It does not modify the agent’s execution flow. It records events to a local SQLite database and exposes a web UI for analysis.
// Simplified instrumentation hook
async function interceptModelCall(
sessionId: string,
model: string,
prompt: string,
tools: Tool[]
) {
const startTime = Date.now();
const response = await llmProvider.call(model, prompt, tools);
await db.insert('model_calls', {
session_id: sessionId,
model: model,
input_tokens: response.usage.input_tokens,
output_tokens: response.usage.output_tokens,
cached_tokens: response.usage.cache_read_tokens || 0,
cache_miss: response.usage.cache_creation_tokens > 0,
duration_ms: Date.now() - startTime,
timestamp: new Date().toISOString()
});
return response;
}The key is that the hook returns the original response unchanged. The agent sees no difference. The database write happens asynchronously.
Cache Miss Explorer: When Context Windows Expire
The session explorer shows individual model calls in chronological order. You can jump directly to cache misses. This reveals patterns:
- Cold starts: First call in a session always misses.
- TTL expiration: Anthropic’s 5-minute cache TTL expires during long debugging sessions.
- Context window overflow: When the prompt grows beyond the cache window, the model re-embeds everything.
The explorer displays:
- Input token count (cached vs. uncached)
- Output token count
- Tool calls made during that turn
- Cost estimate for that single call
You can see exactly when a session transitions from cached to uncached. For example, a 20-minute debugging session might show 8 cache hits, then a miss at minute 6 when the TTL expires, then another miss at minute 12 when the context window overflows.
Session Overlap and Estimated Working Time
The tool tracks when multiple sessions run concurrently. This matters because:
- Overlapping sessions share no cache state.
- Each session pays full token costs.
- Peak hours can amplify spend without warning.
The dashboard shows:
- Total session count
- Estimated working time (sum of session durations)
- Actual wall-clock time
- Overlap factor (working time / wall-clock time)
An overlap factor of 2.5 means you ran 2.5 sessions on average at any given time. If your sessions cost $0.50 each, your hourly spend is $1.25, not $0.50.
Cost Comparison: Hypothetical Pricing Models
The cost comparison feature takes a recorded session and recalculates what it would cost with:
- Different model pricing (GPT-4 vs. Claude vs. Gemini)
- Different cache TTLs (Anthropic’s 5-minute vs. 1-hour)
- Different cache strategies (no cache, prompt cache, full cache)
This is useful for:
- Evaluating model switches before committing
- Understanding cache TTL trade-offs
- Estimating cost impact of architectural changes
The comparison table shows:
| Model | Base Cost | With 5m Cache | With 1h Cache | Delta |
|---|---|---|---|---|
| GPT-4 Turbo | $2.40 | $1.80 | $1.20 | -50% |
| Claude 3.5 Sonnet | $1.80 | $1.20 | $0.90 | -50% |
| Gemini 1.5 Pro | $1.50 | N/A | N/A | N/A |
The delta column shows potential savings. The comparison uses actual token counts from the recorded session, so the estimates are grounded in real usage.
Installation and Data Flow
Installation is a single Deno command:
deno install -A -n frugal-tokens https://deno.land/x/frugal_tokens/cli.tsThe tool runs as a local web server. It does not send data to external services. The SQLite database stays on your machine.
Data flow:
- Agent makes LLM API call
- Instrumentation hook intercepts and logs
- Original response returns to agent
- Web UI queries SQLite for analysis
The web UI refreshes in real time. You can watch sessions as they run.
Failure Modes and Observability Gaps
Database write failures: If SQLite locks or the disk fills, the instrumentation hook can block the agent. The tool should fail open and log errors instead of crashing the agent.
Cache attribution errors: The tool assumes cache tokens are reported correctly by the LLM provider. Anthropic and OpenAI report cache hits differently. The tool normalizes these, but edge cases exist.
Session boundary detection: The tool infers session boundaries from idle time. If you pause a session for 10 minutes, it might split into two sessions. This inflates session count and skews overlap metrics.
Cost model drift: LLM pricing changes frequently. The tool uses hardcoded pricing tables. If a provider changes rates, the cost estimates become stale until the tool updates.
Technical Verdict
Use Frugal Tokens when:
- You run coding agents in production and need to understand cost drivers.
- You want to compare model pricing or cache strategies before switching.
- You need to debug why some sessions cost 10x more than others.
- You want session-level visibility into cache hit rates and tool usage.
Avoid it when:
- You run agents in a language runtime other than Deno (no instrumentation support yet).
- You need real-time alerting or cost budgets (this is a post-hoc analysis tool).
- You require multi-user or team-level cost tracking (single-user SQLite database).
- You need integration with existing observability platforms (no export to Datadog, Prometheus, etc.).
The tool fills a gap in agent cost observability. It shows you what drives spend at the session level. It does not prevent overspend or enforce budgets. It gives you the data to make informed decisions about model choice, cache strategy, and session design.