Skip to content
←Back to Open Source

OPEN SOURCE DEEP DIVE

LLMCoding AgentToken OptimizationCost Reduction

TokenEconomizer: Cross-Session Cost Control for Coding Agents

A black-box plugin that learns across coding-agent sessions to reduce repeated repository discovery, unused tools, and failure loops.

microsoft/TokenEconomizer7.2kPythonApache-2.09 min read

Positioning: a measurable cost controller for repeated work

TokenEconomizer is an open-source cost-control plugin for coding agents. It does not replace the model, change its weights, or require the agent to adopt a new work interface. Instead, it uses the session lifecycle already exposed by Claude Code, Codex, GitHub Copilot CLI, Pi, and other supported hosts: it reads their logs, injects a small amount of context, and changes tool exposure through each agent's own settings. The target is not a better single answer. It is the recurring cost of rediscovering the same repository facts, loading the same unused tools, and entering similar failure loops across many sessions.

The design is deliberately black-box. The underlying agent and model remain unchanged, while learned state stays locally under ~/.tokeneconomizer. When the plugin needs a model call, it runs through the user's existing agent login and plan rather than requiring a separate API key. That makes the optimization a repository-level investment: some cost is paid to learn, and the return appears in later sessions.

Where the cost comes from

A coding-agent request repeatedly carries system instructions, tool definitions, and the conversation so far. As a session grows, much of that context is read from a provider cache. Counting those cached reads at the same price as fresh input or output produces a misleading comparison. TokenEconomizer attacks three forms of repeated waste: rediscovering repository commands and environment facts, sending tool definitions that are never used, and continuing down a path after failure signals have already started to repeat.

MechanismLearning signalActionSafety boundary
Micro-skillsRepository facts and failures repeated across sessionsInject short facts at session startRequires repeated confirmation; contradicting evidence retires a fact; off by default
Tool pruningLoaded tools that remain unusedWrite a deny list through the agent's settingsProtected and previously used tools are retained
Loop guardRepeated failures, accumulating errors, and abnormal turn lengthAdd a redirect to context and notify the userIt never stops the turn itself

Micro-skills: carrying verified repository knowledge forward

A micro-skill is a short, actionable fact about the repository. It might record that tests require a build step first, or that python -m pytest works here where a bare pytest does not. Facts can come from two sources: failures repeated across sessions and an optional model proposer that reviews recent sessions. The proposer runs once every three sessions by default, but the whole micro-skill mechanism is off by default because those calls have a real cost and because injected facts are derived from command output that the repository can influence.

A fact becomes eligible for injection only after three sessions confirm it. If later evidence contradicts it, the fact is dropped. The mechanism is therefore not a one-shot prompt trick that hard-codes an accidental command. It is a reversible memory layer that promotes only repository knowledge with repeated support. Users who want learning without model calls can use the free environment-fact layer, which forms facts from repeated failures.

Tool pruning: evidence before removing a definition

Tool pruning addresses a different fixed cost: every request includes definitions for the tools currently loaded by the agent. A tool that has not been used in the current repository is not automatically safe to remove, since one task may simply not need it. The project therefore uses two alternative thresholds. A tool becomes a candidate after remaining unused across ten prompts spread over at least two sessions, or after five sessions of any length.

Protected work-surface tools are never removed, and a tool that has already been used successfully stays protected. Writing the resulting deny list also waits until the repository's sessions have been idle for about sixty minutes, with a one-day latest bound. The delay protects prompt caching: changing the tool prefix while a conversation is warm can force the resumed session to write its history to the cache again instead of reading it. The practical unit differs by host. Claude Code prunes individual tools, Codex adjusts capability packages, and Copilot CLI handles whole MCP servers. Pi, Cursor, Grok, opencode, Hermes, Antigravity, and Droid do not currently expose an equivalent persistent pruning surface.

The plugin adds only its own settings entries and does not overwrite deny rules written by the user. Files it creates are kept out of version control. If a settings file is unsafe to edit, the result stays a proposal rather than a partially applied change. Users can restore removed tools, and a tool restored by hand will not be removed again automatically.

The loop guard: detect a stuck turn without taking control

The loop guard observes one prompt-turn at a time. It first waits for at least five turns in the current repository to end normally, meaning no failing check remains after the last edit. Those turns establish an envelope for normal length and edit/error behavior. Once that baseline exists, repeated failures or accumulating errors can send the trajectory to a judge model. If the judge is at least 0.7 confident that the turn is stuck, the agent receives one short redirect, with one redirect per turn by default.

A second rule can act without the judge. If a turn runs to 2.5 times its usual length while churning through repeated edits and failures, it receives a redirect as a backstop. If the turn remains stuck, TokenEconomizer shows a one-line notice explaining how to stop the current host. It does not terminate the process itself. A turn that looks unproductive from the outside may still contain useful work that has not yet surfaced, so the final stopping decision remains with the user.

Accounting: cached reads and plugin overhead share one denominator

The ledger adds the agent's execution cost to every model call made by the mechanisms, then converts the result into fresh-input-token equivalents. Cache reads, cache writes, and output tokens are weighted by the price ratios of the exact billed model rather than treating every token as full-price input. The experiment path fails hard on an unknown billed model so a comparison cannot be silently mispriced. The live plugin records a labeled fallback curve so a user's session can still be accounted for.

This accounting choice changes what counts as a saving. Tool pruning reduces cached context, but reporting raw token totals would overstate the benefit. Micro-skills and loop judging add output and fresh-input cost, and those calls must be charged to the same account. A result is only a real improvement when the reduced execution cost outweighs the mechanism's own overhead.

Architecture: mechanisms stay independent of agents and task domains

The core package uses only Python's standard library and isolates variation behind a few narrow interfaces. Trace readers normalize each agent's logs into a common session representation. A domain adapter describes which tools to protect or exclude, how to recognize loop signals, what task text the judge receives, and which vocabulary micro-skills should use. A stage interface attaches mechanisms to task start, tool construction, run arguments, and task completion. The ledger and pricing modules are shared by the live plugin and the experiment harness so their cost definitions cannot drift apart.

Adding a new agent mainly requires work in the trace and host adapter layers rather than duplicating the three mechanisms. Adding a task domain requires a domain adapter. This separation lets the same core operate in interactive coding sessions and benchmark streams while keeping Claude, Codex, and Copilot event formats outside the mechanism logic.

Experimental evidence: lower cost in all thirty repository cells

The paper evaluated five agent-model combinations on the same 277 SWE-Bench tasks from six repositories. Every task ran as a separate session, tasks from a repository ran in order, and each repository started with nothing learned. Costs included every model call made by TokenEconomizer. Across the resulting thirty repository-by-combination cells, mean cost was lower than the default in all thirty. The combination-level reduction ranged from 18.4% to 30.9%.

Agent and modelMean cost changePass@1 change
Claude Code / Sonnet 523.6% lower1.3 points higher
Claude Code / Opus 4.830.9% lower1.0 point lower
Codex / GPT-5.6 Terra24.7% lower0.9 points higher
Codex / GPT-5.6 Sol26.1% lower1.6 points higher
Copilot CLI / Sonnet 518.4% lower2.6 points higher

Four of the five combinations improved Pass@1. The Opus 4.8 run illustrates the tradeoff: the largest cost reduction came with a one-point decline in correctness. Cost fell by 16% to 21% on Web-Bench. Two security-auditing benchmarks, where every task starts in a fresh repository, still showed 6% to 20% lower cost, indicating that the gains are not limited to reusing facts inside one repository. Within individual repositories and combinations, the paper reports a 5% to 40% cost range. These are controlled benchmark results, not a fixed savings rate for every production repository.

Installation, support, and operating model

TokenEconomizer requires Python 3.10 or newer and an agent that is already signed in. Primary support covers Claude Code, Codex, GitHub Copilot CLI, and Pi. opencode, Cursor Agent, Grok Build, Hermes Agent, Antigravity CLI, and Factory Droid are experimental, with different subsets of the mechanisms available. After installation, users can inspect status, list learned facts, review pruning evidence, restore tools, and change configuration. These commands cost one model turn inside an agent and are free from a terminal.

The default operating model emphasizes reversibility. Tool pruning and the loop guard are enabled; micro-skills are disabled. Hooks fail open, so an error in the cost controller does not block the coding session. Expensive background work such as session folding and loop judging is detached from the hook's critical path wherever the host allows it.

Where it fits, and where it does not

TokenEconomizer is most useful to teams that repeatedly use the same repositories, follow stable workflows, and can observe agent cost over time. The benefit accumulates across sessions; one-off tasks and constantly changing workspaces provide little opportunity for repository-level learning. On an untrusted repository, micro-skills should remain off or be limited to the model-free environment layer, because learned facts are derived from repository-controlled command output. The project also cannot replace a host permission surface that does not exist: if an agent offers no persistent tool controls, TokenEconomizer cannot add them.

The broader value is not just the three optimization techniques. It is the combination of evidence thresholds, reversible settings, cache-aware accounting, and host-specific adapters around them. TokenEconomizer turns the question of whether a coding agent is learning from previous work into something that can be observed, measured, and rolled back instead of being inferred from a single impressive session.

Related Projects

alibaba/open-code-reviewApache-2.0

Alibaba open-sources Open Code Review: battle-tested hybrid code review CLI

Originating from Alibaba's internal official AI review assistant (tens of thousands of devs, millions of defects over two years): hybrid deterministic pipelines + LLM Agent, line-level comments, built-in NPE/thread-safety/XSS/SQL-injection rulesets, plugins for Claude Code/Codex/Cursor and more, CI/CD integration, and a public benchmark of 200 real PRs with 1,505 annotated issues. OpenAI/Anthropic compatible.

Alibabaopen-code-review
Go43k3.1k133
p-e-w/hereticAGPL-3.0

Heretic: Rewriting Model Behavior Without Retraining

A training-free behavior editor combining ARA/abliteration, Optuna, pluggable scorers, and reproducible manifests; it changes refusal boundaries rather than proving broad capability preservation.

LLMAbliteration
Python39k4.4k200
NandhaKishorM/layaApache-2.0

Laya: A Single-Pass Typed Decision Engine

An open-source, multilingual non-autoregressive decision engine that answers choice, ordinal, and yes/no questions in one forward pass, with checkpoint routing and production serving APIs.

LLMDecision Engine
Python34k3.1k122
Niko1221/StrataMIT

Strata: Run a 125B-Parameter MoE Model on Your Gaming PC

Strata is an open-source local inference engine built on llama.cpp/ggml that runs the 125-billion-parameter Qwen3.8-Flash-Next MoE model on consumer gaming PCs with as little as 12GB VRAM, using tiered expert caching (GPU→RAM→SSD) and guess-and-check speculative decoding to achieve 53-94 tokens/second.

LLMInferenceOptimization
C++7.8k68351

As an Amazon Associate, we earn from qualifying purchases.