Every large language model (LLM) request consumes tokens, which directly affect inference costs and response time. As AI agents, retrieval-augmented generation (RAG) pipelines, and enterprise assistants move into production, unnecessary tokens can increase operating expenses across repeated model calls.
LLM token optimization reduces token usage while preserving the context, accuracy, and information required to complete a task. This blog examines three techniques: ASD-STE100 for clearer instructions, CAVEMAN for shorter model responses, and TOON for compact structured data. Each technique addresses a different part of the token workload, with savings that depend on the model, task, and implementation.
Why Every Token Optimization
- ASD-STE100 trims the instructions you write.
- CAVEMAN trims the answers the model writes.
- TOON trims the structured data you feed in.
Where each technique acts · input instructions, input data, output answers
The three streams meet at the model. Fix the biggest stream first, then stack the others.
1. ASD-STE100: Write prompts like an aircraft manual
ASD-STE100 Simplified Technical English (STE) is a controlled version of English designed so that a reader is unlikely to misread an instruction. It began as AECMA Simplified English in the aerospace industry, took the ASD-STE100 name in 2004, and reached Issue 9 on January 15, 2025. Airlines asked for it because most of them were not from English-speaking countries, yet their mechanics worked from English manuals.
The standard has two parts: writing rules for grammar and style, and a dictionary of approved words with one meaning each. A copy is free on request from the maintainers.
Why it saves tokens
A model is a non-native reader too. Hedges such as “perhaps” and “if at all possible” cost tokens and carry no instruction. Vague verbs force the model to guess, and a wrong guess costs a whole retry. STE removes both kinds of waste.
ASD-STE100 before and after · word counts of this example
The rewrite keeps every instruction and drops 19 words. The bigger win is the missing retry: three numbered commands leave nothing to guess.
How to apply it
Use STE where text is read on every call: system prompts, CLAUDE.md files, agent rules and tool descriptions. A one-time rewrite pays back on each request. You can also ask the model to answer in STE, which shortens its output. One community output style for Claude Code applies Issue 9 rules to agent prose.
Write all prose in Simplified Technical English (ASD-STE100):
– Use one meaning for each word.
– Use the active voice.
– Write one instruction in each sentence.
– Keep procedure sentences to 20 words or fewer.
– Do not apply these rules to code, commands or error text.
– Do not apply these rules to code, commands or error text.
ASD publishes no token-saving figure, because STE is a writing standard, not an AI tool. Measure your own prompts before and after.
2. CAVEMAN: Make the model talk less
CAVEMAN is an open-source skill that makes a coding agent answer in terse, telegraphic English. Julius Brussee released it for Claude Code, and it now runs in Codex, Cursor, Gemini CLI and other agents. Its tagline sums it up: why use many token when few token do trick.
The skill is a set of instructions, not a model change. It tells the agent what to cut and what to protect.
| Cut | Kept exact |
|---|---|
| Pleasantries (“Great question!”) | Code blocks |
| Hedges (“you might want to consider”) | Error messages, quoted verbatim |
| Articles (a, an, the) | Technical terms and identifiers |
| Long intros and recaps | Git commits and pull request text |
Normal reply vs caveman reply · word counts of this example
The fix is the same in both replies. Only the wrapping is gone.
How to use it
In Claude Code, install it from the plugin marketplace, then type /caveman in a session. Say “normal mode” to switch it off.
claude plugin marketplace add JuliusBrussee/caveman
claude plugin install caveman@caveman
claude plugin install caveman@caveman
The project reports about 65–75% fewer output tokens in its examples; one reply fell from 69 tokens to 19. A companion skill, caveman-compress, rewrites memory files such as CLAUDE.md in the same style and keeps a readable backup. The project puts that input saving at about 45%.
The reality check
Caveman only shrinks what the model writes. Reasoning tokens are untouched. In a typical agent session, output is a small slice of the bill: an Implicator analysis put it at 0.6–2.5% of billed tokens, against cached prompts, tool definitions and file reads. So the memory-file compressor often saves more than the headline skill.
The other cost is the missing “why”. Terse answers suit quick fixes. They suit teaching and design reviews less well.
3. TOON: Send data without the punctuation tax
TOON (Token-Oriented Object Notation) is a compact, lossless way to write JSON data for an LLM. Johann Schopplich created it, and it is MIT-licensed. It borrows indentation from YAML for nested objects and rows from CSV for lists of similar records.
JSON spends tokens on structure: every key is quoted and repeated, every record is wrapped in braces. TOON declares the field names once in a header, then streams one row per record.
JSON vs TOON · the same two records
The header does double duty. [2] states how many rows follow, and {id,name,role} states how wide each row is. A model or a parser can spot a truncated or malformed list at once.
The numbers
On the project’s weather-forecast example, the JSON costs about 117 tokens and the TOON about 66, a 43.6% saving. Across its mixed benchmark of 244 questions on 4 models, TOON used 42.6% fewer tokens than JSON with slightly higher accuracy. On flat, fully tabular data it was 58.7% smaller than pretty-printed JSON.
How to use it
Treat TOON as a translation layer. Keep JSON in your code, and encode to TOON only at the prompt boundary.
npm install @toon-format/toon
cat data.json | npx @toon-format/cli –stats # shows the saving on your own data
cat data.json | npx @toon-format/cli –stats # shows the saving on your own data
When not to use it
- Deeply nested or irregular data: Compact JSON is often as small or smaller.
- Purely flat tables: Plain CSV is about 6% smaller, though it has no row-count guardrail.
- Latency-bound local models: Some process compact JSON faster despite more tokens. Measure first.
Which token optimization technique to use, and when
The three techniques do not compete. Each one works on a different stream, so you can stack them.
| Technique | Trims | Reported saving | Effort | Best for | Watch out for |
|---|---|---|---|---|---|
| ASD-STE100 | Your instructions (input) | No published figure; depends on your prose | Rewrite once, by hand or with a style rule | System prompts, CLAUDE.md, tool descriptions | Prompts so bare they lose needed context |
| CAVEMAN | Model replies (output); memory files with caveman-compress | ~65–75% of output tokens (project claim); ~45% on memory files | Install a skill, type /caveman | Long agent sessions, quick fixes | Answers that skip the reasoning |
| TOON | Structured data (input) | ~40–60% vs JSON on uniform records | One encode() call at the prompt boundary | Tables, logs, API and tool results, RAG records | Nested or irregular data |
Stack them in order of size
Start with the biggest stream. In agent and RAG workloads that is usually input: data, files and standing instructions. So encode records with TOON and tighten standing prompts with STE first. Add caveman mode last, for the output that remains.
Four checks before you trim
- Measure with your own tokenizer. Token counts differ by provider; TOON’s benchmarks used the GPT-5 o200k_base tokenizer.
- Test accuracy, not only token counts. A cheaper prompt that needs a second try costs more.
- Never compress code, identifiers or error strings. All three techniques leave them exact for good reason.
- Keep a readable original. Compress what the model reads, and keep the full version for the humans who maintain it.
The takeaway
Write instructions the way aircraft manuals are written, send data as tables instead of punctuation, and let the model skip the small talk. Each habit is small. Harbinger helps enterprises build AI agents and AI-enabled workflows that hold up in production. To discuss making an existing AI workflow more efficient, contact Harbinger’s experts.
Additional Perspectives on the FDE Model
Forward-deployed engineering is gaining attention as a way to bring engineers closer to customer problems and real-world implementation. Several organizations describe this model as a way to align engineering work with customer needs, systems, and workflows. Here are a few resources that offer additional perspectives on the FDE model:https://www.griddynamics.com/glossary/forward-deployed-engineering
https://miragemetrics.com/guides/forward-deployed-engineering
https://www.cognizant.com/us/en/glossary/forward-deployed-engineer
Frequently asked questions
What is a token in a large language model?
A token is the unit of text an LLM reads and writes. It can be a whole word, part of a word, or a punctuation mark. OpenAI estimates that one token is roughly four characters of English text, though each provider’s tokenizer splits text differently.
Why do output tokens often cost more than input tokens?
Many LLM providers price output tokens higher than input tokens because the model generates output one token at a time, which takes more compute than reading input. This makes output-trimming techniques more valuable per token, even when output is a smaller share of the total.
How does prompt caching relate to token optimization?
Prompt caching lets a provider reuse a repeated prompt prefix, such as a long system prompt, at a reduced cost on later calls. It complements the techniques in this blog: caching lowers the price of repeated input, while trimming techniques reduce the volume of tokens in the first place.
Do these techniques work with any LLM provider?
ASD-STE100 and TOON are plain-text approaches, so they work with any model that reads text. Caveman is an agent skill and depends on support in the agent tool a team uses. Token savings vary by provider, so results should be measured on each model in use.
Can removing too much context hurt AI results?
Yes. A prompt trimmed too far can lose the background a model needs to answer correctly, which leads to errors and retries. The goal is to remove padding, not necessary context, and to test accuracy after every change.
About The Author:
Amit Jain
AI Architect
