Skip to content
vavio

Why does my Hermes model bill grow during a long conversation?

A long Hermes Agent conversation costs more per message because every new turn re-sends the conversation so far to your model provider. Message forty carries the first thirty-nine with it, along with the system prompt and tool definitions. Prompt caching can bill that repeated part at a discounted rate, but only on providers and models that support it, and only while the cache stays warm. The Hermes sessions guide says a fresh session is almost always cheaper than a month-old thread, so starting one when a task ends is the simplest saving.

This answer follows Hermes v0.23.1, commit 3b42157, checked on 15 November 2026. We reviewed official source and documentation; we did not measure a bill or run a caching trial on a Vavio computer. We quote no model prices: check your provider’s pricing page, including its cached-input rate. Defaults are stated where they matter; a profile that changed them can behave differently.

What does Hermes send to the model on each turn?

On every turn, the model receives three things: the system prompt, the current conversation window and anything Hermes adds for that turn. The system prompt includes Hermes’s instructions, loaded context files, the skills index and the definitions of every enabled tool. The conversation window includes your messages, Hermes’s replies and the tool output it produced, such as a command result or a page it read.

Providers bill input tokens on each call. As the conversation window grows, so does the input on the next message. A single request can also take several model calls, because Hermes calls the model again after each tool runs, and each call re-sends the context.

Raw attachments are not re-sent. The sessions guide says an image or audio file is handled for the turn it arrives in; later turns carry only what was written into the conversation, such as a description or a file path. The guide names verbose text as the most common cause of growth: pasted transcripts, full logs, large tool output, long diffs and repeated status reports.

How do I see what is filling the context?

Hermes has three read-only commands for this. They work in the terminal and in messaging chats such as Telegram.

Command What it shows
/usage This session’s input and output tokens, API calls, current context size and compressions. Where your provider exposes it, remaining credits or quota.
/context How full the model’s window is, and an estimated split between system prompt, tool definitions, skills, memory and conversation. /context all adds per-skill and per-toolset detail.
/insights Usage over the last 30 days by default; /insights 7 picks another period.

/context is computed locally and makes no model call, so checking it does not add to the bill. Your provider’s own usage page remains the authority for what you were charged.

When does prompt caching lower the bill?

Prompt caching lets a provider recognize the start of a request it has just seen and bill that part at its cached-input rate. A conversation only grows at the end, so most of each turn can be a cache hit. Hermes’s developer guide puts the saving at roughly 75% of input cost on multi-turn conversations with Claude; its model guide describes cached input as roughly 75–90% cheaper than uncached input. Your provider’s pricing page sets the actual rate.

Whether Hermes asks for caching depends on the route:

Route What Hermes does
Claude through Anthropic, OpenRouter, Nous Portal, AWS Bedrock or another Anthropic-compatible gateway Adds cache markers automatically; on Anthropic-style routes, on the system prompt and the latest messages.
Kimi or Moonshot through OpenRouter or Nous Portal; Qwen through Nous Portal, OpenCode or Alibaba; MiniMax M2 models on MiniMax’s Anthropic-compatible endpoint Adds the same kind of cache markers.
xAI Grok Sends a per-conversation ID so xAI can reuse its cache.
Most other providers Hermes adds no cache markers. Some providers cache repeated prompts on their own; check whether yours lists a cached-input price.

Cache entries expire. Hermes’s default cache lifetime on marked routes is five minutes (prompt_caching.cache_ttl: "5m"). If you reply less often than that, the next message re-writes the cache instead of reading it. cache_ttl: "1h" keeps it longer, but the docs note that Anthropic-style one-hour writes cost twice the base input price, against 1.25 times for five minutes. They only pay off when your turns are more than five minutes apart. Most Qwen routes stay at five minutes whatever you set. cache_ttl: "auto" picks one hour for chats a person types into and five minutes for scheduled jobs, subagents and other machine-paced runs.

What makes the cache miss?

A cache hit needs the start of the request to match byte for byte. These actions change it, and the next message pays the full input price for everything after the change. A model switch or tool reload changes the start, so the whole conversation is re-read:

  • Switching model mid-conversation. The cache belongs to the model and account that served it. The same applies to an automatic fallback to another provider, or a credential-pool rotation onto a different account. Hermes asks for confirmation before a switch when the live session already holds more than 100,000 tokens (model.switch_context_confirm_tokens).
  • Reloading MCP tools. /reload-mcp rebuilds the tool definitions, which live in the system prompt.
  • Compression. When Hermes summarizes older turns to fit the window, the rewritten part misses the cache. The developer guide says the system prompt cache survives and caching recovers within one or two turns.
  • A long pause. Waiting longer than the cache lifetime between messages.

Switch models early in a conversation or right after /new rather than deep into a long one. Does changing Hermes’s model keep the conversation? covers the rest of a switch.

Does compression make it cheaper?

Compression keeps a long session working, but it does not make it cheap. Hermes compresses when the context nears the model’s window: by default at 75% for models with windows under 512K tokens, and at 50% for larger ones. The summary is written by a separate model call. By default that call uses your main model, as do other side tasks such as session titles and image description.

After compression, every turn still carries the summary, the recent turns and the system prompt. The Hermes sessions guide is direct about the result: “A fresh session with distilled memory is almost always cheaper than a month-old thread.” Why did Hermes compress my conversation? explains what the summary keeps.

How do I keep a long-running Hermes cheaper?

  1. Start a new session when a task ends. Run /new, optionally with a name such as /new invoices, at natural stopping points. Messaging chats never expire on their own, so a Telegram thread can run for weeks unless you reset it. Restarting the gateway or the computer resumes the same session. Before resetting, ask Hermes to remember anything worth keeping; what Hermes remembers after a new chat explains what carries over.
  2. Keep bulky output out of the chat. Ask Hermes to write long logs and results to a file and reply with a short excerpt. Paste a path or a focused extract instead of a full transcript.
  3. Compress on your terms. /compress here 4 keeps your last four exchanges word for word and summarizes the rest.
  4. Give side tasks a cheaper model. If your main model is expensive, set auxiliary.compression, auxiliary.vision or auxiliary.title_generation to a cheaper provider and model, or choose Configure auxiliary models in hermes model. A weaker compression model writes a weaker summary.
  5. Trim large tool output on big-window models. On a model with a very large window, compression rarely fires, so old tool output is re-sent for a long time. compression.proactive_prune_tokens: 48000 makes Hermes summarize old oversized tool results without a model call once re-sent history passes that size. Each prune breaks the cache once, so it is off by default and only commits when it saves a minimum amount.
  6. Match the cache lifetime to how you chat. If you reply every twenty minutes through the day on a cached route, try prompt_caching.cache_ttl: "auto" and compare /usage and your provider’s bill over a few days.

Scheduled jobs need less of this. Each cron run starts a fresh session, so a daily job does not inherit yesterday’s conversation.

What does this mean for my monthly budget?

Model usage is a separate line from hosting. How much does it cost to run Hermes Agent each month? shows how to add up the computer, the model and any assistant you already pay for. Long conversations affect the model line only. A computer that stays online makes long-running messaging threads easy to keep open, so the session habits above matter more there.

Vavio gives your Hermes its own managed computer for $14.99 per month, with model usage billed separately by your provider. Hosting does not change what Hermes sends to the model; your session habits and model choice do. Compare the current hosting offer with running Hermes yourself.

Sources

Written by Vavio with AI assistance and checked against pinned official Hermes v0.23.1 source and documentation. This is a source review, not a measured bill or a live caching trial.

More from the Vavio blog: Hermes Agent guides