<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[HiveGate Blog]]></title><description><![CDATA[Practical guides to running Agno agents in production: deployment, per-user OAuth, human approval of tool calls and LLM cost tracking. By HiveGate.]]></description><link>https://hivegate.hashnode.dev</link><image><url>https://cdn.hashnode.com/uploads/logos/6ad12040f068019d985e74d3/3325ca13-7370-4da8-8ebd-236d880ddbc2.png</url><title>HiveGate Blog</title><link>https://hivegate.hashnode.dev</link></image><generator>RSS for Node</generator><lastBuildDate>Sun, 08 Nov 2026 09:34:01 GMT</lastBuildDate><atom:link href="https://hivegate.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[AI agent REST API from pip install: HiveGate 0.3.1]]></title><description><![CDATA[Until last week, trying HiveGate meant cloning the repo, copying .env.example, running a setup script and starting the server from the checkout. That's…]]></description><link>https://hivegate.hashnode.dev/ai-agent-rest-api-from-pip-install-hivegate-0-3-1</link><guid isPermaLink="true">https://hivegate.hashnode.dev/ai-agent-rest-api-from-pip-install-hivegate-0-3-1</guid><category><![CDATA[Python]]></category><category><![CDATA[AI]]></category><category><![CDATA[Open Source]]></category><category><![CDATA[Programming Blogs]]></category><dc:creator><![CDATA[HiveGate]]></dc:creator><pubDate>Sat, 07 Nov 2026 19:50:52 GMT</pubDate><content:encoded><![CDATA[<p>Until last week, trying <a href="https://github.com/hivegate-ai/hivegate">HiveGate</a> meant cloning the repo, copying <code>.env.example</code>, running a setup script and starting the server from the checkout. That's fine if you want to work on HiveGate. It's a lot to ask if you only want to see whether it fits your agents.</p>
<p>HiveGate 0.3.1, released on October 31, is the first release on PyPI. You install it as a tool, write a starter config with one command, start Postgres and Qdrant, and serve. This post walks through that path from an empty directory to an AI agent REST API with per-user sessions, a paused tool call you approve over HTTP, and a cost record for every model call. It ends with what changes when you take it off your laptop.</p>
<p>HiveGate is an MIT-licensed FastAPI service built on <a href="https://github.com/agno-agi/agno">Agno</a>. You write the agent. HiveGate serves it. It is not a request-routing API gateway like Kong.</p>
<h2>What you need</h2>
<ul>
<li>Python 3.11 or newer, and <a href="https://docs.astral.sh/uv/">uv</a> or <a href="https://pipx.pypa.io/">pipx</a></li>
<li>Docker, for Postgres and Qdrant</li>
<li>An API key for a model provider. The seeded demo agents use Gemini, so <code>GOOGLE_API_KEY</code> is the shortest path. OpenAI, Anthropic, xAI, Z.ai and DeepSeek keys also work.</li>
</ul>
<h2>Install</h2>
<pre><code class="language-sh">uv tool install hivegate-ai   # or: pipx install hivegate-ai
</code></pre>
<p>The package is <code>hivegate-ai</code> because PyPI refused <code>hivegate</code> as too similar to an existing project. The command is still <code>hivegate</code>.</p>
<p>Install it as a tool rather than into a project's virtual environment. The package's top-level modules have generic names (<code>api</code>, <code>agents</code>, <code>db</code>), and they can clash with other libraries in a shared environment. <code>uv tool</code> and <code>pipx</code> give it an environment of its own.</p>
<h2>Write the config</h2>
<pre><code class="language-sh">mkdir my-agents &amp;&amp; cd my-agents
hivegate init
</code></pre>
<p><code>init</code> writes four files: <code>compose.yaml</code>, <code>.env</code>, <code>db/migrations/setup.sql</code> and <code>sql/sample_data.sql</code>. The <code>.env</code> is not a copy of the example file. Three values are filled in for a first local run:</p>
<ul>
<li><code>AUTH_DISABLED=true</code>, so you can call <code>/v2</code> without an API key while the server only listens on your machine.</li>
<li><code>ADMIN_SECRET</code>, set to a random 32-byte value instead of the placeholder.</li>
<li><code>SECRET_TOKEN_ENC_KEY</code>, a fresh Fernet key. HiveGate encrypts the per-user OAuth tokens it stores with this key, so keep it with your data.</li>
</ul>
<p>Running <code>init</code> again refuses to overwrite anything. With <code>--force</code> it rewrites the compose files but still keeps an existing <code>.env</code>, because that file holds your API keys and the key your stored tokens are encrypted with.</p>
<p>Open <code>.env</code> and set your model key:</p>
<pre><code class="language-sh">GOOGLE_API_KEY=...
</code></pre>
<h2>Start the stack</h2>
<pre><code class="language-sh">docker compose up -d   # Postgres + Qdrant, seeds demo agents
hivegate serve         # http://127.0.0.1:8000/docs
</code></pre>
<p>Compose starts Postgres and Qdrant, creates the schema and seeds a handful of demo agents and teams. Since 0.3.1 it publishes their ports on <code>127.0.0.1</code> only, so the databases aren't reachable from your network by accident.</p>
<p>One known wart: <code>docker compose ps</code> reports Qdrant as unhealthy even though it works. The healthcheck calls <code>curl</code>, which the Qdrant image doesn't have, so <code>docker compose up -d --wait</code> fails too. That's <a href="https://github.com/hivegate-ai/hivegate/issues/135">#135</a>, a good first issue if you want to fix it.</p>
<p><code>hivegate serve</code> loads <code>.env</code> from the current directory and starts the API on <code>127.0.0.1:8000</code>. If no model key is set, it starts anyway and prints a warning, because agents can't answer without one. The interactive OpenAPI docs are at <code>/docs</code>.</p>
<h2>First calls</h2>
<p>Check that it's up, then list the agents:</p>
<pre><code class="language-bash">curl http://localhost:8000/health
curl http://localhost:8000/v2/agents
</code></pre>
<p>You'll see the seeded agents, including <code>demo-assistant</code>. Chat with it:</p>
<pre><code class="language-bash">curl -X POST http://localhost:8000/v2/agents/demo-assistant/chat \
  -H "Content-Type: application/json" \
  -d '{
    "message": "What is the capital of France?",
    "stream": false,
    "user_id": "user-1",
    "session_id": "session-1",
    "timezone": "UTC",
    "locale": "en-US"
  }'
</code></pre>
<p>The response carries the answer and the identifiers you'll use later:</p>
<pre><code class="language-json">{
  "content": "The capital of France is Paris.",
  "agent_id": "demo-assistant",
  "session_id": "session-1",
  "model": "gemini-3-flash-preview",
  "status": "completed",
  "run_id": "run-f47ac10b",
  "tools": []
}
</code></pre>
<p>Leave out <code>"stream": false</code> and you get server-sent events instead, which is what a chat UI wants.</p>
<p>Every chat names a <code>user_id</code> and a <code>session_id</code>. Send another message with the same pair and the agent sees the earlier turns, because history lives in Postgres, not in the process. Use a different <code>user_id</code> and that user gets their own sessions and memory. Nothing in your agent code has to track who is who.</p>
<h2>Agents are data</h2>
<p>The demo agents came from SQL, but you create your own over the same API. An agent is an id, a name and a prompt template, plus optional settings for memory, history and reasoning:</p>
<pre><code class="language-bash">curl -X POST http://localhost:8000/v2/agents \
  -H "Content-Type: application/json" \
  -d '{
    "id": "my-first-agent",
    "name": "My First Agent",
    "template": "You are a helpful assistant that answers questions concisely.",
    "description": "A simple Q&amp;A agent",
    "tags": ["qa", "general"]
  }'
</code></pre>
<p>It's available at <code>/v2/agents/my-first-agent/chat</code> straight away, with no redeploy. Changing a prompt is an API call, not a release. The <a href="https://hivegate.dev/docs/api-agents/">agents API docs</a> list every field.</p>
<h2>A tool call that waits for a person</h2>
<p>Letting an agent search an inbox is low risk. Letting it send an email is not. HiveGate's built-in Google and Microsoft toolkits mark every write as <code>requires_confirmation</code>: <code>send_email</code>, <code>trash_email</code>, <code>schedule_meeting</code> and the rest. Your own Agno toolkits opt in with <code>requires_confirmation_tools=[...]</code>.</p>
<p>When the model calls one of those tools, the run stops before the tool executes, and the chat response says so:</p>
<pre><code class="language-json">{
  "content": null,
  "status": "PAUSED",
  "run_id": "run-f47ac10b",
  "tools": [
    {
      "tool_call_id": "tc-001",
      "tool_name": "send_email",
      "requires_confirmation": true,
      "tool_args": {"to": "jane@acme.com", "subject": "Password Reset"}
    }
  ]
}
</code></pre>
<p>Nothing has been sent yet. Show the pending call wherever your users are, then send the decision back. The reviewer can edit the arguments, not only say yes or no:</p>
<pre><code class="language-bash">curl -X POST http://localhost:8000/v2/agents/my-first-agent/chat/commit \
  -H "Content-Type: application/json" \
  -d '{
    "run_id": "run-f47ac10b",
    "stream": false,
    "user_id": "user-1",
    "session_id": "session-1",
    "updated_tools": [
      {
        "tool_call_id": "tc-001",
        "confirmed": true,
        "tool_args": {"to": "jane@acme.com", "subject": "Password Reset Instructions"}
      }
    ]
  }'
</code></pre>
<p>Set <code>"confirmed": false</code> and the run ends with <code>status: "cancelled"</code> without calling the tool. Since 0.3.1 the same works for team runs, which used to pause and then couldn't be committed (#116).</p>
<p>Two limits to know. Paused runs live in the server's memory, so a restart or a request that lands on another replica gets <code>410 Gone</code>. Run a single replica, or route each session to the same instance, when you use approvals. And tools from MCP servers don't have an approval gate yet; that's <a href="https://github.com/hivegate-ai/hivegate/issues/72">roadmap item #72</a>. The <a href="https://hivegate.dev/blog/human-approval-agent-tool-calls/">human approval post</a> covers the flow in depth.</p>
<h2>What did that cost?</h2>
<p>Every model call is metered: input, output, cache-read and cache-write tokens, and the USD cost from a versioned price table. The read-only <code>/usage</code> API groups it by agent, user, tenant, model, API key or status:</p>
<pre><code class="language-bash">curl "http://localhost:8000/usage/summary?window=24h&amp;group_by=user"
curl "http://localhost:8000/usage/calls?order=cost&amp;limit=10"
</code></pre>
<p><code>/usage/calls</code> returns the most expensive calls first, with session and run ids, so you can find the conversation behind a spike. With a Claude model, prompt caching is on by default in 0.3.1, and the cache-read tokens show up in the same records. The <a href="https://hivegate.dev/blog/llm-token-usage-cost-tracking/">token cost post</a> shows the queries behind a per-user cost report.</p>
<h2>Off your laptop</h2>
<p><code>AUTH_DISABLED=true</code> is only acceptable while nothing else can reach the server, and <code>hivegate serve</code> enforces that. Ask it to listen on another host with auth off and it refuses:</p>
<pre><code class="language-sh">$ hivegate serve --host 0.0.0.0
Refusing to serve on 0.0.0.0 with AUTH_DISABLED on: every route would be open. Remove it from .env.
</code></pre>
<p>It refuses the placeholder <code>ADMIN_SECRET</code> the same way. Remove <code>AUTH_DISABLED</code> from <code>.env</code>, restart, and create an API key with the admin secret <code>init</code> generated:</p>
<pre><code class="language-bash">curl -X POST http://localhost:8000/admin/api-keys \
  -H "X-Admin-Secret: $ADMIN_SECRET" \
  -H "Content-Type: application/json" \
  -d '{"name": "Frontend", "owner_id": "team-frontend", "scopes": ["agents:read", "agents:chat"]}'
</code></pre>
<p>The raw key (<code>agk_...</code>) is returned once and stored as a SHA-256 hash. Send it as <code>X-API-Key</code> on every <code>/v2</code> call. For the cost dashboard, a key with only the <code>usage:read</code> scope can read <code>/usage</code> and nothing else. The <a href="https://hivegate.dev/docs/authentication/">authentication docs</a> have the details.</p>
<p>For a hosted deployment you don't need the package at all. The prebuilt image deploys in one click to Railway, Render or Koyeb from the <a href="https://hivegate.dev/deploy/">deploy page</a>, and the <a href="https://hivegate.dev/docs/deployment/">deployment docs</a> cover a production setup of your own.</p>
<h2>Upgrading</h2>
<pre><code class="language-sh">uv tool upgrade hivegate-ai
hivegate migrate
</code></pre>
<p><code>migrate</code> reads <code>.env</code> from the current directory and applies any database changes the new version ships. Coming from a clone of 0.3.0, apply <code>db/migrations/011_token_usage_cost.sql</code> (it adds the metering columns), and note that <code>compose.yaml</code> now reads <code>DB_PASS</code> and <code>DB_DATABASE</code> instead of <code>DB_PASSWORD</code> and <code>DB_NAME</code>.</p>
<h2>What else is in 0.3.1</h2>
<p>Everything merged since 0.3.0 on September 29:</p>
<ul>
<li>Per-call token cost metering and the <code>/usage</code> API (#109)</li>
<li>Prompt caching for Claude models (#110)</li>
<li>Approvable paused team runs (#116)</li>
<li><code>error_code: "provider_billing"</code> and a plain message when a provider account is out of credits, instead of the provider's raw error (#125)</li>
<li>A warning that names the server when an MCP server with an unsupported transport is dropped, instead of the agent silently getting fewer tools (#146, from first-time contributor @coughlmao)</li>
<li>Dependency upgrades that clear the open security advisories (#134)</li>
</ul>
<p>The full list is in the <a href="https://github.com/hivegate-ai/hivegate/blob/main/CHANGELOG.md">changelog</a>.</p>
<h2>When you don't need HiveGate</h2>
<p>Agno ships its own <a href="https://docs.agno.com/agent-os/introduction">AgentOS</a>, which serves agents defined in code over HTTP with sessions, human approval, RBAC and tracing. If your agents live in your codebase and AgentOS covers what you need, use AgentOS. It's maintained by the Agno team and tracks Agno releases.</p>
<p>HiveGate is for the case where agents and prompts are data you change over an API, each of your users connects their own Google or Microsoft account, approvals happen in your product over REST, and you need to know what each user cost you.</p>
<h2>Try it</h2>
<pre><code class="language-sh">uv tool install hivegate-ai
mkdir my-agents &amp;&amp; cd my-agents &amp;&amp; hivegate init
# set GOOGLE_API_KEY in .env
docker compose up -d &amp;&amp; hivegate serve
</code></pre>
<p>Then open <code>http://127.0.0.1:8000/docs</code>. The <a href="https://hivegate.dev/docs/quickstart/">quickstart</a> and the rest of the <a href="https://hivegate.dev/docs/">docs</a> go from there, and issues and questions are welcome on <a href="https://github.com/hivegate-ai/hivegate">GitHub</a>.</p>
<p>If this is useful, a star helps other people find the project: <a href="https://github.com/hivegate-ai/hivegate">⭐ github.com/hivegate-ai/hivegate</a>.</p>
]]></content:encoded></item><item><title><![CDATA[LLM token usage and cost tracking per user and tenant]]></title><description><![CDATA[Originally published on hivegate.dev. An agent turn is rarely one model call. The model reads the tools and the system prompt, calls a tool, reads the…]]></description><link>https://hivegate.hashnode.dev/llm-token-usage-and-cost-tracking-per-user-and-tenant</link><guid isPermaLink="true">https://hivegate.hashnode.dev/llm-token-usage-and-cost-tracking-per-user-and-tenant</guid><category><![CDATA[Python]]></category><category><![CDATA[AI]]></category><category><![CDATA[Open Source]]></category><category><![CDATA[Machine Learning]]></category><dc:creator><![CDATA[HiveGate]]></dc:creator><pubDate>Sat, 31 Oct 2026 17:21:41 GMT</pubDate><content:encoded><![CDATA[<p><em>Originally published on <a href="https://hivegate.dev/blog/llm-token-usage-cost-tracking/">hivegate.dev</a>.</em></p>
<p>An agent turn is rarely one model call. The model reads the tools and the system prompt, calls a tool, reads the result, calls another tool, and only then answers. Every one of those round trips re-sends everything before it. A support agent that looks harmless in testing can cost far more for the user who keeps it busy all day, and a run that fails after three round trips is still billed.</p>
<p>Your provider's dashboard shows one number per API key. It doesn't know which of your users, tenants or agents spent it. If you want to bill, cap or simply understand that spend, you need the cost of each call recorded next to who made it.</p>
<p>This post shows how that works in <a href="https://github.com/hivegate-ai/hivegate">HiveGate</a>, an open-source FastAPI service for <a href="https://github.com/agno-agi/agno">Agno</a> agents: what gets recorded, how to query it, how prices are applied, and how Claude prompt caching changes the numbers. Both features landed on <code>main</code> this month (<a href="https://github.com/hivegate-ai/hivegate/pull/109">#109</a> and <a href="https://github.com/hivegate-ai/hivegate/pull/110">#110</a>). They are not in a tagged release yet; the latest is v0.3.0.</p>
<h2>First: you may already have this</h2>
<p>If every model call already goes through an LLM proxy or an observability tool that prices calls, such as LiteLLM, Langfuse or Helicone, you have per-call cost data. Tag those calls with your user and tenant ids and you may not need anything else.</p>
<p>HiveGate's meter is for the case where the agents run inside HiveGate and you want the cost in the same Postgres database as the sessions, keyed by the same <code>user_id</code>, <code>session_id</code> and <code>tenant_id</code>, without running another service.</p>
<h2>What gets recorded</h2>
<p>Every chat, commit, MCP and team run writes one row to the <code>token_usage</code> table. That includes streamed and non-streamed runs, and also runs that failed, were refused, or were abandoned when the client hung up. A row has:</p>
<ul>
<li><strong>Who:</strong> <code>agent_id</code>, <code>user_id</code>, <code>session_id</code>, <code>run_id</code>, <code>tenant_id</code>, and the <code>api_key_name</code> that made the request.</li>
<li><strong>Tokens:</strong> <code>prompt_tokens</code>, <code>completion_tokens</code>, <code>cache_read_tokens</code>, <code>cache_write_tokens</code> and <code>reasoning_tokens</code>.</li>
<li><strong>Work:</strong> <code>model_requests</code> (round trips), <code>tool_calls</code>, <code>duration_ms</code> and <code>ttft_ms</code> (time to first token).</li>
<li><strong>Money:</strong> <code>cost_usd</code>, the <code>prices_version</code> it was priced with, and <code>unpriced_models</code> for anything the price table doesn't know.</li>
<li><strong>Outcome:</strong> <code>status</code>, one of <code>completed</code>, <code>paused</code>, <code>cancelled</code>, <code>error</code> or <code>refused</code>, plus the error text, which stays server-side.</li>
<li><strong>Breakdown:</strong> a JSONB <code>breakdown</code> that splits the cost per model type, so the main model, the memory model and the compression model are each priced at their own rate.</li>
</ul>
<p><code>tenant_id</code> comes from the request's <code>tenant_profile.tenant_id</code>, or from <code>user_profile.tenant_id</code> if there's no tenant profile. Send it on every chat request and you can group by tenant later.</p>
<p>Failed runs are recorded with their real tokens on purpose. A run that errors after the model was called is still billed by the provider, and a client retrying such a run in a loop can spend a lot without anyone noticing. A run that pauses for <a href="https://hivegate.dev/blog/human-approval-agent-tool-calls/">human approval</a> is recorded as <code>paused</code>, and the resumed part gets its own row when it's committed.</p>
<p>Recording runs off the event loop in a thread pool with its own database session, and it never fails the caller's request. If the write fails, the error is logged and the user still gets their answer.</p>
<h2>The cost of a single call</h2>
<p>A non-streaming chat response carries the usage of that call in <code>token_usage</code> (trimmed):</p>
<pre><code class="language-json">{
  "status": "completed",
  "content": "Your order shipped on Tuesday.",
  "token_usage": {
    "input_tokens": 412,
    "output_tokens": 86,
    "cache_read_tokens": 8240,
    "cache_write_tokens": 388,
    "reasoning_tokens": 0,
    "model_requests": 2,
    "model": "claude-sonnet-5-5",
    "cost_usd": 0.004302,
    "unpriced_models": null,
    "status": "completed"
  }
}
</code></pre>
<p>The numbers above are an example, not a measurement, but the fields are the real ones. That's enough to show a running total in your own UI, or to stop a session when it crosses a budget you set.</p>
<h2>A key that can read usage and nothing else</h2>
<p>The <code>/usage</code> endpoints accept an API key with the <code>usage:read</code> scope (or <code>admin</code>), or the <code>X-Admin-Secret</code> header. Create a key for your dashboard with the admin API (see <a href="https://hivegate.dev/docs/api-admin/">API keys in the admin docs</a>):</p>
<pre><code class="language-sh">curl -X POST http://localhost:8000/admin/api-keys \
  -H "X-Admin-Secret: $ADMIN_SECRET" \
  -H "Content-Type: application/json" \
  -d '{"name": "usage-dashboard", "owner_id": "finance", "scopes": ["usage:read"]}'
</code></pre>
<p>The raw key is in the <code>api_key</code> field of the response, and it's shown only once. A key whose only scope is <code>usage:read</code> is refused on every <code>/v2</code> route with a 403, so a key you hand to a dashboard can't call agents and spend tokens.</p>
<h2>Which users cost the most</h2>
<p><code>GET /usage/summary</code> returns the totals for a window and a breakdown by one dimension. <code>window</code> is one of <code>1h</code>, <code>6h</code>, <code>24h</code>, <code>7d</code> or <code>30d</code>, and <code>group_by</code> is one of <code>agent</code>, <code>tenant</code>, <code>model</code>, <code>api_key</code>, <code>user</code> or <code>status</code>:</p>
<pre><code class="language-sh">curl -H "X-API-Key: $USAGE_KEY" \
  "http://localhost:8000/usage/summary?window=7d&amp;group_by=user"
</code></pre>
<p>Each group, and the totals, has <code>calls</code>, <code>cost_usd</code>, <code>input_tokens</code>, <code>output_tokens</code>, <code>cache_read_tokens</code>, <code>cache_write_tokens</code>, <code>reasoning_tokens</code>, <code>model_requests</code>, <code>failed_calls</code> and <code>failed_cost_usd</code>. Groups are sorted by cost, most expensive first. The response also has <code>model_types</code>, the cost split between the main model and helper models such as the memory model, which is where unexpected spend often hides.</p>
<p>Swap <code>group_by=user</code> for <code>tenant</code> to see the cost per customer, or for <code>status</code> to see how much you paid for runs that failed.</p>
<h2>The most expensive calls</h2>
<p><code>GET /usage/calls</code> lists individual calls, the most expensive first by default:</p>
<pre><code class="language-sh">curl -H "X-API-Key: $USAGE_KEY" \
  "http://localhost:8000/usage/calls?window=24h&amp;tenant=acme&amp;min_cost=0.05&amp;limit=20"
</code></pre>
<p>You can filter by <code>agent</code>, <code>tenant</code>, <code>status</code> and <code>min_cost</code>, and set <code>order=recent</code> to get the newest first. Every call carries its <code>session_id</code> and <code>run_id</code>, so you can open the conversation that produced it and see why it took twelve round trips.</p>
<p>Two more endpoints cover dashboards:</p>
<ul>
<li><code>GET /usage/timeseries?window=7d&amp;bucket=1d&amp;group_by=agent</code> returns the cost per time bucket for the top groups by cost (8 by default, <code>top</code> goes up to 20). The rest are folded into <code>(other)</code> so a chart stays readable.</li>
<li><code>GET /usage/live?after_id=0</code> returns the latest calls, oldest first. Poll it again with the <code>last_id</code> from the response to get only what's new.</li>
</ul>
<h2>How prices are applied</h2>
<p>Costs come from the price table in <code>agents/pricing.py</code>, in USD per million tokens, for Anthropic, Gemini and OpenAI models. <code>GET /usage/prices</code> returns the table that's in use.</p>
<p>Providers count tokens differently, and the meter follows each convention. Anthropic reports cache reads and writes separately from <code>input_tokens</code>. Gemini and OpenAI include cached tokens inside <code>input_tokens</code>, so the meter subtracts them before applying the full input price. Gemini reports thinking tokens separately and bills them at the output rate.</p>
<p>A model that isn't in the table is never guessed. Its calls are stored with <code>cost_usd</code> set to null and the model listed in <code>unpriced_models</code>. The summary counts those in <code>unpriced_calls</code> and <code>partially_priced_calls</code>. When either is above zero, read <code>cost_usd</code> as a lower bound, not a total.</p>
<p>To add or correct a price without a redeploy, set <code>PRICING_OVERRIDES</code>:</p>
<pre><code class="language-sh">PRICING_OVERRIDES='{"my-model": {"input": 5, "output": 25, "cache_read": 0.5, "cache_write": 6.25}}'
</code></pre>
<h2>Prompt caching for Claude models</h2>
<p>Each round trip of an agent re-sends the tools, the system prompt and the whole conversation so far. Without caching, all of that bills at the full input price every time. The PR that turned caching on (<a href="https://github.com/hivegate-ai/hivegate/pull/110">#110</a>) notes that on our own deployment, 0% of input tokens had been cached across all agents in the previous 30 days, and one agent was sending about 50K input tokens per run.</p>
<p>Claude models in HiveGate now run with two cache breakpoints, set in <code>agents/model_factory.py</code>:</p>
<pre><code class="language-python">ANTHROPIC_PROMPT_CACHING: dict[str, Any] = {
    "cache_system_prompt": True,
    "request_params": {"cache_control": {"type": "ephemeral"}},
}
</code></pre>
<ul>
<li><code>cache_system_prompt</code> marks the system prompt. Tools render before it, so the tools and the system prompt are cached together. HiveGate passes a fixed system message per agent, so that prefix is shared by every run of the agent, across users.</li>
<li>The top-level <code>cache_control</code> is Anthropic's automatic caching. It marks the last block and moves forward as the conversation grows, so each tool-loop round trip and each follow-up turn reads everything before it from cache.</li>
</ul>
<p>There's nothing to configure. Both breakpoints use the 5-minute TTL, and each read refreshes it. A prompt shorter than the model's minimum cacheable length (1024 tokens on Sonnet 4.6, 4096 on Haiku 4.5) is simply sent uncached, with no error.</p>
<h2>What caching does to the bill</h2>
<p>Cache reads bill at a fraction of the input price, and cache writes at 1.25x. For <code>claude-sonnet-5-5</code>, the price table has $2 per million input tokens, $0.20 for cache reads and $2.50 for cache writes.</p>
<p>Take an agent whose tools and system prompt add up to 10,000 tokens, and a turn with four round trips. Counting only that prefix:</p>
<ul>
<li><strong>Uncached:</strong> 4 × 10,000 tokens × $2/M = $0.080.</li>
<li><strong>Cached:</strong> one write, 10,000 × $2.50/M = $0.025, then three reads, 3 × 10,000 × $0.20/M = $0.006. Total $0.031.</li>
</ul>
<p>That's arithmetic from the price table, not a benchmark. The conversation part grows with every round trip and is cached the same way, so long tool loops gain the most. The cache lives for five minutes after its last read: an agent nobody has used for longer pays the write again, and so does a conversation that sat idle.</p>
<p>You don't have to trust the arithmetic. The meter records <code>cache_read_tokens</code> and <code>cache_write_tokens</code> for every call, so <code>/usage/summary?group_by=agent</code> shows, per agent, how much of the input came from cache and what it cost.</p>
<h2>Upgrading an existing database</h2>
<p>On a fresh install, HiveGate creates the <code>token_usage</code> table with all the columns on startup. An existing database needs the new columns from <code>db/migrations/011_token_usage_cost.sql</code>. Every statement is <code>ADD COLUMN IF NOT EXISTS</code>, and existing rows keep working:</p>
<pre><code class="language-sh">psql "$DATABASE_URL" -f db/migrations/011_token_usage_cost.sql
</code></pre>
<p>For history from before the meter existed, <code>scripts/usage_from_runs.py</code> builds a read-only per-agent cost report from Agno's stored runs.</p>
<h2>Try it</h2>
<ul>
<li>Deploy HiveGate from the <a href="https://hivegate.dev/deploy/">deploy page</a>, or run it locally from <code>main</code>. Create a <code>usage:read</code> key, send a few chat requests with a <code>user_id</code> and a <code>tenant_profile</code>, and call <code>/usage/summary?group_by=user</code>.</li>
<li>The meter is in <code>api/services/usage.py</code>, the queries in <code>db/usage_queries.py</code> and the prices in <code>agents/pricing.py</code>, all in the <a href="https://github.com/hivegate-ai/hivegate#usage-and-cost">GitHub repo</a>. The <a href="https://hivegate.dev/blog/deploy-agno-agents-production-rest-api/">Agno deploy post</a> covers sessions, API keys and streaming.</li>
</ul>
<p>If this is useful, a star helps other people find the project: <a href="https://github.com/hivegate-ai/hivegate">⭐ github.com/hivegate-ai/hivegate</a>.</p>
]]></content:encoded></item><item><title><![CDATA[Human approval before risky agent tool calls]]></title><description><![CDATA[Originally published on hivegate.dev. Letting an agent search an inbox is low risk. Letting it send an email, cancel a meeting or permanently delete a…]]></description><link>https://hivegate.hashnode.dev/human-approval-before-risky-agent-tool-calls</link><guid isPermaLink="true">https://hivegate.hashnode.dev/human-approval-before-risky-agent-tool-calls</guid><category><![CDATA[Python]]></category><category><![CDATA[AI]]></category><category><![CDATA[Open Source]]></category><category><![CDATA[agents]]></category><dc:creator><![CDATA[HiveGate]]></dc:creator><pubDate>Thu, 22 Oct 2026 17:41:43 GMT</pubDate><content:encoded><![CDATA[<p><em>Originally published on</em> <a href="https://hivegate.dev/blog/human-approval-agent-tool-calls/"><em>hivegate.dev</em></a><em>.</em></p>
<p>Letting an agent search an inbox is low risk. Letting it send an email, cancel a meeting or permanently delete a message is not. A wrong recipient or a hallucinated date becomes a real-world mistake. The usual answer is human-in-the-loop: the agent proposes the call, a person reviews it, and only then does it run.</p>
<p>This post shows how that works in <a href="https://github.com/hivegate-ai/hivegate">HiveGate</a>, an open-source FastAPI service for <a href="https://github.com/agno-agi/agno">Agno</a> agents. The run pauses, your app gets the proposed call, and a second request approves, edits or denies it.</p>
<h2>Which tools pause</h2>
<p>Agno lets a toolkit mark tools as <code>requires_confirmation</code>. When the model calls one, Agno stops the run before executing it. HiveGate's built-in toolkits mark every write:</p>
<ul>
<li><strong>Email:</strong> <code>send_email</code>, <code>create_draft</code>, <code>send_draft</code>, <code>trash_email</code>, <code>delete_email_permanently</code>, <code>modify_labels</code></li>
<li><strong>Calendar:</strong> <code>schedule_meeting</code>, <code>schedule_meeting_find_time</code>, <code>cancel_meeting</code></li>
<li><strong>Contacts:</strong> <code>create_contact</code>, <code>update_contact</code>, <code>delete_contact</code></li>
<li><strong>Drive:</strong> <code>read_file</code></li>
<li><strong>Workers:</strong> <code>run_claude_code</code>, <code>run_managed_agent</code></li>
</ul>
<p>Read-only tools such as <code>search_emails</code> and <code>list_events</code> run without a pause. Your own Agno toolkits opt in the same way, through <code>requires_confirmation_tools=[...]</code>. Tools from MCP servers don't have an approval gate yet. That's on the <a href="https://github.com/hivegate-ai/hivegate/issues/72">roadmap</a>.</p>
<h2>Step 1: the run pauses</h2>
<p>A normal chat request:</p>
<pre><code class="language-sh">curl -X POST http://localhost:8000/v2/agents/support-agent/chat \
  -H "X-API-Key: $HIVEGATE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Email jordan@acme.example that the renewal call moves to Thursday 3pm.",
    "user_id": "user-1842",
    "session_id": "inbox-1",
    "timezone": "America/Chicago",
    "locale": "en-US",
    "stream": false,
    "user_profile": {
      "profile_id": "user-1842",
      "email": "dana@acme.example",
      "full_name": "Dana Lee",
      "role": "Account manager",
      "tenant_id": "acme"
    }
  }'
</code></pre>
<p>Instead of a finished answer, the response has <code>status: "PAUSED"</code>, a <code>run_id</code>, and the pending call with its arguments (trimmed here):</p>
<pre><code class="language-json">{
  "agent_id": "support-agent",
  "session_id": "inbox-1",
  "status": "PAUSED",
  "run_id": "6f1c9a2e-...",
  "tools": [
    {
      "tool_call_id": "call_8Jx2...",
      "tool_name": "send_email",
      "requires_confirmation": true,
      "tool_args": {
        "to": ["jordan@acme.example"],
        "subject": "Renewal call moved to Thursday",
        "body_text": "Hi Jordan, the renewal call is now Thursday at 3pm CT. Thanks, Dana"
      },
      "result": null
    }
  ]
}
</code></pre>
<p>Nothing has been sent yet. With <code>"stream": true</code>, you get the same data as an SSE event whose <code>event</code> field is <code>RunPaused</code>, and then the stream ends.</p>
<p>Show <code>tool_name</code> and <code>tool_args</code> to the user, or to whoever reviews actions for them, in whatever form fits your app: a confirmation card, an editable form, a Slack message.</p>
<h2>Step 2: approve, edit or deny</h2>
<p>The decision goes to <code>POST /v2/agents/{agent_id}/chat/commit</code> with the <code>run_id</code>, the same <code>user_id</code> and <code>session_id</code>, and one entry per pending tool in <code>updated_tools</code>.</p>
<p><strong>Approve as proposed:</strong></p>
<pre><code class="language-sh">curl -X POST http://localhost:8000/v2/agents/support-agent/chat/commit \
  -H "X-API-Key: $HIVEGATE_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "run_id": "6f1c9a2e-...",
    "user_id": "user-1842",
    "session_id": "inbox-1",
    "stream": false,
    "updated_tools": [
      {"tool_call_id": "call_8Jx2...", "confirmed": true}
    ]
  }'
</code></pre>
<p>The agent resumes, the email is sent through the user's own Gmail or Outlook account, and the response has the agent's final message, <code>status: "completed"</code> and the token usage for the resumed part.</p>
<p><strong>Edit, then approve.</strong> The reviewer fixed the time, so send the corrected arguments with the approval. <code>tool_args</code> replaces the proposed arguments, so send the full set:</p>
<pre><code class="language-json">{"tool_call_id": "call_8Jx2...", "confirmed": true,
 "tool_args": {
   "to": ["jordan@acme.example"],
   "subject": "Renewal call moved to Thursday",
   "body_text": "Hi Jordan, the renewal call is now Thursday at 4pm CT. Thanks, Dana"
 },
 "confirmation_note": "Corrected time to 4pm"}
</code></pre>
<p><strong>Deny:</strong></p>
<pre><code class="language-json">{"tool_call_id": "call_8Jx2...", "confirmed": false, "confirmation_note": "Wrong recipient"}
</code></pre>
<p>If every pending tool is denied, HiveGate doesn't resume the run at all. It returns <code>status: "cancelled"</code> with the content <code>"Tool execution cancelled by user."</code>, and no model call is made. If you approve some tools and deny others, the run resumes: Agno executes the approved ones, skips the denied ones, and tells the model they were rejected.</p>
<p>The fields HiveGate accepts in each <code>updated_tools</code> entry are <code>confirmed</code>, <code>tool_args</code> (or <code>args</code>) and <code>confirmation_note</code>. Anything else is ignored.</p>
<h2>The same thing from Python</h2>
<p>A minimal backend helper, with the review step left to your UI:</p>
<pre><code class="language-python">import httpx

BASE = "http://localhost:8000/v2/agents/support-agent"
HEADERS = {"X-API-Key": HIVEGATE_KEY}

def chat(message: str, user: dict, session_id: str) -&gt; dict:
    body = {
        "message": message,
        "user_id": user["profile_id"],
        "session_id": session_id,
        "timezone": "UTC",
        "locale": "en",
        "stream": False,
        "user_profile": user,
    }
    return httpx.post(f"{BASE}/chat", headers=HEADERS, json=body, timeout=120).json()

def decide(paused: dict, user: dict, decisions: list[dict]) -&gt; dict:
    """decisions: one {"tool_call_id", "confirmed", optional "tool_args", "confirmation_note"} per pending tool."""
    body = {
        "run_id": paused["run_id"],
        "user_id": user["profile_id"],
        "session_id": paused["session_id"],
        "stream": False,
        "updated_tools": decisions,
    }
    response = httpx.post(f"{BASE}/chat/commit", headers=HEADERS, json=body, timeout=120)
    if response.status_code == 410:
        raise RuntimeError("Paused run is gone; ask the agent again.")
    response.raise_for_status()
    return response.json()

reply = chat("Cancel my 2pm with Jordan.", user, "inbox-1")
if reply["status"] == "PAUSED":
    pending = [t for t in reply["tools"] if t.get("requires_confirmation")]
    decisions = review_in_ui(pending)  # your code: returns approve/edit/deny per tool
    reply = decide(reply, user, decisions)
print(reply["content"])
</code></pre>
<p>If the chat request named a <code>model</code>, send the same <code>model</code> on commit. HiveGate uses it to find the paused agent.</p>
<h2>Where paused runs live</h2>
<p>A paused run is held in the memory of the HiveGate process that paused it, until it's committed. If the process restarts, or the commit lands on a different replica, commit returns <strong>410 Gone</strong> ("Confirmation time window has elapsed"). Two practical consequences:</p>
<ul>
<li>Run a single replica, or route each session to the same instance, when you use approvals.</li>
<li>Treat a 410 as "the proposal expired": ask the agent again rather than retrying the commit.</li>
</ul>
<p>The conversation itself is stored in Postgres, so the user doesn't lose their history when a pending action expires.</p>
<h2>The audit trail</h2>
<p>Here is what HiveGate records about a decision:</p>
<ul>
<li><strong>The run, in the agent's session.</strong> Agno stores each run in the agent's Postgres session table. A resumed run's tool executions carry the arguments that actually ran (including any edits), <code>requires_confirmation</code>, <code>confirmed</code> and the <code>confirmation_note</code>.</li>
<li><strong>The cost of the resumed run.</strong> The resumed part is metered like any other call: a row in <code>token_usage</code> with the user, session, API key name, number of tool calls, status and USD cost. You can list them with <code>GET /usage/calls?agent=support-agent&amp;order=recent</code>, which needs a key with the <code>usage:read</code> scope.</li>
<li><strong>Logs.</strong> Every commit is logged with its <code>run_id</code>, and a fully denied run is logged as such. Set <code>OTEL_TRACING_BACKEND=otlp</code> and <code>OTEL_LOGGING_BACKEND=otlp</code> (Sentry and Logtail are also supported) to send traces and logs to your observability stack.</li>
</ul>
<p>What it doesn't record is <em>which person</em> clicked approve. HiveGate sees the API key that made the commit, not your end user's identity. If you need that for compliance, record the reviewer and the decision in your own database before you call commit. You can also put the reviewer in <code>confirmation_note</code>, which is stored with the tool execution. A fully denied run is never resumed, so it leaves no usage row. Your own record and the log line are the trace for those.</p>
<h2>A note on <code>/v2/approvals</code></h2>
<p>HiveGate also has an approvals inbox at <code>/v2/approvals</code>, used by the supervisor platform. <code>GET /v2/approvals</code> lists pending requests, and <code>POST /v2/approvals/{job_id}/approve</code>, <code>/deny</code> or <code>/decide</code> records a decision. It can push requests out through notification plugins (webhook, Slack, Telegram, Discord and WhatsApp are built in; <code>GET /v2/approvals/plugins/list</code> shows them). Those requests are also held in process memory. For the agent chat flow in this post, the commit call is what resumes the run.</p>
<h2>Try it</h2>
<ul>
<li>Deploy HiveGate from the <a href="https://hivegate.dev/deploy/">deploy page</a>. Create an agent tagged <code>available_tools:email</code>, store a test Gmail token for one user (see the <a href="https://hivegate.dev/blog/per-user-google-microsoft-oauth-ai-agents/">OAuth post</a>), and ask the agent to email you. The run will pause.</li>
<li>The commit endpoint is in <code>api/routes/v2/agents.py</code> and the Agno resume logic is in <code>agents/hitl.py</code> in the <a href="https://github.com/hivegate-ai/hivegate">GitHub repo</a>.</li>
</ul>
<p>If this is useful, a star helps other people find the project: <a href="https://github.com/hivegate-ai/hivegate">⭐ github.com/hivegate-ai/hivegate</a>.</p>
]]></content:encoded></item></channel></rss>