Idea checked

an LLM cost tracking tool

The space is crowded: LLM cost tracking is already a feature in gateways, observability tools, dashboards, and CLI/proxy utilities, so a new standalone tool needs a sharp wedge.

Confidence: high — There is a lot of fresh evidence from GitHub, Hacker News, and web pages showing many existing products and repeated user pain around cost visibility.

59 signals collected
13.3s to run
2026-08-01

hackernews 30
github 20
tavily 9

Who is already building this From data

LiteLLM is a major player: 55,238 stars and 10,239 forks, and it already includes cost tracking as part of an AI gateway.
- github BerriAI/litellm 2023-07-27
AgentOps focuses on AI agent monitoring and LLM cost tracking, with 5,747 stars and 611 forks.
- github AgentOps-AI/agentops 2023-08-15
GoModel positions itself as a LiteLLM alternative with observability, guardrails, real-time logs, usage tracking, and cost tracking, and it has 1,033 stars.
- github ENTERPILOT/GoModel 2025-12-05
Other open-source competitors with cost tracking include llmio, toktrack, Tokdash, OpenLLM Monitor, llm_cost_tracker, Routerly, llm-proxy, and tokview.
Some tools are already packaging cost tracking inside broader products: ThinkWatch, ruby_llm-agents, pydantic-ai-shields, and openclaw-dashboard all include it alongside monitoring, guardrails, or dashboards.

What people actually say From data

Founders repeatedly say they built these tools because they could not tell where LLM spend was going, or had to check multiple dashboards.
HN posts describe cost surprises as common in production, especially with agent workflows that trigger many LLM calls per user action.
- hackernews Ask HN: How are people forecasting AI API costs for agent workflows? 2026-03-11
- hackernews Ask HN: How are you monitoring AI agents in production? 2026-03-08
Web writeups say tracing is central to cost tracking because you need token counts, model IDs, tool calls, and per-span attribution.
Some users want cost tracking paired with optimization, evals, or budgeting rather than raw spend charts alone.
There is demand for local-first or self-hosted setups, with projects explicitly advertising zero-config local proxies, self-hosting, or local-first monitoring.

Where the opening is Model estimate

The model's read of the signals below — not something anyone measured.

The market looks feature-complete at the basic level: cost tracking is already bundled into gateways, tracing platforms, dashboards, and proxies, so a generic tracker is unlikely to stand out.
A possible gap is forecasting and budget enforcement for agent workflows; one HN thread explicitly asks how people forecast AI API costs for agents.
- hackernews Ask HN: How are people forecasting AI API costs for agent workflows? 2026-03-11
- hackernews Ask HN: How are people forecasting AI API costs for agent workflows? 2026-03-11
Another gap is attribution at the per-user, per-feature, or per-tool-call level inside product analytics, not just per-provider totals.
Local-first privacy and on-device use may still be under-served relative to cloud observability platforms, based on how often projects emphasize local proxies and self-hosting.

How big the market might be Model estimate

The model's read of the signals below — not something anyone measured.

This is a real category, not a niche side feature: the biggest repo in the signals, LiteLLM, is over 55k stars, and several alternatives have hundreds to thousands of stars.
The number of separate projects and repeated HN launches suggests many teams are still assembling this layer themselves instead of standardizing on one winner.
But the market also looks crowded enough that a plain cost dashboard is probably a low-margin product unless it is attached to governance, routing, evals, or billing.

What could go wrong Model estimate

The model's read of the signals below — not something anyone measured.

Direct competition from entrenched open-source gateways and observability tools is the biggest risk, especially LiteLLM and AgentOps.
The feature set is easy to copy: many projects expose similar claims around token usage, cost, latency, logs, and budget guardrails.
Several projects are already bundling cost tracking into broader systems, which makes standalone pricing harder.
If you only provide passive tracking, users may still need separate tools for evaluation, governance, redaction, routing, and enforcement.

What to do this week Model estimate

The model's read of the signals below — not something anyone measured.

Pick one narrow wedge: forecast + budget enforcement for agent workflows, or per-feature/per-user chargeback, rather than a general-purpose tracker.
Integrate as a local proxy or SDK wrapper that captures token counts, model IDs, tool spans, and USD cost, because that is the pattern users and competitors already understand.
Make the product useful without replacing the whole stack: for example, exports or connectors into existing gateways, tracing, and billing systems.
If you want a differentiator, focus on local-first privacy, self-hosting, and simple setup for teams that do not want a cloud observability vendor.
Test demand against people already complaining about surprise bills and multi-dashboard pain; those pain points are explicit in HN discussions.

Check another idea