AI agent tools for LLM observability
Short answer: Our top pick for LLM observability: bifrost, with the highest AgentPick Score (82/100) among tools tagged "llm-observability". Runners-up: voltagent (80) and future-agi (74). Compare what each does, its license and how recently it was updated below.
| # | Tool | What it does | Score | Stars | License | Updated |
|---|---|---|---|---|---|---|
| #1 | bifrost | Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, … | 82/100 | ★ 8.8k | Apache-2.0 | 14 Oct 2026 |
| #2 | voltagent | AI Agent Engineering Platform built on an Open Source TypeScript AI Agent Framework | 80/100 | ★ 11k | MIT | 13 Oct 2026 |
| #3 | future-agi | Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Ga… | 74/100 | ★ 2.1k | Apache-2.0 | 14 Oct 2026 |
| #4 | judgeval | The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring. | 68/100 | ★ 1.1k | Apache-2.0 | 13 Oct 2026 |
| #5 | traceroot | TraceRoot - Open Source AI Agent Improvement Platform YC S25 | 66/100 | ★ 803 | See repo | 14 Oct 2026 |
| #6 | opik-mcp | Model Context Protocol (MCP) server for Opik, the open-source LLM observability and evaluation platform, built by Comet. Read traces, log scores, and … | 63/100 | ★ 222 | Apache-2.0 | 8 Oct 2026 |
| #7 | opik-openclaw | 🦞 Official plugin for OpenClaw that exports agent traces to Opik. See and monitor agent behaviour, cost, tokens, errors and more. | 62/100 | ★ 725 | Apache-2.0 | 5 Oct 2026 |
| #8 | databuff | DataBuff is an AI-native APM built on Opentelemetry，with multi-agent troubleshooting out of the box. | 60/100 | ★ 704 | AGPL-3.0 | 6 Oct 2026 |
| #9 | 1flowbase | Build for agents — from the AI Gateway to the entire application stack. | 60/100 | ★ 277 | Apache-2.0 | 14 Oct 2026 |
| #10 | OrcaReplay | OrcaReplay — Time travel for AI agents. Record, replay, fork, and debug any agent run with any model. Built by the OrcaRouter.ai team. | 57/100 | ★ 284 | Apache-2.0 | 1 Oct 2026 |
| #11 | Aegis | Runtime policy enforcement for AI agents. Cryptographic audit trail, human-in-the-loop approvals, kill switch. Zero code changes. | 51/100 | ★ 542 | MIT | 9 Oct 2026 |
| #12 | Acontext | Agent Skills as a Memory Layer | 39/100 | ★ 3.7k | Apache-2.0 | 14 Jul 2026 |
Source: each project's public GitHub repository (description, stars, license, last push). Listed tools aren't tested or ranked by AgentPick.
How to choose
- Read the license before using it commercially.
- Check how recently it was updated and how active its issues are.
- Run it on a small, low-risk task first.
Get the best new agent tools every week. Free, with a verdict on each. Get the weekly digest
Related
AI agent tools for agent memory →
Agent building
AI agent tools for self-hosted setups →
Agent building
AI agent tools for LangChain →
Agent building
AI agent tools for sandbox →
Agent building
AI agent tools for harness engineering →
Agent building
AI agent tools for penetration testing →
Agent building
AgentPick Weekly
The week's new agent tools with a verdict on each, every week. Free. We'll email you a link to confirm, and you can unsubscribe any time. See our Privacy Policy.