AI agent tools for evals
Short answer: Our top pick for evals: trulens, with the highest AgentPick Score (75/100) among tools tagged "evals". Runners-up: future-agi (74) and waku-agent (73). Compare what each does, its license and how recently it was updated below.
| # | Tool | What it does | Score | Stars | License | Updated |
|---|---|---|---|---|---|---|
| #1 | trulens | Evaluation and Tracking for LLM Experiments and AI Agents | 75/100 | ★ 3.6k | MIT | 13 Oct 2026 |
| #2 | future-agi | Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Ga… | 74/100 | ★ 2.1k | Apache-2.0 | 14 Oct 2026 |
| #3 | waku-agent | Waku Waku! Waku Agent is a local-first AI agent harness you actually own, including loop, memory, eval, all in code built to stay legible as it grows. | 73/100 | ★ 2k | MIT | 11 Oct 2026 |
| #4 | failproofai | Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement. | 71/100 | ★ 5.4k | See repo | 14 Oct 2026 |
| #5 | inspector | Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps. | 71/100 | ★ 2.3k | See repo | 14 Oct 2026 |
| #6 | Kiln | Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more. | 69/100 | ★ 5.2k | See repo | 14 Oct 2026 |
| #7 | heym | Build agentic systems. Run them with confidence. Orchestrate agents, automate business processes, inspect every execution, and keep humans in control.… | 66/100 | ★ 1.2k | See repo | 14 Oct 2026 |
| #8 | overmind | The platform for continuously improving AI agents. | 63/100 | ★ 659 | AGPL-3.0 | 14 Oct 2026 |
| #9 | agent-skills-platform | Build tested agent skills and govern their lifecycle through a user-defined marketplace: evidence, discovery, updates, rollback, quarantine, and 17-pl… | 54/100 | ★ 2.4k | MIT | 15 Sept 2026 |
| #10 | ai-engineer-notebooks | Hands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set — model APIs, structured output, tool calling… | 50/100 | ★ 699 | MIT | 8 Oct 2026 |
| #11 | agentops | Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, … | 41/100 | ★ 5.9k | MIT | 25 Jun 2026 |
| #12 | SkillForge | A skill creator that proves its skills work. Evidence-driven skill creation for Claude Code and Codex: baseline-tested generation, per-skill regressio… | 41/100 | ★ 909 | MIT | 29 Jul 2026 |
Source: each project's public GitHub repository (description, stars, license, last push). Listed tools aren't tested or ranked by AgentPick.
How to choose
- Grant the narrowest permissions the server allows.
- Check whether it runs locally or calls a remote service.
- Prefer servers listed in the official MCP registry.
Get the best new agent tools every week. Free, with a verdict on each. Get the weekly digest
Related
AI agent tools for local-first setups →
MCP servers
AI agent tools for knowledge graphs and bases →
MCP servers
AI agent tools for RAG (retrieval-augmented generation) →
MCP servers
AI agent tools for MCP client →
MCP servers
AI agent tools for cybersecurity →
MCP servers
AI agent tools for DevOps →
MCP servers
AgentPick Weekly
The week's new agent tools with a verdict on each, every week. Free. We'll email you a link to confirm, and you can unsubscribe any time. See our Privacy Policy.