AI agent tools for evaluation
Short answer: Our top pick for evaluation: WeKnora, with the highest AgentPick Score (83/100) among tools tagged "evaluation". Runners-up: inspector (71) and Kiln (69). Compare what each does, its license and how recently it was updated below.
| # | Tool | What it does | Score | Stars | License | Updated |
|---|---|---|---|---|---|---|
| #1 | WeKnora | Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki. | 83/100 | ★ 34k | See repo | 14 Oct 2026 |
| #2 | inspector | Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps. | 71/100 | ★ 2.3k | See repo | 14 Oct 2026 |
| #3 | Kiln | Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more. | 69/100 | ★ 5.2k | See repo | 14 Oct 2026 |
| #4 | trpc-agent-go | A Go framework for building production agent systems with graph workflows, tools, memory, A2A, AG-UI, MCP, evaluation, and observability. | 68/100 | ★ 1.9k | Apache-2.0 | 14 Oct 2026 |
| #5 | opik-openclaw | 🦞 Official plugin for OpenClaw that exports agent traces to Opik. See and monitor agent behaviour, cost, tokens, errors and more. | 62/100 | ★ 725 | Apache-2.0 | 5 Oct 2026 |
| #6 | SkillEvaluator | Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live … | 62/100 | ★ 568 | Apache-2.0 | 14 Oct 2026 |
| #7 | nautilus-compass | Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) —… | 61/100 | ★ 1.3k | See repo | 14 Oct 2026 |
| #8 | TrustLLM | [ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for loc… | 53/100 | ★ 632 | MIT | 4 Oct 2026 |
| #9 | fable-method | The Fable Workflow: how Claude Fable 5 worked, distilled into skills any model can run, with the eval that keeps it honest. Think / act / prove. | 51/100 | ★ 2.3k | MIT | 3 Oct 2026 |
| #10 | WFGY | WFGY is heading toward WFGY 5.0 Polaris Protocol, a major open-source release for AI reasoning, RAG, agents, and real-world workflows. Includes Proble… | 47/100 | ★ 1.8k | See repo | 14 Oct 2026 |
| #11 | Hypha | Harness-oriented agent system framework for production-grade LLM agent applications | 43/100 | ★ 447 | Apache-2.0 | 4 Sept 2026 |
| #12 | OpenJudge | OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards | 38/100 | ★ 876 | Apache-2.0 | 11 Sept 2026 |
| #13 | meta-agents-research-environments | Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks… | 37/100 | ★ 567 | MIT | 30 Sept 2026 |
| #14 | yao-meta-skill | YAO = Yielding AI Outcomes. A rigorous engineering, evaluation, governance, and portability system for reusable agent skills. | 35/100 | ★ 2.7k | MIT | 17 Aug 2026 |
Source: each project's public GitHub repository (description, stars, license, last push). Listed tools aren't tested or ranked by AgentPick.
How to choose
- Read the license before using it commercially.
- Check how recently it was updated and how active its issues are.
- Run it on a small, low-risk task first.
Get the best new agent tools every week. Free, with a verdict on each. Get the weekly digest
Related
AI agent tools for agent memory →
Agent building
AI agent tools for self-hosted setups →
Agent building
AI agent tools for LangChain →
Agent building
AI agent tools for sandbox →
Agent building
AI agent tools for harness engineering →
Agent building
AI agent tools for penetration testing →
Agent building
AgentPick Weekly
The week's new agent tools with a verdict on each, every week. Free. We'll email you a link to confirm, and you can unsubscribe any time. See our Privacy Policy.