Your agent framework and your evaluation engine are separate choices. A LangGraph team keeps its LangGraph agent and runs evaluations through Harnest. You do not need to rewrite the agent in ADK or create a separate ADK application.
Start here
- To check known answers or tool calls, author an eval set and select a built-in metric.
- To score your own business rule, follow Create custom evals.
- To test follow-up behavior, put multiple turns in one case or use a user simulation.
- To make evaluation a CI gate, run and save results.
Choose an evaluation
Start with deterministic reference metrics where they fit. Add model-judged metrics for meaning and qualitative behavior. Judge and simulation metrics add cost and can vary between runs.
How evaluation works
1
Author an eval set
Put one or more
*.evalset.json files directly under the root agent’s evals/ directory. Each case contains either a static golden conversation or a conversation scenario.2
Select metrics
Add
evals/test_config.json. Set a threshold for every metric that should contribute to the case result.3
Run the quality gate
Run
harnest test AGENT_DIR --evals. Unit tests run first, followed by every validated eval set in filename order.4
Inspect or retain the result
Read the structured
EvalRunResult printed to the terminal, or select a JSON file with --eval-output FILE.Framework support
LangGraph agents still execute in LangGraph. Harnest converts their public responses and tool events into ADK
Invocation objects for scoring.
Reuse text eval sets and scorers across frameworks, but review tool-trajectory expectations after switching. Use smoke tests for LangGraph media or bidirectional behavior.
Deterministic metrics need no judge model. LLM judges and text simulators can use your configured model; service-backed metrics may need separate credentials. See model configuration and metric prerequisites.
Learn each part
Eval sets and configuration
Define static conversations, expected responses, tools, sessions, and shared criteria.
Metrics reference
Compare every built-in metric, score range, input requirement, and evaluation backend.
Simulations and live evaluation
Generate user turns, configure text or audio simulation, and select ADK live inference.
Create custom evals
Build and test a business-specific scorer, register it, and run it on either framework.
Custom metric API
Look up scorer arguments, result types, score intervals, and registration rules.
Run and inspect results
Run locally or in CI and capture the complete versioned JSON result.