Agent evaluation is often reduced to a final-answer score or an LLM-as-a-judge prompt. For real agentic systems, that is rarely enough. This talk presents a layered approach to evaluating agents across outcome correctness, environment state, trajectory quality, tool-use behavior, latency and cost, repeated-run reliability, and evaluator quality. It also looks at where deterministic checks work, where model-based judging is useful, and how trace-level evaluation can be combined with human calibration.
Tahreem Rasul is a Google Developer Expert in Generative AI and an AI/software engineer specializing in production GenAI systems, including agentic workflows, RAG, evaluation, and cloud-native AI applications. She regularly speaks at developer conferences and technical communities on practical AI engineering, with a particular interest in reliability, context engineering, retrieval, and evaluating agentic systems.
