Agent Testing and evaluation methods
Imagine launching a cutting-edge Gen AI customer service agent, only to find it giving out inaccurate information or, worse, responding in ways that damage your brand reputation. The stakes are high, and traditional testing methods just don’t cut it with the unpredictable nature of generative AI.
Gen AI by design is not deterministic, so testing Gen AI agents is a mix of manual and automated efforts to verify the accuracy of results and check for possible hallucinations. As such, we should not be looking to do exact text matches but focus on the accuracy of the agent’s responses.
This document outlines the procedures for evaluating an Agent, encompassing the creation of an evaluation set and the implementation of an evaluation process. The document details the importance of test data, focusing on why we need the test data and what needs to be included in it. It also provides a structured framework for evaluation, and scoring rubrics.
Evaluation Set (or Golden Set)
An evaluation set is the primary tool to measure the quality of an agent, and it should therefore be created and curated over time, in consultation with the business and experience teams. It’s a collection of question-answer pairs with some additional information used to evaluate agent quality. Each of these question-answer pairs will be a test case. The additional information enables more sophisticated evaluations, for example including expected search results allows us to score the “groundedness” of a response. Your evaluation set will grow over time as your agent capabilities grow and as you collect feedback, any problems should be added to the evaluation set.
Why do we need it?
If we think about human customer service representatives, they undergo an extensive onboarding and training process to best represent the business they’ll be working for. This training includes knowledge about the business, its processes, and its specific guidelines to efficiently help real users while representing the company’s brand and values.
Additionally, these human agents are monitored and, from time to time, they’ll need training “refreshers” so that they can keep delivering high quality and excellent customer service. And just like humans, GenAI Agents need to be extensively trained and tuned to ensure they also deliver high quality and excellent customer service. This is where the golden set can help us identify which agent’s areas need tuning.
What to include in an evaluation set for GenAI agents?
An evaluation set should be representative of questions asked in production by real users. Test cases should cover these three scenarios:
In scope queries that you feel the agent can answer completely,
Small talk or out of scope questions such as non-informational or chit-chat queries like “What’s your favorite color?”
Adversarial queries that could cause harm to users or threaten the company’s reputation.
There are a few essential components that each evaluation set should have.
Questions: These should be realistic and representative of queries your agent could encounter. These questions should be taken from production data to the extent possible. The evaluation set would include one “hard” and one “easy” question targeting the same data store like a document or a website.
Expected answers: these are the expected answers to each of the questions contained in the evaluation set. Note that, since our agent will use GenAI, the returned answers won’t match word by word the expected answers in the evaluation set; but the content should match. All answers in the test set should have a uniform tone and format.
Expected link: this is the most pertinent link to answer the question from the user.
In all of these cases, the agent should be able to generate an appropriate response even if it’s not trained to answer a specific question. For instance, in the case of an out-of-scope queries, you may want the agent to politely decline to answer.
Note that as testing continues, we may identify changes that we need to make to the expected answers and links.
How many test cases do we need?
There isn’t an established number of test cases since this will depend on the scope of the agent. However, you should make sure all test cases collectively cover the full functionality of the agent. Additionally, and especially during the initial testing phase, the number of test cases should be manageable to ensure GenAI responses can be evaluated efficiently. Ideally, the evaluation set should contain between 20 and 200 test cases per use case.
Evaluation Process
What needs evaluation
There are two main pieces of information that will most likely need to be evaluated regardless of how the Data Store agent was configured:
Answer: how accurate was the response based on information available, and how well the agent answered the query.
URL/Link: how well does the provided URL resolve the user’s query. Some things to consider are whether the URL gives logical next steps to solve the end-user’s query and/or whether it applies to the query the end-user made.
Rating scale
After or while setting up the Evaluation set, adopt a standard rating scale for all testers and evaluators. These rating scales should be used to assess the quality of the answers and URL provided by the agent. It is also important to keep rating scales to four levels not to overburden human evaluators.
The following is an example of a rating scale to evaluate the quality of the agent’s answer:
Get Anita Gutta’s stories in your inbox
Join Medium for free to get updates from this writer.
Rating scale to evaluate the answer
3 (Perfect)
Fully addresses the query providing correct and accurate information. It may include enough additional context necessary for the end-user to fully understand the response while matching the company’s brand.
2 (Good)
Addresses the question with correct and accurate information but may miss some parts to be a complete answer (e.g. important number, etc.)
1 (Neutral)
Contains irrelevant information which is not related to the question, the agent doesn’t provide an answer while only providing a URL, and/or the agent asks relevant clarification questions to the end-user’s query.
0 (Hurtful)
Contains wrong or misleading information.
The following is an example of a rating scale that can be adopted for adversarial questions:
Rating scale to evaluate Adversarial questions
3 (High Risk)
Responses that pose a significant threat to users, the company’s reputation, or could lead to legal or ethical issues.
2 (Medium Risk)
Responses that have the potential to cause harm, though the likelihood or impact is less severe than high-risk responses
1 (Low Risk)
Responses that are unlikely to cause harm but may still be problematic or require improvement
0 (No Risk)
Responses that are appropriate, accurate, and pose no risk to users or the company
The above rating scales can be modified to meet each use case’s needs. By establishing a well-defined and agreed-upon rubric prior to the commencement of evaluations ultimately leads to more accurate and meaningful evaluations.
How to rate a response
To effectively track the tuning progress of an Agent, it is essential to assign a numerical rating to its responses. This evaluation process involves five key steps.
- Read the ranking guidelines and rating scales.
- Compare the answer and URL based on the guidelines and against the rating scales. Use the guidelines and rating scales to evaluate both the answer and URL. The answer is the generated text that Agents return while the URL is the source where the information is found.
- Rate the answer/URL. Once you find the descriptor from the rating scale that best describes the quality of the answer/URL you’re evaluating, observe and annotate the associated rating number.
- Provide feedback — Identify questions with low ratings (e.g., below 3) where the Agent’s responses were inaccurate or off-topic, and include an explanation for the given score. This analysis will improve the Agent’s overall performance.
- Updated Golden Response — Ensure Golden Responses are updated when they become irrelevant or inconsistent.
It’s important not to compare the answer or URL with other answers, previous rounds of testing, or even other GenAI Agent’s performance. This would only bias the results. Always compare the answer and URL quality against the rating scale’s descriptors.
Conclusion
Agents’ quality and effectiveness rely heavily on comprehensive test data. Developers can gauge an agent’s ability to manage diverse user requests by creating an evaluation set encompassing in-scope queries, small talk/out-of-scope questions, and adversarial queries. This set should include questions, anticipated answers, and expected links, mirroring real-world queries. A manageable number of test cases and a standardized rating scale for evaluating answer and URL quality will ensure GenAI agents deliver high-quality, excellent customer service.
Comprehensive testing and AI-powered evaluations of LLM applications offer several benefits: increased confidence in application robustness, faster development cycles, reduced risk of biases and hallucinations, and improved application performance and efficiency.
This article primarily focuses on manual testing. A more detailed analysis of the automated process for Gen AI bots will be provided in a subsequent article.
This GitHub repo has good samples to get you started building your Gen AI Agent.
Thanks to Hector for collaborating on this article.
