Testing Meta Business Agent: How to check your answers before going live
The setup is complete, the knowledge base has been populated, and the instructions have been written. Now comes the step that most people skip and later regret: systematic testing. An agent that hasn’t been thoroughly tested may give customers incorrect information, respond in the wrong tone, or fail completely when faced with simple questions. This article shows you how to avoid that.
This article is part of our comprehensive Meta Business Agent Guide – there you’ll find all topics relating to the Meta Business Agent at a glance.
Why testing is so important
An AI agent behaves differently to a traditional chatbot. With a rule-based system, you know exactly what the output will be for any given input, because you programmed it that way yourself. With an LLM-based agent such as the Meta Business Agent, the behaviour is context-dependent and therefore harder to predict.
This is a strength in practice, as the agent can respond flexibly to free-form input. But it also means that you need to systematically test it before it goes live. A testing process gives you the assurance that the agent does what it’s supposed to do, and not what it isn’t supposed to do.
The built-in test mode
Meta provides an integrated test mode in the WhatsApp Business app and in Meta Business Suite. There, you can chat with the agent just like a real customer, without real customers seeing the replies. Make sure to use this mode thoroughly before activating the agent.
Test mode also shows you which knowledge source the agent used for a particular response. This is useful for understanding whether the agent is accessing the correct sections of the knowledge base or relying on generic model knowledge.
The four test areas
Section 1: Testing key questions
Start with the questions your customers ask most frequently. These are typically questions about prices, availability, opening hours and product features. The agent should answer these questions precisely, correctly and in the right tone.
Draw up a list of at least 20 typical customer questions in advance and test each one individually. Make a note of where the answers are good and where there is room for improvement.
Section 2: Testing borderline cases
Borderline cases are queries that aren’t directly answered in your knowledge base. How does the agent react when they don’t know something? Do they admit they don’t have an answer? Do they make something up? Do they escalate the query?
A well-configured agent clearly states when they don’t know something and offers to involve a human. A poorly configured agent fabricates answers that sound plausible but are incorrect. This is by far the most dangerous type of error.
Section 3: Checking the key
Does the agent sound like you? Or does it sound generic, stiff or too casual? Try out different conversation scenarios: a friendly enquiry, an impatient request, a complaint. The agent should respond consistently in your brand’s tone in all situations.
If the tone isn’t right, it’s almost always down to the instructions. Revise them and test again.
Section 4: Testing the handoff
Explicitly test the topics where the agent is supposed to hand over. Write messages specifically on these topics and check whether the handover is triggered correctly. A handover that does not work is a serious problem, particularly when dealing with sensitive matters such as complaints, legal issues or emergencies.
Typical test scenarios
In addition to these four areas, there are specific scenarios that you should always run through:
A customer asks for a product that doesn’t exist. How does the agent respond? A customer asks the same question twice during a conversation, phrasing it slightly differently each time. Is the response consistent? A customer writes in a different language than expected. Does the agent recognise this and respond correctly? A customer is frustrated and writes rudely. Does the agent remain professional?
In the video course, we’ll show you how to systematically run through and document these and other critical scenarios. There, you’ll also find a ready-made test template that you can use straight away for your business.
After the test: Iterate
No agent is perfect after the first test. This is normal and no cause for concern. The important thing is that you systematically document the weaknesses and address them in a targeted manner.
Most of the time, problems stem from one of three areas: the knowledge base does not contain the relevant information, or it is not precise enough; the instructions are unclear or contradictory; or the handoff configuration is too broad or too narrow.
Adjust the relevant section and test again. Allow for at least two to three iterations before putting the agent live.
Go-Live: Don’t do everything at once
A tip that’s often overlooked: don’t go live with the agent for all customers straight away. Use the Audience Control feature to initially activate it for only a small proportion of incoming calls – for example, just for new customers or only for customers from a specific campaign.
This allows you to observe how it performs in real-world conditions without any issues immediately affecting all customers. Only once you have gathered real data over several weeks and are satisfied with the performance should you expand the roll-out.
The Memacon® Meta Business Agent package – currently in development – will provide a comprehensive, tried-and-tested testing process complete with specific checklists, test templates and the systematic approach we have developed through hundreds of projects. Video courses, guides and best practices from the field, all in one place.
Until then: sign up now for the Memacon® Meta Business Agent newsletter using the button or the QR code below, and don’t miss any updates.
WhatsApp-Newsletter
Regular updates on the
Meta Business Agent
