When someone sends me a chatbot that "gives strange answers", the cause is almost never the model. It is usually one of four things, and all of them are fixable.
1. The instructions argue with each other
Prompts grow over time. Someone adds "be brief", someone else adds "always explain your reasoning", and the model follows whichever it noticed last. The fix is to rewrite the instructions in one clean order: role, task, rules, format.
2. The model is missing the facts
If your assistant answers questions about your prices, it needs your prices in front of it. When the right document is not retrieved, the model guesses, and guesses sound confident. I check which passages are actually being passed in for a handful of failing questions.
3. There is no test set
Without a list of questions and the answers you expect, every change is a gamble. Twenty real questions in a spreadsheet is enough to tell whether an edit helped or hurt.
4. Nothing handles the edge cases
What should the assistant do when a question is off-topic, rude or about something it cannot know? If the prompt does not say, the model improvises. A short "if you are not sure, say so and offer to connect the customer with a person" rule prevents most embarrassing answers.
What you walk away with
After a review you have a cleaner prompt, a small test set and a prioritised list of changes, so you can improve the assistant calmly instead of poking at it.
If your AI feature is live and you are unhappy with it, a 45-minute review is usually the cheapest fix available.
