A demo only has to work once. A feature has to work for strangers on a bad day. This is the short list I run through before an LLM feature ships.
1. What happens when the model is wrong?
Decide the failure mode in advance: a fallback answer, a visible caveat, or a human handoff. Never let a wrong answer look as confident as a right one.
2. Can you replay a bad conversation?
Log prompts, retrieved context and outputs with an ID, so a user complaint becomes a reproducible test case rather than a guess.
3. Are the inputs bounded?
Cap message length, history length and request rate. Treat everything a user or a retrieved document says as data, never as instructions.
4. Is there a cheap path?
Answer simple questions without the model where you can. It is faster, cheaper and still works when the provider does not.
5. How do you measure better?
Keep a small set of real questions with known good answers and run it before every prompt change. Without it, every edit is a coin flip.
6. Who pays when it loops?
Set timeouts, token limits and spend alerts. An agent without a stopping rule is a billing incident waiting to happen.
7. Does it say it is AI?
Tell people they are talking to a model and what it can and cannot do. Trust is cheaper to keep than to rebuild.