The Pre-Work You Can’t Skip
Before building topics, decide: which channel (Agent Chat, MS Teams, Slack, Portal), which NLU model, who owns the training data, and what ‘success’ means. Deflection rate is the obvious metric but containment + satisfaction matter more.
Run a 2-week ticket sample. The top 5 intents usually account for 40%+ of volume. Build those first.
Topics, Not Conversations
A topic is a unit of work (request software, reset password, check ticket status). Keep topics narrow — one task per topic. Long conversational topics become unmaintainable.
Include a clear escape hatch at every step so users can reach a human.
NLU Training
Give each topic 20-40 example utterances. Include misspellings, slang, abbreviations. The pre-trained model does the heavy lifting but your examples teach it the vocabulary specific to your org.
Human Handoff
Handoff rules matter. Configure: when confidence is below X, when the user says ‘agent’ or ‘human’, when the same intent fires twice without progress. The handoff carries full conversation context so the live agent doesn’t start cold.
Measuring What Matters
Containment = no handoff needed. Deflection = no ticket created. CSAT = user satisfaction. Publish these weekly. A 60% containment rate is respectable; 80%+ is excellent but rarely achievable across all topics.