Auto Chat Bot Audit: Sort Your Chats Into Automate, Assist and Never-Automate in One Afternoon

Auto chat bot projects usually go wrong in the same way: the owner switches automation on for everything, and the bot ends up answering questions it should never touch. This guide gives you a one-afternoon audit instead. You read a sample of real chats, put each kind of question into one of three lanes, and only then decide what the bot answers alone, what it starts for a person, and what it hands over straight away.

👤 What the visitor seesProduct page · 1st visit
Is COD available for 560034?
🤖 AI Agent · from “Delivery & payment”Yes, cash on delivery works for Bengaluru pin codes up to ₹5,000.
My last order came damaged 😞
Lane 3 topic: handed to your team
Your teamSorry about that, Meera. Can you share the order number? I’ll sort it now.

Illustrative chat. Simple question answered automatically, complaint handed to a person.

⚡ In short

  • Don’t automate by feature list. Automate by question type, using your own chat history.
  • Put every question type into one lane: Automate (bot answers alone), Assist (bot collects details, person finishes) or Never-automate (straight to a person).
  • Start with the five to ten most repeated Lane 1 questions. That’s usually most of the time saved.
  • Re-run the audit every quarter. Questions move between lanes as your answers and policies improve.
Auto chat bot in LiveChatBot: AI Agent knowledge sources and a sample chat answered from them
How it looks in LiveChatBot: AI Agent knowledge sources and a sample chat answered from them.

Why an audit first

Turning everything on is the most common mistake

An auto chat bot is a promise to your visitors: ask a question and you’ll get a useful answer right away. That promise only holds if the bot is answering questions it can actually get right. When a business switches automation on everywhere at once, the bot starts improvising on refunds, complaints and special requests, and the first unhappy customer usually shows up within a week.

The fix isn’t less automation. It’s automation chosen deliberately. Small businesses don’t need a strategy deck for this, just an honest look at the questions customers already ask. Most shops, clinics, coaching centres and service businesses find that a small group of questions makes up most of their chat volume: delivery times, prices, opening hours, location, payment options, how to book. Those are perfect for automation. A smaller group needs judgement, empathy or access to private records. Those should reach a person quickly.

The audit in this guide takes one afternoon. You need your last month of chats (or WhatsApp messages and phone notes if you haven’t started website chat yet), a spreadsheet and a pen. At the end you’ll have a clear list of what to automate first, the words the bot should use, and where the handovers belong.

If you haven’t set up chat yet, read how to set up live chat on your website first, then come back once you have two or three weeks of real conversations.

The three lanes

Every question type goes in exactly one lane

Use these definitions when you sort. If you’re unsure, pick the safer lane, the one further right.

⚡

Lane 1 · Automate

Examples: Opening hours, address, delivery time to a city, payment methods, price of a listed item, how to book, what documents to bring.

Rule: The answer is written down, is the same for everyone, and nobody gets hurt if the visitor reads it twice.

🤝

Lane 2 · Assist

Examples: Custom quotes, bulk orders, appointment changes, product advice with sizes, order status, warranty claims.

Rule: The bot greets, collects the details a person will need, then hands over. The person gives the final answer.

🛑

Lane 3 · Never-automate

Examples: Complaints, refunds outside policy, medical or legal worries, harassment, anything involving money already paid, upset customers.

Rule: The bot only acknowledges and passes the chat to a person, or to the away form when nobody is online.

Step 1

Collect a fair sample of real chats

You need about fifty to a hundred real conversations. Fewer than fifty and you’ll miss patterns; more than a hundred takes longer without changing the result much.

Where to find them

If you already use website chat, export or scroll through last month’s conversations in your inbox. If you’re starting fresh, use what you have: WhatsApp business chats, Instagram DMs, enquiry emails, and a week of notes on what phone callers asked. The questions are what matter, not the channel.

Make the sample fair

Don’t just pick the easy ones. Include chats from busy days and quiet days, chats that ended in a sale and chats where the visitor disappeared, and chats that came in after hours. If you run more than one website, sample each separately; a clinic site and a shop site will have very different lanes.

Strip personal details

As you copy questions into your sheet, leave out names, phone numbers, emails and order numbers. You only need the question in the customer’s own words, such as “do u deliver to Mysore by Friday”, not who asked it.

One row per question, not per chat

Many chats contain two or three questions. Split them. “What are your timings and do you take cards?” becomes two rows: hours and payment. This matters because the answer to each might sit in a different lane.

Step 2

Tag each question with a topic

Keep the topic list short. Twelve or fewer topics is ideal for a small business.

delivery-time | delivery-cost | cod | payment | price | stock | size | returns | order-status | damaged | discount | other
timings | location | fees | booking | reschedule | doctor-availability | reports | insurance | symptoms | complaint | other
fees | batch-timings | syllabus | demo-class | admission | documents | scholarship | refund | results | complaint | other
service-area | price-estimate | booking | reschedule | warranty | payment | technician | complaint | other

Copy the list that’s closest to your business and edit it. Then go through your sheet and give every question one topic. If a question doesn’t fit, use “other” and look at the “other” rows at the end; if several look alike, they deserve their own topic.

Step 3

Count, then sort topics into lanes

Now count how many questions fall under each topic, and sort the topics from most to least common. You’ll almost always see the same shape: a few topics make up most of the questions, and a long tail of topics appears only once or twice.

Next, go down the list and put each topic in a lane using three tests.

Test 1: Is the answer written down?

If the honest answer already exists on your website, price list or policy page, and it’s the same for every customer, the topic is a Lane 1 candidate. If the answer depends on the customer’s situation, it’s Lane 2 at best.

Test 2: What happens if the answer is slightly wrong?

A slightly wrong opening time is annoying. A slightly wrong refund promise costs money and trust. A slightly wrong answer about symptoms could hurt someone. The higher the cost of a mistake, the further right the topic goes.

Test 3: Does the customer need a human to feel heard?

Someone with a damaged parcel isn’t just asking for information. They want to know a real person cares. Automating that moment, even with a perfect answer, tends to make people angrier. Those topics belong in Lane 3.

When a topic passes all three tests, it goes in Lane 1. When it fails Test 1 but passes Tests 2 and 3, it goes in Lane 2. When it fails Test 2 or 3, it goes in Lane 3. Don’t argue with the tests to squeeze a topic into Lane 1. The audit only works if you’re strict.

Try it

Which lane does this question belong in?

Answer three questions about one topic from your sheet. The result tells you the lane.

1. Is the correct answer already written down, and the same for every customer?
2. If the bot got it slightly wrong, what would happen?
3. How is the customer likely to be feeling?

Free tool

Paste your questions and sort them in seconds

A first pass using simple keyword rules. Edit the sample text, add your own monthly numbers, and press the button.

Paste your customers’ questions (one per line)
Chats per month (all topics) Minutes a person spends per chat

Runs only in your browser: nothing you paste is sent anywhere. Keyword rules are a starting point, so check each line yourself.

Example result

One illustrative shop’s audit

A made-up home-textiles store, 80 questions from one month.

TopicCountLaneWhat the bot does
Delivery time171 · AutomateAnswers from the delivery page, by city
COD / payment121 · AutomateAnswers the COD limit and payment options
Price / stock101 · AutomateAnswers from product info; out-of-stock goes to Lane 2
Size / fit92 · AssistAsks bed size and room, then hands to staff
Order status82 · AssistAsks for the order number, then hands over
Bulk / hotel order62 · AssistCollects quantity, city, phone, then hands over
Returns within policy71 · AutomateExplains the 7-day return steps
Damaged item53 · NeverApologises and hands to a person immediately
Refund outside policy33 · NeverHands to the owner
Other3ReviewCheck next month

Illustrative numbers. In this example, Lane 1 topics are 46 of 80 questions, more than half the chat volume, while Lane 3 is only 8. That’s typical: automating a handful of plain, repeated topics frees up most of the team’s time, while the sensitive few still get full human attention.

Step 4

Write Lane 1 answers the bot can’t get wrong

For each Lane 1 topic, write one approved answer. This is the part most people rush, and it decides whether your auto chat bot sounds helpful or robotic.

Answer first, then detail

Start with the direct answer: “Yes, we deliver to Mysore in 3 to 5 working days.” Then add the one detail people usually ask next: “Orders placed before 2 pm ship the same day.”

Use your customers’ words

Your sheet is full of how customers actually phrase things. If people say “COD”, don’t make the answer about “pay-on-delivery services”. Matching their words helps the bot recognise the question and helps the customer recognise the answer.

Include the edge in the answer

If COD works only up to ₹5,000, say so in the same reply. Answers that hide the edge case create Lane 3 complaints later.

One topic, one source

Keep each answer in one place, ideally the page or knowledge entry the bot reads from. If you change your delivery promise during a festival rush, you change one answer and the bot follows. Our guide to writing knowledge your bot can use goes deeper on format.

In LiveChatBot, you give the AI Agent your answers as knowledge, and you can store the same answers as saved replies so the free keyword bot can still answer when AI replies aren’t available.

Before and after

Rewriting Lane 1 answers

Same facts, but the right-hand versions answer first and include the edge case.

Delivery
🚩 VagueWe offer fast and reliable shipping across India through our trusted partners.
✅ ClearMost cities get delivery in 3–5 working days. Orders before 2 pm ship the same day. North-east and J&K take 6–8 days.
Cash on delivery
🚩 VagueMultiple payment options are available for your convenience.
✅ ClearYes, COD is available on orders up to ₹5,000. Above that, please pay by UPI or card.
Returns
🚩 VaguePlease refer to our returns policy for more information.
✅ ClearYou can return unused items within 7 days of delivery. Start the return from your order page and we’ll arrange pickup.

Step 5

Set up Lane 2 so the handover arrives ready

Lane 2 is where an auto chat bot quietly saves the most effort without taking any risk. The bot doesn’t answer; it prepares. When a person picks up the chat, everything they need is already there, so a five-message back-and-forth becomes one reply.

Decide the two or three details per topic

For a bulk order: quantity, city, and a phone number. For order status: the order number. For a sizing question: bed size and fabric preference. Don’t ask for more than three things. Every extra question loses some visitors.

Use a flow, not free text

A simple chat flow with buttons (“Bulk order”, “Order status”, “Help choosing a size”) is more reliable than hoping the bot spots the topic in a long message. Buttons also show visitors what you can help with.

Tell the visitor what happens next

“Thanks! I’ve passed this to our team. They usually reply within 10 minutes during shop hours.” Setting expectations is half of good service. Our guide to live chat response time has realistic targets for small teams.

Plan for when nobody is online

Outside your opening hours, Lane 2 chats should end at the away form, which collects an email or phone number. Those messages can become tickets so nothing is lost overnight.

Step 6

Make Lane 3 handovers fast and kind

Three rules for the topics a bot should never try to resolve.

🫶

Acknowledge, don’t answer

The bot says one short, human line, such as “I’m sorry this happened. I’m getting a person to help you now.”, and stops there. No policy quotes, no troubleshooting.

🧭

Route by keyword and by mood

Words like *damaged*, *refund*, *broken*, *cheated* or *complaint* should trigger a handover, and so should repeated question marks and capital letters. Keep the list short and review it monthly.

⏰

Promise a real follow-up

If nobody is online, say when a person will reply and collect a phone or email. Then make sure someone checks those tickets first thing in the morning.

Measuring it

How to tell if the automation is working

After two to four weeks, check four simple numbers. You don’t need special analytics; your inbox and a spreadsheet are enough.

1. Share of chats the bot finished

Of all chats on Lane 1 topics, how many ended without a person stepping in, and without the visitor asking again in different words? If it’s low, the answers are probably vague or hard to find. Rewrite them.

2. Repeat questions

If visitors ask the same thing twice in one chat, the first answer didn’t land. Look at those chats first; they show you exactly which answer to fix.

3. Time to first human reply on Lane 2 and 3

Automation should make your team faster on the hard chats, not slower. If people wait longer than before, you’ve automated too much or the handover isn’t reaching anyone.

4. Complaints about the bot

Any message like “I want a real person” or “this bot is useless” is a sign. Check which topic caused it. Very often it’s a topic that belongs one lane further right.

Write the four numbers down each month. When a Lane 1 topic keeps failing, move it to Lane 2. When a Lane 2 topic turns out to be the same every time, write an approved answer and promote it to Lane 1.

Your audit checklist

Tick these off as you go

Your ticks stay in this browser tab while you work.

Auto chat bot audit

Common traps

Five mistakes that undo a good audit

Automating the long tail. It’s tempting to write answers for every topic that appeared once. Don’t. Rare questions change often and are easy to get wrong. Let a person handle them until they become common.

Copying a competitor’s bot. Their customers ask different things. Your audit is based on your chats, which is exactly why it works.

Hiding the human option. Even on Lane 1 topics, keep a visible way to reach a person. Visitors trust automation more when they know they’re not trapped in it. See our guide to telling visitors honestly when they’re talking to a bot.

Forgetting seasonal changes. Festival delivery delays, exam-season batch timings and monsoon closures all change Lane 1 answers. Put a reminder in your calendar before each busy season.

Never re-running the audit. Your products, prices and customers change. A quarterly audit takes less time than the first one because the topic list already exists.

Getting started

Run your first audit this week

Block one afternoon. Gather fifty real questions, tag them, count them, sort them into three lanes, and write approved answers for your top Lane 1 topics. You’ll finish with a clear plan for what to automate and a handover setup that protects the conversations that matter most.

LiveChatBot gives you the pieces to put the plan into practice: the AI Agent answers Lane 1 topics from the knowledge you provide, Flows collect Lane 2 details with buttons and conditions, and Lane 3 chats reach your team in the inbox or the Android app. The away form and tickets catch everything after hours. The free plan covers two websites with 100 visitors per website each month, the AI Agent with a small monthly allowance, and the free keyword bot. Start on livechatbot.in or read more about the AI Agent.

Frequently asked questions

What is an auto chat bot?

A bot on your website that answers visitors’ messages automatically, either from approved answers and saved replies or with AI that reads the knowledge you provide, and hands over to a person when needed.

How many questions should I automate at first?

Start with your five to ten most common Lane 1 topics. They usually cover more than half of your chats, and they’re easy to keep accurate.

Should a bot ever handle refunds or complaints?

Only to acknowledge them and pass them to a person. The cost of a wrong or cold answer is too high for automation.

What if nobody is online for a Lane 2 or 3 chat?

Use opening hours and the away form so the visitor leaves an email or phone number. The message can become a ticket for your team to answer first thing.

How often should I repeat the audit?

Every three months, and before any busy season when delivery times, timings or policies change.

Keep reading

Related guides

📘

Artificial Intelligence Chatbot Project: A Practical Blueprint from Brief to Demo

Read the guide →

🧭

Application Chatbot for Admissions, Jobs and Loans: Help People Finish the Form Instead of Abandoning It

Read the guide →

🛠️

AI to Chat With Customers While You Sleep: A Night-Shift Setup for Small Businesses

Read the guide →

Automate the easy half of your chats safely

Sort your chats into three lanes, then switch on automation only where it belongs. Start free on livechatbot.in.