AI Chat Bot API, One Request at a Time: What You Send, What Comes Back, and What Each Chat Costs

AI chat bot API documentation tends to start with authentication and end with fifty parameters. What most developers and technical owners actually need is a clear picture of one round trip: what goes into a request, what comes back, why the bill grows with every message in a conversation, and what to do when the call fails. This guide walks through a single real-shaped request and response, explains the parts that drive cost, shows a minimal retry pattern, and separates model APIs from chat-platform APIs, which people mix up all the time.

1Your server

Builds instructions + knowledge + history + question

2Model API

Generates a reply, counts tokens

3Your server

Checks the reply, logs usage, sends it to the visitor

โšก In short

  • Every message is a fresh request: you send the instructions, the knowledge and the whole conversation again each time.
  • That’s why cost per chat grows faster than the number of messages. History and knowledge are usually the biggest parts.
  • Read stop_reason and usage on every response; log them.
  • Retry rate-limit and server errors with backoff, then fall back to a non-AI reply. Never show a raw error to a visitor.
Ai chat bot api in LiveChatBot: AI Agent knowledge sources and a sample chat answered from them
How it looks in LiveChatBot: AI Agent knowledge sources and a sample chat answered from them.

Two kinds of API

Model APIs and chat-platform APIs are different things

When people say “AI chat bot API” they usually mean one of two very different things, and confusing them wastes days.

A model API is what AI providers offer: you send text, you get generated text back. It knows nothing about your website, your visitors or your team. You build everything around it: the chat window, the history, the knowledge, the hand-over to staff, the storage. This guide is mostly about this kind.

A chat-platform API is what a live chat or chatbot product offers so you can connect it to your own systems. It deals with visitors, conversations, tickets and events, not with generating text. The AI is already inside the product.

For example, LiveChatBot has a JavaScript API on the page (window.LiveChatBot, with open, close, show, hide, setVisitor, sendMessage, pageView and event listeners), a REST API with webhooks for events like ticket.created and ticket.solved, and a Zapier connection. Its AI Agent already answers from your knowledge, so you don’t call a model API yourself. If you need the developer documentation, ask our team.

Which one do you need? If you want a working website assistant this week, a platform with AI built in is far less work. If you’re building a product where the conversation itself is the product, or you need full control, use a model API directly. Our make-vs-buy guide goes through the trade-offs.

The request

Anatomy of one request

A raw HTTP call to one provider’s messages endpoint. Other providers use different names for the same parts.

curl https://api.anthropic.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-5-5",
    "max_tokens": 400,
    "system": "You are the website assistant for an illustrative bakery. Answer only from the FACTS. If the facts do not cover it, say so and offer the team.\n\nFACTS:\nOpen 9 am to 9 pm daily. Delivery within 5 km, Rs 40.",
    "messages": [
      {"role": "user", "content": "Do you deliver to Indiranagar?"},
      {"role": "assistant", "content": "We deliver within 5 km of the shop for Rs 40. Could you share your pin code so I can check?"},
      {"role": "user", "content": "560038"}
    ]
  }'

Four parts matter:

  • model: which model answers. Names change over time, so keep it in configuration, not hard-coded across your code.
  • max_tokens: the ceiling on the reply length. Too low and answers get cut off mid-sentence.
  • system: your instructions and, in a simple setup, the knowledge. It’s sent with every request.
  • messages: the whole conversation so far, alternating user and assistant. The model has no memory between requests; this list *is* its memory.

The API key goes in a header and lives on your server only. Never put it in browser JavaScript, a mobile app or a public repository: anyone who finds it can spend your money.

The response

What comes back, and the fields to check

Illustrative response body, trimmed.

{
  "id": "msg_...",
  "type": "message",
  "role": "assistant",
  "content": [
    {"type": "text", "text": "I can't confirm distances from a pin code. Our team can check 560038 for you. Shall I pass this on?"}
  ],
  "stop_reason": "end_turn",
  "usage": {"input_tokens": 142, "output_tokens": 31}
}

content

A list of blocks. For a plain chat, read the text blocks and join them.

stop_reason

Why the model stopped. end_turn means it finished normally. max_tokens means it hit your ceiling and the answer is probably cut off: either raise the limit or ask for shorter answers. Some providers also return a refusal reason when a request is declined; handle it with a polite fallback.

usage

How many input and output tokens this call used. Log it with the conversation id. It’s the only reliable way to know what your chats actually cost, and it shows you quickly when a prompt change makes every call more expensive.

Why cost grows

Why the third message costs more than the first

Because the model has no memory, every request carries everything again: the instructions, the knowledge, every earlier question and every earlier answer, plus the new question. In a four-message chat, the instructions and knowledge are sent four times, and the first answer is sent three more times as history.

That has three consequences.

Long chats get expensive non-linearly. The history part grows with each turn, so a ten-message chat costs much more than ten times a one-message chat.

Knowledge is usually the biggest cost. Pasting a whole 3,000-word FAQ into every request is simple, but you pay for those words on every message. Sending only the relevant sections (retrieval) cuts this sharply. Some providers also offer prompt caching, which reduces the price of repeated text at the start of a request.

Output costs more per token. Most providers charge several times more for generated tokens than for input tokens. Short, direct answers are better for visitors and for the bill.

The estimator below makes these effects visible with your own numbers. For a deeper look at what to feed the bot, see our knowledge audit guide.

Free tool

Estimate tokens and monthly cost for your chats

Enter your typical sizes and your provider’s prices. The bars show which part of each chat costs most.

Prices: enter your provider’s current rates, in any currency. The defaults are placeholders, not real prices.

A rough estimate: real token counts depend on the model’s tokenizer. Runs in your browser; nothing is sent.

When calls fail

Errors, retries and the fallback

Every AI chat bot API fails sometimes. Plan for it in the first version.

Most failures fall into three groups:

  • Your mistake (400, 401, 403, 404): a malformed request, a wrong key, a model name that doesn’t exist. Retrying won’t help. Log it and alert yourself.
  • Too many requests (429): you’ve hit a rate limit. Wait and retry, ideally honouring a retry-after header if one is sent.
  • Their side (500, 503, 529 and similar): the service is overloaded or briefly down. Retry with exponential backoff.

The official SDKs from major providers already retry rate-limit and server errors a couple of times, so check before writing your own. If you call the HTTP endpoint directly, a minimal pattern looks like this:

import time, random

RETRYABLE = {429, 500, 502, 503, 529}

def call_with_retry(send, max_tries=4):
    """send() makes one HTTP request and returns (status, body)."""
    for attempt in range(max_tries):
        status, body = send()
        if status == 200:
            return body
        if status not in RETRYABLE:
            raise RuntimeError(f"Not retryable: {status} {body}")
        wait = (2 ** attempt) + random.random()   # 1s, 2s, 4s... plus jitter
        time.sleep(wait)
    return None   # caller shows the fallback reply instead of an error

When retries are used up, don’t show “Error 529” to a visitor. Send a fallback: a saved answer to the most likely question, or “I’m having trouble right now; leave your number and our team will call you.” That’s the same principle LiveChatBot uses: when AI replies aren’t available, a free keyword bot answers from saved replies so the chat is never silent.

Keeping answers grounded

Instructions that keep the bot honest

The API will happily answer anything. Keeping it on your facts is your job, and it lives mostly in the system instructions.

Tell it what it may use

“Answer only from the FACTS below” plus the facts. Without this, the model fills gaps from general knowledge, which is how bots invent delivery areas and discount codes.

Tell it what to do when it doesn’t know

“If the facts don’t cover the question, say you’re not sure and offer to pass it to the team.” Give it the exact words if you want consistency.

Limit the scope

“Don’t give medical, legal or financial advice. Don’t discuss competitors.” Short, clear rules work better than long lists.

Set the style

“Two or three short sentences, key fact first, in the visitor’s language.” This also keeps output costs down.

Test before launch

Keep a list of twenty real questions with expected answers and re-run them after every prompt change, as described in our golden-question regression guide.

Security basics

Five rules for any AI chat bot API integration

  1. Keys stay on the server. The browser talks to your server; your server talks to the model API. Rotate keys if one ever leaks.
  2. Limit per visitor. Cap messages per conversation and per IP so one script can’t run up your bill overnight.
  3. Don’t send what you don’t need. Strip phone numbers, emails and ID numbers from messages before they go to the model, unless the answer needs them.
  4. Treat visitor text as untrusted. Visitors can type “ignore your instructions”. Keep secrets out of the prompt entirely, so there’s nothing to leak.
  5. Log, but carefully. Store conversation ids, token usage and errors. Store message text only as long as you need it, and tell visitors in your privacy policy.

If that list feels like a lot to build and maintain, it’s a good sign a platform is the better route. LiveChatBot’s AI Agent handles the model calls, knowledge, fallback and hand-over for you, and you can connect your own systems through webhooks and Zapier. Start free on livechatbot.in.

Frequently asked questions

What is an AI chat bot API?

Usually a model API: you send instructions and a conversation, and it returns a generated reply plus token usage. Chat platforms also have APIs, but those manage visitors, chats and events rather than generating text.

Why does each message cost more than the last?

The model has no memory, so every request re-sends the instructions, knowledge and full history. The history part grows with every turn.

Where should the API key live?

Only on your server. Never in browser JavaScript, a mobile app or a public code repository.

What should happen when the API fails?

Retry rate-limit and server errors with backoff, then send a fallback reply such as a saved answer or a request for contact details.

Does LiveChatBot have an API?

Yes: a JavaScript API on the page, a REST API with webhooks such as ticket.created and ticket.solved, and Zapier. Ask our team for the developer documentation.

Keep reading

Related guides

๐Ÿ“˜

AI Bots to Talk To for Practice: Rehearse Interviews, Spoken English, Hard Calls and Sales Pitches

Read the guide โ†’

๐Ÿงญ

AI Bot Website Accessibility: Make Your Chat Usable by Keyboard, Screen Reader and Low-Vision Visitors

Read the guide โ†’

๐Ÿ› ๏ธ

AI Bot That Talks to You: A Practical Guide to Voice and Text Assistants, Used Safely

Read the guide โ†’

Understand the round trip, then decide

Build on a model API if you need full control, or use a platform with AI built in. Start free on livechatbot.in.