How to add an AI chatbot to your website (and what it actually costs)

Putting a chat bubble on your site and wiring it to an LLM takes an afternoon. Making it answer correctly, about your product, without inventing things and without a surprise bill — that's the part worth planning.

Decide what it's for first

"An AI chatbot" isn't a goal. These are three different products:

• Support deflection — answers questions your docs already cover, so people stop emailing you.
• Sales guidance — helps a visitor pick the right product from your catalogue.
• Navigation — turns "I need X" into the right page or search result.

They need different content, different prompts, and different definitions of success. Pick one. A bot that tries to be all three is vague at all three.

Why you can't just paste your site into the prompt

The obvious approach — stuff all your content into the system prompt — breaks down fast. You pay for those tokens on every single message, models get less accurate as the context fills with irrelevant text, and eventually you simply don't fit.

The standard fix is retrieval: store your content in chunks, and at question time fetch only the handful of chunks that are actually relevant. Then the model answers from those.

What that means in practice

1. 1Split your content into chunks of a few hundred words, keeping headings attached so a chunk still makes sense alone.
2. 2Turn each chunk into an embedding and store it — pgvector on Postgres is plenty for most sites.
3. 3On each question, embed it, pull the top few matching chunks, and pass only those to the model.
4. 4Instruct the model to answer strictly from those chunks and to say it doesn't know otherwise.

That last instruction is the difference between a useful bot and an embarrassing one.

Hallucination is a product problem, not a model problem

A bot confidently inventing a feature you don't have, or a refund policy you never wrote, is worse than no bot at all. Retrieval reduces it. What actually contains it is boring discipline:

• Answer only from retrieved content — and say so in the system prompt, explicitly.
• Show sources. If it can't cite a chunk, it shouldn't be asserting it.
• Give it an exit: "I don't know, here's how to reach a human" is a valid, good answer.
• Never let it invent prices, policies or dates. Those come from your database, not the model.

What it costs

Cheaper than people expect, and in an unexpected shape. Embeddings are close to free and you pay for them once per chunk. The recurring cost is per conversation: the retrieved chunks plus the question plus the answer. Keep retrieval tight and you're in fractions of a cent per message.

The real costs are elsewhere: someone has to keep the content fresh, watch what people actually ask, and fix the answers that are wrong. A chatbot is not a feature you ship once — it's a small ongoing job.

Streaming, or it feels broken

Stream the response token by token. A bot that stares back for six seconds and then dumps a paragraph reads as broken, even when it's faster overall than one that starts typing immediately. Perceived speed is the feature.

The realistic build

A genuinely good chatbot is roughly: an ingestion job, a vector store, a retrieval endpoint, a streaming API route, a UI with history and error states, plus logging so you can see what people ask. Each piece is simple. Assembled and debugged, it's a week or two — and then it's yours to maintain.

Which is exactly why starting from something that already works, and shaping it to your content, tends to beat starting from an empty file.