Choosing an AI API for chatbots

What chatbots needs

• Streaming responses so users see text immediately
• A system prompt that fixes tone and scope
• A cheap default model with a stronger fallback

Models that fit

Somnus column uses the minimum 2× credit; larger top-ups go further.

Cost example

1,000 requests of about 1,500 input and 400 output tokens on Gemini 3.8 Flash cost about $2.63 at list price — roughly $1.31 with Somnus credit.

Tips

• Keep conversation history short — summarise older turns instead of resending them.
• Start on a small model and route hard questions to a larger one.
• Cap max tokens per reply so a runaway answer can't drain your balance.

Common mistakes

• Resending the entire chat history on every turn multiplies input cost.
• Not streaming makes the bot feel slow even when it isn't.

Example

from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["SOMNUS_API_KEY"],
    base_url="https://gateway-production-c837.up.railway.app/v1",
)

resp = client.chat.completions.create(
    model="claude-sonnet-5",
    messages=[
      {"role": "system", "content": "You are a friendly, concise assistant."},
      {"role": "user", "content": "Hi! What can you help me with?"}
    ],
    stream=True,
)
for chunk in resp:
    print(chunk.choices[0].delta.content or "", end="")

Questions

Related