What does it mean that an LLM API is stateless?
The model keeps no memory between calls. Each request must contain everything it needs, including earlier conversation turns. Chat apps create the feeling of memory by resending the history, which also means the history uses tokens every time.
Explain it simply
Calling an LLM is like calling a helpline where a new agent picks up every time and has no notes. To continue yesterday's problem, you must retell the story at the start of the call.
Chat apps hide this. They store your messages and, on every new turn, send the whole conversation, perhaps trimmed or summarised, along with your latest question. The model reads it all fresh and replies.
Two practical results follow. Longer chats cost more and run slower because the input grows each turn. And once the history no longer fits the context window, something must be dropped or summarised, or the model will not see it.
Example
history = []
def ask(user_text):
history.append({"role": "user", "content": user_text})
reply = call_llm(messages=history) # whole history sent each time
history.append({"role": "assistant", "content": reply})
return replyThe list history lives in your code, not inside the model.
What to say in the interview
"LLM calls are stateless. The application owns the memory and resends prior turns in each request. That makes conversation length a cost and context-window concern, and it means I must decide what to keep, trim or summarise."
Common mistakes
- Saying the model remembers a user across sessions by default. It does not, unless the app stores and re-injects information.