10 posts found
What a large language model actually predicts
A model does not look anything up and does not decide what is true. It estimates which token comes next. Almost everythi...
Tokens, not words: how a model reads your text
Models do not see characters or words. They see tokens — and once you know how text becomes tokens, several odd behaviou...
Embeddings explained without the linear algebra
An embedding turns text into coordinates, and nearby coordinates mean related meaning. That single idea is what makes se...
The context window is a budget, not a memory
Bigger context windows did not give models memory. They gave you a larger envelope to fill on every single request — and...
Why models hallucinate, and what actually reduces it
Hallucination is not a glitch that a better model will one day remove. It is what generation does when it has nothing to...
Giving an agent memory without giving it amnesia
Memory is not a feature you enable. It is a set of decisions about what to keep, what to summarise and what to let go —...
RAG in one page: retrieve, rank, answer
Retrieval-augmented generation is three steps and a lot of tuning. The architecture is simple; the quality lives almost...
Chunking is the part of RAG nobody tunes
Teams spend weeks on rerankers and leave chunking at the default. It is usually the other way round that pays — how you...
Prompt caching: the cheapest speedup you are not using
If every request begins with the same two thousand tokens of instructions, you are paying full price to resend them. Ord...
The cost model of an AI feature
Per-token pricing looks trivial until you multiply by retries, conversation history and the context you resend on every...