9 posts found
What a large language model actually predicts
A model does not look anything up and does not decide what is true. It estimates which token comes next. Almost everythi...
Tokens, not words: how a model reads your text
Models do not see characters or words. They see tokens — and once you know how text becomes tokens, several odd behaviou...
Training, fine-tuning and prompting are three different tools
Teams reach for fine-tuning when they need context, and for prompting when they need behaviour. Knowing which problem ea...
The context window is a budget, not a memory
Bigger context windows did not give models memory. They gave you a larger envelope to fill on every single request — and...
Choosing between a large model and a small one
Most production traffic does not need the largest model available. Routing by task rather than defaulting to the top of...
Prompt caching: the cheapest speedup you are not using
If every request begins with the same two thousand tokens of instructions, you are paying full price to resend them. Ord...
Rate limits, retries and the backoff you actually need
Every AI feature meets a 429 eventually. Whether that is a blip or an outage depends on retry logic written before you n...
The cost model of an AI feature
Per-token pricing looks trivial until you multiply by retries, conversation history and the context you resend on every...
AI adoption without a rewrite
The best first AI feature is small, measurable and easy to switch off. Start where a wrong answer is cheap and the value...