Writing
Field notes
What breaks when an LLM system meets production, and the order to fix it in.Subscribe by RSS.
The half of your GenAI system nobody put on call
Nobody's RAG system breaks — it quietly stops being right. The jobs nobody wrote down, three mechanisms that make them safe, and what to put on a dashboard.
Temperature zero is not deterministic
Same prompt, same model, different answer. The mechanism behind non-determinism, the arithmetic that kills agent pilots, and what to do about it.
Your RAG doesn't have a retrieval problem. It has a ranking problem.
The document was in the index and the answer was still wrong. How to tell a retrieval failure from a ranking failure, what to fix first, and what each costs.
The gates are the product, not the orchestration
The orchestration in an AI content pipeline is the boring part. What decides whether text ships is a set of model-free functions with an exit code.