Adding a medical LLM made medication search worse. Here's why.
MedGemma fixed 11 searches and broke 22 that plain search already had right. A frozen benchmark, a held-out check, and a live GPU rerun show where always-on LLM reranking hurts.
POSTS / MEASURED FIELD NOTES
Claims with their setup, uncertainty, and caveats attached. A field note when an experiment has a result, with shorter laboratory notes between.
MedGemma fixed 11 searches and broke 22 that plain search already had right. A frozen benchmark, a held-out check, and a live GPU rerun show where always-on LLM reranking hurts.
Derive how Prefill, Decode, and the KV cache turn context length and active sequences into GPU-memory demand.
A distilled understanding of what happens when an LLM generates one token, with an interactive explainer.
Why a closed-loop load test can understate the queueing real arrivals experience, and how that changes my next AI-serving experiment.
“Two defensible answers to 'how fast is this box?', and whose question each answer serves.”
New field notes arrive by email when an experiment has a result.
No noise. Unsubscribe at any time.