Sitemap
Data Science Collective

Advice, insights, and ideas from the Medium data science community

Member-only story

Is Running a Local LLM Still Worth It After Haiku 5.5?

8 min read6 days ago

--

Read the article for free here.

Press enter or click to view image in full size

If you run a model on your own machine, your reason is probably one of these. It’s good enough for what I need. It’s cheaper. Nobody can change it on me. My data stays with me. I can run whatever I want.

The first two used to be easy to defend. Haiku 4.5, Anthropic’s cheap model until last week, wasn’t as good as Qwen-3.8-27B. So you’d be paying Anthropic for a weaker model than the one already running on your own card.

Anthropic released Haiku 5.5 last week at a tenth of the old Haiku’s price. On independent tests it does as well as Qwen 3.8-27B.

And the hardware has been going the other way. A used RTX 3090, the usual card for running Qwen at home, costs a lot more now than it did last year, because the memory shortage pushed up prices on anything with 24 GB on it.

So which of those reasons still hold?

Haiku matches Qwen while thinking far less. Qwen 3.8 ships set to think as hard as it can before it answers, and it…

--

--

I write deep-dives on agents, RAG, evals, and what actually breaks when you ship AI systems https://github.com/Anubhav-Goyal01

See more from me

Data Science Collective
Data Science Collective

Published in Data Science Collective

Advice, insights, and ideas from the Medium data science community