PinnedArsen Apostolov·Dec 28, 2023Use your (PC’s) potential — expand your knowledgeHello everyone,A response icon1A response icon1
Arsen Apostolov·Sep 23Qwen3.8–27B on One RTX 3090 vs Two: +20% Decode, +14% Cold Prefill, and 3x on Cached PromptsTL;DR: Qwen3.8–27B — the dense 27.8B that dropped on August 14 — fits on a single RTX 3090 at W4A16 and decodes at ~125–155 tokens/sec…
Arsen Apostolov·Sep 20What Does a Local LLM Actually Cost per Month? I Read the Meters.One controlled experiment. One number. One verdict.
Arsen Apostolov·Sep 8Qwen3.8–27B on 2 RTX 3090: My First Local Model I Actually TrustFor years “local LLM” meant a toy — a chatbot you played with on weekends, too slow or too dumb to trust with real work. This month, for…A response icon1A response icon1
Arsen Apostolov·Aug 11Can a Local LLM Run My AI Assistant? | Towards Data ScienceTL;DR — I replayed 27 real tasks from my own AI agent against two local models, one hardware upgrade apart, scoring both against the same…
Arsen Apostolov·Jul 30Does Power Limiting Hurt LLM Inference Speed? I Swept 3× RTX 3090 From 100 W to 350 WDecode’s knee is at 175 W. Prefill keeps scaling all the way to 350. And 460 real jobs say the best power limit for a 3090 depends on…
Arsen Apostolov·Jul 8Four Whisper Fine-Tunes Later, My Idle GPU Still Doesn’t Have a JobI trained four Bulgarian Whisper models for a 5GB Quadro P2000, found a community model whose headline number looked far better than mine — until I measured it on my own eval — and still rolled everything back within five seconds. A good WER didn’t mean good transcription, and here’s the honest data.
Arsen Apostolov·Jul 53 LLM Backends, 1 RTX 3090: Who Wins the RAM-Spill Test?llama.cpp, Ollama, and vLLM benchmarked across 5 models (1B to 117B) on a single 24GB card — vLLM wins concurrency decisively, then fails…
Arsen Apostolov·Jul 3Can a Local LLM Run My AI Assistant?I replayed 27 real production tasks through qwen3-coder:30b on my home RTX 3090 to see if a self-hosted model could replace Claude as the…A response icon1A response icon1