Vishnu D. Pathak·1h agoServing a 70B LLM to Millions: Designing a Distributed Inference PlatformThe problem nobody warns you about
Agent Native·1d agoLocal Inference Engine That Makes Agents Faster on MacLocal models have improved enough to handle useful development work.
0xtb | Tyrrell·5d agoMy Local AI Lab: Running Large Language Models on Aging HardwarePart 2 of 2. What Strata looks like in practice, from 3-bit quants and hardware benchmarks to local agentic cybersecurity research.A response icon1A response icon1
Vishnu D. Pathak·Oct 28The Hidden Engineering Behind Cheaper and Faster AIWhy the next leap in AI may come not just from smarter models, but from better systems around them.A response icon1A response icon1
Kris Holysheva·Oct 15DeepSeek V4 Flash. Keln vs OpenRouter vs DeepSeek. How we caught a provider billing double.For 12 hours we sent the same 720 reasoning puzzles, one a minute: to Keln, to OpenRouter (:nitro) and to DeepSeek’s official API. One of…
Inhumaineeti-aibyAtanu Roy·Oct 14Chasing Lower Latency for a Fine-Tuned Whisper: From SageMaker JIT to TensorRT-LLM + TritonCo-authored by Debjit Khaskel and Atanu Roy
Dineshkumar Anandan·Aug 5Mobile Intelligence: The AI Assistant That Never Leaves Your PhoneBuilding a fully offline, ChatGPT-style document assistant on Android — from converting Gemma 4 down to a single mobile file, to hosting…
