InTowards AIbyYuval Mehta·Oct 25TurboQuant in Production: Qdrant, vLLM, and What SurvivedQdrant keeps TurboQuant's MSE variant and vLLM drops QJL. Recall and tail-error numbers show what survives in vector search and KV cache.A response icon1A response icon1
InTowards AIbyYuval Mehta·Sep 26Fine-Tuning to Quantization: What Free-Tier Hardware Can ProveA full LLM fine-tuning lifecycle on free-tier GPUs: QLoRA results, GPTQ vs AWQ, and where serving benchmarks hit a hardware wall.A response icon1A response icon1
InLevel Up CodingbyYuval Mehta·Sep 21Agent Cost Optimization Is an Architecture Decision, Not a Model ChoiceWhen an agent’s inference bill spikes, the first instinct is almost always the same: swap to a cheaper model. Drop from a frontier model to…
InTowards AIbyYuval Mehta·Aug 25From Lumiere to Veo 3: How Google Solved the Hardest Problem in Video GenerationEvery early text-to-video model had the same tell: watch for more than three seconds and something drifts. A car’s badge changes shape. A…
InTowards AIbyYuval Mehta·Jul 26Prefill-Decode Disaggregation: When and Why to Split Your Inference StackEvery major inference engine now supports it. vLLM shipped disaggregated prefill as a stable feature. SGLang has it. LLM-d built its entire…A response icon1A response icon1
InLevel Up CodingbyYuval Mehta·Jul 20The Hidden Cost of Agent Memory: What Mem0, Zep, and Letta Don’t Tell YouEvery agent memory comparison post follows the same template: a feature table, a benchmark score, and a winner. Mem0 wins on latency, Zep…
InTowards AIbyYuval Mehta·Jul 11Reward Design Is the Hard Part: Building Verifiable Rewards for Tool-Using AgentsRLVR works beautifully for math. Generate a response, extract the final answer, compare it to ground truth, and assign 1 or 0. No reward…
InTowards AIbyYuval Mehta·Jun 30LangGraph Checkpointing Is Not Free: A Production PostmortemThe LangGraph docs make checkpointing look like a two-line concern. Add a PostgresSaver, wire it into your graph config, and you get…A response icon2A response icon2
InLevel Up CodingbyYuval Mehta·Jun 22GRPO in Production: The Failure Modes Nobody Writes AboutGRPO is now the default RL algorithm for post-training LLMs. DeepSeek-R1 popularized it, and it’s since been adopted in Nemotron 3 Super…
InTowards AIbyYuval Mehta·Jun 8Mechanistic Interpretability Is Having Its Moment: What Engineers Actually Need to KnowIt just made MIT’s top-10 breakthrough technologies list. Anthropic, DeepMind, and OpenAI are all actively using it. Here’s what…