Dilip Kumar Narayanaswamy·Oct 12Building a Portable AI Assistant on a USB Drive with Llamafile and Qwen3–4BThis article demonstrates how to create a fully portable AI assistant that runs directly from a USB drive without installation. Using…
Xero·Oct 10Fine-Tuning Meta’s NLLB-200 (1.3B)How I fine-tuned a 1.3-billion-parameter machine translation model on a single Google Colab T4 GPU and improved its SacreBLEU score from…
I E·Oct 2LLM quantization explained: what rounding the weights saves and what it costs (AI Shorts #5)Quantization makes a model smaller by rounding its weights, but this rounding costs some quality.A response icon1A response icon1
InOpen IntelligencebyDr. Fadi Shaar·Oct 2How to Make Giant AI Models Smaller Without Breaking ThemInside Tencent’s AngelSlim toolkit for quantization, speculative decoding, sparse attention, token pruning, caching, and distillation
MLEngineerByFluke·Oct 1LLM Quantization, Part 1: Understanding Post-Training Quantization (PTQ)From FP32 to INT8 and INT4: calibration, outliers, quantization strategies, and the foundations of efficient LLM inference
Rohit Kushwaha·Sep 12OpenDynamicGGUF: An Open, Measured, Reproducible Optimizer for Per-Tensor GGUF QuantizationWhy uniform Q4_K_M is leaving quality on the table — and how an open 15-step pipeline picks the right bits for every tensor, under your…
Sylvester Oguikpu·Sep 9Inside OpenAI’s 10,000-Agent Swarm That Cracked Navier-StokesWhat a 90-year-old math problem reveals about coordinating agents at extreme scale
CRAZYELON·Sep 5What Actually Happens When You Quantize an LLM?I built a bit-level floating-point simulator from scratch to find out — here’s what I learned, and the tool it turned into.
Nsharma·Aug 31The Math behind QUANTIZATION — Why shrinking a model doesn’t break it.How LLMs go from 14GB to under 4GB, and why they still mostly work.A response icon1A response icon1
