xbill·Oct 16Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUsThis article provides a step by step guide to serving Gemma 4 E2B and E4B with Google’s QAT assistant drafters through llama.cpp’s…
xbill·Oct 15Inference Engineering with Gemma4This article provides a step by step tour of inference engineering across every platform I have served Gemma 4 on: Hugging Face as the…
InTowards AWSbyxbill·Sep 30Gemma 4 on Amazon SageMaker: 4-Bit Embeddings Decode up to 1.39x Faster on One L4This article repacks Gemma 4’s quantization-aware trained (QAT) weights into several 4-bit and 8-bit formats and measures each one on the…
xbill·Sep 26Google’s QAT Gemma 4 26B-A4B on One TPU v6e: 15.6x the KV Cache and 1.9x the Throughput of FP8This article provides a step by step guide to serving Google’s quantization-aware-trained (QAT) Gemma 4 26B-A4B on one Google Cloud TPU v6e…
Abu Bin Fahd·Sep 26AI Model Quantization: পার্ট ৩PTQ vs QAT: Calibration ও Fake Quantization-এর ভেতরের কাজ
