PinnedInLearnWithNKbyNadeem Khan(NK)·Oct 27AirLLM: How a 70B Model Runs on a 4 GB GPULayer-wise inference streams one transformer layer at a time from disk. How it works, and the token-rate bill it hands you.
PinnedInLearnWithNKbyNadeem Khan(NK)·Oct 28How llm-d 1.0 Stops Agent Traffic From Recomputing the Same PromptWhy a round-robin load balancer breaks under agent traffic, and the four pieces llm-d 1.0 puts in front of vLLM to fix it.
PinnedInLearnWithNKbyNadeem Khan(NK)·Sep 24Decision Models: How Jev and Laya Decide Without Writing TextWhat sits behind a model that returns a probability per option instead of text, taken apart through Laya and set against Jev.
PinnedInLearnWithNKbyNadeem Khan(NK)·Sep 15A Framework for Deciding Where Retrieval Work HappensProfile your corpus and questions, then choose between sending everything, filtering first, indexing ahead or searching in place.
PinnedInLearnWithNKbyNadeem Khan(NK)·Aug 30Indexing Deep Dive: HNSW, Under the HoodInside HNSW’s layered graph: how greedy descent, neighbor heuristics, and the ef parameters actually deliver fast, high-recall search.A response icon3A response icon3
InLearnWithNKbyNadeem Khan(NK)·Oct 28LMCache: Keeping the KV Cache After the Request EndsHow LMCache stores KV cache in 256-token chunks, finds it again by prefix, and decides when loading beats prefill
InLearnWithNKbyNadeem Khan(NK)·Oct 12Structured Retrieval: Text-to-SQL, Tables and Mixed AnswersWhen a RAG answer lives in rows: curated views, typed tools, text-to-SQL that fails safely, and rows merged with passages
InLearnWithNKbyNadeem Khan(NK)·Oct 5Query Understanding Beyond Rewriting: Conversation Condensing, Self-Querying Filters and RoutingWhat a RAG chat pipeline decides before search: condense the conversation, skip or clarify, route, and turn the question into filters
InLearnWithNKbyNadeem Khan(NK)·Oct 4The Document Lifecycle: Versions, Deletes, Orphaned Chunks and Blue/Green Re-indexingWhat engineers actually do to keep an index correct through updates, deletes, erasure requests and full re-indexes
InLearnWithNKbyNadeem Khan(NK)·Oct 3Choosing and Operating an Embedding Model for RAGWhat engineers actually do: profile the task, filter, do the napkin math, test two or three models, and keep reindexing cheap