Shilpa Thota·6d agoInside LLM Serving: How vLLM and SGLang Turn Expensive GPUs Into Scalable AI SystemsFrom transformer inference and GPU memory bandwidth to KV caching, PagedAttention, continuous batching, RadixAttention, and the economics…
Shilpa Thota·Oct 13Teaching LLMs to Use Tools Better with GRPO: A Hands-On ToolRL Experiment in Google ColabA practical Google Colab experiment with Qwen, LoRA and GRPO to understand why fine-grained rewards matter for tool-using LLMs
Shilpa Thota·Apr 27Can we reduce the LLM model size during the training?Instead of compressing a model after training, the paper proposes compressing during training by using control theory (balanced truncation…
Shilpa Thota·Apr 5Deep Understanding of Claude Code — Part 1: High-Level Design & ArchitectureFor years, AI in software development has followed a predictable pattern.
Shilpa Thota·Mar 29Run Hugging Face Models Locally Using Docker (CPU Only) with example Summarizer AppMost developers today hear about AI, LLMs, and Hugging Face… …but think:
Shilpa Thota·Feb 27HydraServe: Solving Cold Starts for Serverless LLMs in the Cloud — A Technical Deep DiveOver the past few years, serverless computing has become the go-to model for deploying scalable APIs, microservices, and bursty workloads…A response icon1A response icon1
Shilpa Thota·Feb 3TrioXpert: Automating Incident Management in Microservice Systems with LLMsModern software systems increasingly use microservice architectures — applications built from many small services that work together…
Shilpa Thota·Jan 24Next-Gen Event-Driven Architectures: Performance, Scalability, and Intelligent OrchestrationIn today’s world, systems that react to events as they happen — whether it’s a bank transaction, IoT sensor update, or AI model request —…
Shilpa Thota·Jan 17MicroRCA-Agent: Using Large Language Models to Find Root Causes in MicroservicesHow LLM agents, logs, traces, and metrics come together to solve one of cloud computing’s hardest problems
Shilpa Thota·Jan 11Simplifying Root Cause Analysis in Kubernetes with StateGraph and LLMThis article is about the research paper in the link. This research introduces a new method — called SynergyRCA — to find the root cause…A response icon1A response icon1