PinnedInAWS in Plain EnglishbySidharth Kumar·Jan 7Spark Result Fragment Caching: Solving the “Every Day It Gets Slower” Problem on AWS EMRHow RFC Transforms Cumulative Workloads, Accelerates Analytics, and Slashes Compute Costs
Sidharth Kumar·Aug 13Building a Workload-Aware AWS DataSync Agent Manager: How the Scheduler Actually Works (Part 2)A deep dive into the DataSync Agent Manager’s three critical component that deliver load-aware, subnet-scoped agent selection, crash-safe…
InTowards Data EngineeringbySidharth Kumar·Aug 13Choosing the Right AWS DataSync Agent for Every Task at Petabyte Scale (Part-1)With AWS DataSync, the agent you pick for a task is locked in for good. At petabyte scale that quietly becomes a scheduling problem, here’s…A response icon1A response icon1
Sidharth Kumar·Feb 21Iceberg vs Hudi vs Delta: Why There’s No One-Size-Fits-All Table FormatFrom batch analytics to streaming upserts — how to pick the right table format with Iceberg, Hudi, or Delta.