InTowards Data EngineeringbyThink Data·Sep 16The One-Line Spark Change That Can Turn a Slow Join Into a Fast OneThe optimization worked the last time someone checked. Nobody had checked in months.A response icon2
Adhwaith Shankara Narayanan·Jan 6Apache Spark — Push Based ShuffleIn the previous blogs I had explained as to how Apache spark shuffles work. If you have not checked it, you can find it here → Apache…
Adhwaith Shankara Narayanan·Dec 15, 2025Apache Spark — Nuts & Bolts of Shuffle Part 2In my previous post, I explained what shuffle is in Apache Spark, why it occurs, and provided a high-level overview of how it works. In…A response icon1
InArt of Data EngineeringbySantosh Joshi·Feb 1, 2025How Spark Performs Joins: A Quick Look into Small and Large Table JoinsOptimizing join operations in Spark for different table sizes and distributions.
Feruz Urazaliev·Aug 27, 2024How Can You Optimize Spark Join Operations for Better Performance in Big Data?Unlock the secrets to speeding up your Spark joins and take your data pipelines to the next level
