DED(Data Engineering Digest)·Aug 5When Spark can SPILL to a disk, Why does it still OOM?Spill is the process of temporarily writing in memory data structures (from shuffle or aggregation buffers) to local disk when an executor…
InData And BeyondbySriw World of Coding·May 11Data Engineering Interview Question : How Much of Your 32GB Executor Heap is Actually for Caching…Imagine launching a Spark job on a beefy executor with 32GB heap memory, only to watch it crash with OOM errors during a simple cache…
