InData Engineer ThingsbySatadru·Oct 12When you should prefer INSERT INTO over MERGE INTO!One of the common challenges in any data platform is keeping warehouse tables aligned with changes happening in source systems. Customer…A response icon1
Himanshi Dahiya·Sep 3What exactly are the conditions that make spark to chose sort merge , broadcast, shuffle or…we will understand step by step, How the each join strategy works under the hood?
Himanshi Dahiya·Aug 26Solving the Data Skew Problem using few techniquesSalting → Extremely skewed/hot keys
InT3CHbySriw World of Coding·Jul 27PySpark Scenario Interview QuestionYou write a PySpark query joining two DataFrames on what you believe is a composite key: df1.join(df2, (df1.id == df2.id) & (df1.type ==…A response icon1
InDevOps.devbyBharani·Jul 17Broadcast Join and Beyond: When Moving the Mouse Stops WorkingSpark in 60 Seconds Series
