data engineering·Sep 14Apache Spark Jobs, Stages, Tasks, and Partitions ExplainedIf you’ve ever looked at a Spark application and wondered:
data engineering·Sep 12How Apache Spark WorksYou read a dataset, apply a few transformations, join some tables, aggregate the results, and write the output. The syntax is relatively…
data engineering·Aug 18ETL vs ELT for beginnersIf you are getting into data engineering, you have probably come across two terms very quickly: ETL and ELT.