PinnedMahmoud Hassanen·May 19The Pure JVM Era of Big Data is Ending and it’s about timeFor 15 years, the JVM (Java/Scala) was the undisputed king of Big Data. From Hadoop to Spark, Kafka and Trino if you were processing…
PinnedMahmoud Hassanen·May 1GPS for Your Data: How Apache Iceberg v4 is Turning Lakehouses into Real-Time DatabasesThe "Data Lakehouse" has always promised the best of both worlds: the massive scale of a Data Lake and the ACID compliance of a Data…
PinnedMahmoud Hassanen·Apr 12The Apple Ecosystem of Big Data: Why Spark Streaming Bypasses Your Kafka Consumer GroupStop Confusing Spark Parallelism vs Kafka Consumer Groups
PinnedMahmoud Hassanen·Jan 6The Engineering Behind the Governance: A Deep Dive into Apache Ranger + TrinoRecently, my team and I successfully deployed a Data Virtualization layer using Trino to unify our data access as highlighted by my…
PinnedMahmoud Hassanen·Nov 9, 2025From Unqueryable Chaos to Lightning-Fast Analytics: How I Optimized 170 Billion Records with…Working with Big Data often feels like trying to navigate a storm with a paper boat. Recently, I faced a real challenge: handling 3 TB of…
Mahmoud Hassanen·Dec 8, 2025From 21 Minutes to just Seconds: How a Schema Redesign + PySpark ETL Transformed Query PerformanceLarge analytical tables often evolve over time, and it’s common for early versions to use generic data types such as VARCHAR for…
Mahmoud Hassanen·Jan 4, 2025How to Store Your Data: Choosing the Right Format Like a Pro Data EngineerManaging data efficiently is crucial in today’s data-driven world. With different pipelines (batch, streaming, analytical) and destinations…
Mahmoud Hassanen·Dec 9, 2024How to Capture Changes in Data (CDC)?The simplest way to Capture Data Changes (CDC)