PinnedInData Engineer ThingsbyGeorge Zefkilis·Jul 13, 2024Building a Local Data Lake from scratch with MinIO, Iceberg, Spark, StarRocks, Mage, and DockerHello again, fellow technology enthusiasts! I am a software/data engineer who transitioned from data science. The learning curve in this…A response icon8A response icon8
PinnedInData Engineer ThingsbyGeorge Zefkilis·Oct 15, 2024Data Pipeline Development with MinIO, Iceberg, Nessie, Polars, StarRocks, Mage, and DockerHello again, fellow technology enthusiasts!A response icon2A response icon2
InData Engineer ThingsbyGeorge Zefkilis·Oct 6Query the Stream in Practice: A Local Streamhouse with Fluss, Flink and IcebergIn my previous article, An Introduction to the Streamhouse Pattern, I covered what the streamhouse pattern is and how Apache Fluss makes it…A response icon1A response icon1
InData Engineer ThingsbyGeorge Zefkilis·Aug 31Query the Stream: An Introduction to the Streamhouse PatternOver the years, a few patterns have become the default answers for building data pipelines most notably Lambda and Kappa. In a nutshell…
InData Engineer ThingsbyGeorge Zefkilis·Jun 4Building a CDC pipeline, part 3 : From Kafka events to an analytical event logIn the first 2 parts of this series (part 1, part 2) I dived into the technical details of PostgreSQL WAL and Debezium. Postgres is a…A response icon1A response icon1
InData Engineer ThingsbyGeorge Zefkilis·May 7Building a CDC pipeline, part 2: Debezium InternalsIntroductionA response icon1A response icon1
InData Engineer ThingsbyGeorge Zefkilis·Apr 25Building a CDC Pipeline, Part 1: PostgreSQL WAL InternalsIntroduction
InData Engineer ThingsbyGeorge Zefkilis·Mar 27From MapReduce to the Modern Data Lakehouse: A Complete Technical JourneyHello again, fellow tech enthusiasts.A response icon2A response icon2
InLevel Up CodingbyGeorge Zefkilis·Feb 18Building a Personal Finance Management App: Deploying to Production with Vercel, Render, and Neon —…Hello again fellow tech enthusiasts.
InData Engineer ThingsbyGeorge Zefkilis·Jan 29Building a Local Data Platform with Kubernetes and TerraformThe more time I spend working as a Data Engineer, the more I realize how much the role overlaps with that of a Data Platform Engineer…