itcraftsman·23h agoThe Data Engineering Stack for 2027: What I’d Keep, What I’d DropFewer tools, more of them owned by the same company, and a “skip it if” for every recommendation I make.
angga faizul·1d agoBuilding Resilient Airflow Data Pipelines: From Task Retries to Data CheckpointsBusiness Problem
Bechir Bejaoui·4d agoSame guarantees, without programming them: Hydra ETL vs Airflow + dlt + dbt vs Airflow + Airbyte +…Most comparisons of data tools show the happy path. I wanted to see the other one: what happens when the source file is missing, when one…A response icon1A response icon1
Nicolas Pogeant·5d agodbt Knows the Order, Not the ScheduleOrdering models is dbt’s job. Deciding when anything runs, and what must finish first, belongs to the layer around it
Alexander Daniel Pratama·5d agoOrchestrate your Arcpy in AirflowFor years, all my ETL scripts did its job well. Then one day a colleague asked me to explain how it worked, and I froze because it will…
Michalzarnecki·6d agoFrom Data to Production Decisions, Models and Prototypes: Classification, Streamlit, and Airflow —…So far, we have worked mainly without ready-made answers.
Ahmed Abdulwahid·6d agoApache Airflow for Data Engineers🌪️: From Beginner to Production (Part 2)This is Part 2 of the Apache Airflow Course. Before we dive in, make sure you’ve gone through Part 1, as many of the concepts we’ll discuss…A response icon2A response icon2
Ahmed Abdulwahid·Nov 3Apache Airflow for Data Engineers🌪️: From Beginner to Production (Part 1)Imagine You’re Running a Restaurant 🍳. Every day, you must:A response icon3A response icon3
Think Data·Nov 3When Python Loops Become a Problem in AirflowOne builds tasks when Airflow parses your DAG. The other creates task instances when the DAG runs. Understanding that difference can save…A response icon1A response icon1
