InApache AirflowbyDavid Blain·Apr 15Rethinking partial/expand usage in Apache AirflowIn Apache Airflow, partial() and expand() (Dynamic Task Mapping) are powerful features for handling collections of work. In many scenarios…
InApache AirflowbyDavid Blain·Mar 3Scaling with Airflow 3.2: When to Defer and When to Use Native Async OperatorsIn Apache Airflow, we have long favoured dedicated operators over PythonOperators to improve ease of integration. For years, Deferrable…
InApache AirflowbyDavid Blain·Feb 25, 2025Using the MS Graph Operator to trigger and monitor an MS Fabric pipeline in Apache AirflowMicrosoft Fabric provides a unified analytics platform for data engineering, data science, real-time analytics, and business intelligence…
InApache AirflowbyDavid Blain·Feb 7, 2025How to load CSV Files in Airflow using the GenericTransfer with DuckDBOverviewA response icon1A response icon1
InApache AirflowbyDavid Blain·Oct 24, 2024Transfering data from SAP HANA to MSSQL using the Airflow GenericTransferFor the migration of historical data from SAP HANA to MSSQL, we needed an efficient solution to copy large datasets between these systems…A response icon1A response icon1
InApache AirflowbyDavid Blain·Mar 15, 2024Optimizing Integration with the MS Graph API and Power BI with the MSGraphAsyncOperator in AirflowInfrabel orchestrates a myriad of workflows, leveraging Apache Airflow as a pivotal tool for task scheduling and management.