Asindu Gayangana·Jun 23How CDF Actually Works in DatabricksStop reprocessing everything againA response icon1A response icon1
Asindu Gayangana·Dec 25, 2025Databricks Auto Loader: Best Practices for Reliable and Scalable Data IngestionThis article provides a comprehensive overview of Databricks Auto Loader and its role in building scalable, reliable, and production-ready…
Asindu Gayangana·Nov 21, 2025Building a Real-Time Data Pipeline on Google Cloud Platform (GCP)I have designed and implemented a real-time data pipeline on Google Cloud Platform (GCP). The primary objective of this project was to…
Asindu Gayangana·Oct 13, 2025Best Practices for Azure Synapse CI/CD DeploymentThis article describes how to develop a Continuous Integration and Continuous Delivery (CI/CD) pipeline for Azure Synapse Analytics using…
Asindu Gayangana·Sep 10, 2025Delta TablesA Delta Table is a modern table format built on top of Parquet files. It provides version control, schema enforcement, and schema…