Filipa·Mar 20, 2024Data Architecture (& Agreements) — from ingestion to visualisationA solid data architecture is a blueprint that helps align your company’s data with its business strategies. Build a solid one!
Filipa·Feb 27, 2024How to save a pandas DataFrame to Redshift locallyIn this article, we’ll make use of awswrangler and redshift-connector libraries to seamlessly copy data to your database locally. It goes…
Filipa·Feb 15, 2024Dynamically trim incoming data if it exceeds what specified in schemaCopy S3 to Redshift — String length exceeds DDL length: the length of the data column ‘comment’ is longer than the length defined in the…
Filipa·Feb 7, 2024Creating a Table from a Schema defined in YAMLThere is a lot we can do with YAML files. In this article, I’ll give you some ready-to-use methods to unleash the power of YAML in your…
Filipa·Feb 1, 2024Why should you automate the refresh of your Tableau dashboards in your Airflow pipelinesStop scheduled Extract Refreshes in Tableau and spare your database.
Filipa·Jan 31, 2024How to troubleshoot a failed Spark step in Amazon EMR?EMR (Elastic MapReduce) is designed for processing large-scale data using frameworks like Apache Spark. You want to use EMR when you are…