InMiro EngineeringbyRonald Ángel·Jan 26, 2023Writing data product pipelines with AirflowA framework to make your data organization transition from simple DAGs to trustworthy data products.A response icon5A response icon5
InMiro EngineeringbyRonald Ángel·Mar 14, 20225 Strategies for data workflows scheduling at MiroSome strategies we follow to make our scheduling capability escalable
InMiro EngineeringbyRonald Ángel·Nov 22, 2021Agile Data Engineering at MiroThis is how we do it: at Miro we leverage Agile concepts to drive data engineering projects. We get improved predictability, and a smoother…
InTDS ArchivebyRonald Ángel·May 5, 2020Introducing a new pySpark’s library: owl-data-sanitizerA library to democratize data quality within companies with pySpark data pipelines.
InAnalytics at INGbyRonald Ángel·Mar 23, 2020How to review ETL pySpark pipelinesA guide about how to perform pySpark code reviews to comply with python standards, guarantee data quality and keep your code extensible.
InTDS ArchivebyRonald Ángel·Dec 30, 2019Write Clean and SOLID Scala Spark Jobsnowadays extensive pipelines are written as simple SQL queries, neglecting important development concepts as writing clean and testeable…A response icon6A response icon6
InTDS ArchivebyRonald Ángel·Oct 22, 2019Understanding the Spark insertInto functionProblems found while using the spark insertInto with HiveA response icon2A response icon2
InThe StartupbyRonald Ángel·Oct 15, 2019Ingesting Raw Data with Kafka Connect and Spark DatasetsIn this blog post I will explain how we use kafka-connect and spark orchestrated by platforms like kubernetes and airflow to create a Raw…
Ronald Ángel·Mar 21, 2019Dockerizing a Hand Written Digits Predictor ServiceDockerizing a Flask service that uses keras to classify Hand Written Digits.
InTDS ArchivebyRonald Ángel·Mar 17, 2019Dataset deduplication using spark’s MLlibSpark Machine Learning: 2 approaches to deduplicate Dataframes using sparkMLib and Scala.A response icon1A response icon1