PinnedInTDS ArchivebyJoão Pedro·May 1, 2024My First Billion (of Rows) in DuckDBFirst Impressions of DuckDB handling 450Gb in a real projectA response icon11A response icon11
InData Engineer ThingsbyJoão Pedro·Oct 13We need to talk about JEV (in data pipelines)Thoughts after processing 100K records in a data pipeline using JEVA response icon2A response icon2
João Pedro·Oct 3O JEV peita dados brasileiros?Analisando a performance do JEV na classificação temática de notícias em PT-BR
João Pedro·Apr 26DuckLake: A new face for DuckDBExploring its main functionalities with practical examplesA response icon1A response icon1
InTDS ArchivebyJoão Pedro·Jun 11, 2024Anatomy of Windows FunctionsTheory and practice of an underappreciated SQL operationA response icon1A response icon1
InTDS ArchivebyJoão Pedro·Jul 22, 2023Automatically Detecting Label Errors in Datasets with CleanLabA Tale of AI and wrongly-classified Brazilian Federal Laws
InTDS ArchivebyJoão Pedro·May 2, 2023Automatically Managing Data Pipeline Infrastructures With TerraformI know the manual work you did last summer
InTDS ArchivebyJoão Pedro·Apr 6, 2023Data Pipeline with Airflow and AWS Tools (S3, Lambda & Glue)Learning a little about these tools and how to integrate themA response icon2A response icon2
InTDS ArchivebyJoão Pedro·Mar 6, 2023Creating a Data Pipeline with Spark, Google Cloud Storage and Big QueryOn-premise and cloud working together to deliver a data productA response icon2A response icon2
InTDS ArchivebyJoão Pedro·Feb 16, 2023Hands-On Introduction to Delta Lake with (py)SparkConcepts, theory, and functionalities of this modern data storage frameworkA response icon3A response icon3