PinnedVincent DANIEL·Oct 15, 2025Deletion vectors and Puffin files merge in the new v3 Iceberg formatTravelling Penguins team up with efficiency on better Iceberg.
PinnedVincent DANIEL·Jul 2, 2025Boost Iceberg Performance and Cut Compute Costs with Well-Scoped MERGE StatementsAvoid full-table scans, slash costs, and simplify ETL pipelines with partition-targeted merges.A response icon1A response icon1
InExpedia Group Tech BlogbyVincent DANIEL·Nov 10How I Taught an AI to Read My LogsAI-assisted data engineering at Expedia Group — and less squinting at Spark execution plansA response icon1A response icon1
Vincent DANIEL·Jul 21Optimizing CTE on Apache Spark: When to Cache and When Not toApache Spark is a powerful engine for distributed data processing, but achieving optimal performance requires careful management of…
InExpedia Group Tech BlogbyVincent DANIEL·Sep 30, 2025Why You Should Prefer MERGE INTO Over INSERT OVERWRITE in Apache IcebergStop overwriting —start merging: a smarter approach to updating Iceberg tables
InExpedia Group Tech BlogbyVincent DANIEL·Sep 2, 2025Chill Your Data with Iceberg Write Audit PublishA strategic shift toward data flows that are version-controlled, testable, and auditableA response icon1A response icon1
Vincent DANIEL·Aug 29, 2025Optimizing Spark Performance with AQE: Mastering Shuffle Partition CoalescingLearn how Adaptive Query Execution dynamically merges partitions, balances workloads, and reduces small files for faster, more efficient…
Vincent DANIEL·May 29, 2025🚀 Right-Sizing Spark Executors on EMR Instances: A Practical GuideRunning Apache Spark applications on Amazon EMR using EC2 Spot Instances offers significant cost savings, but it also introduces…
Vincent DANIEL·May 23, 2025🚀 Mastering Amazon EMR Instance Fleets: Guidelines and a Real-World Configuration ExampleAmazon EMR (Elastic MapReduce) is a powerhouse for big data processing, offering flexible and scalable clusters to run Apache Spark…
Vincent DANIEL·May 18, 2025Automating Apache Iceberg Maintenance with Spark and PythonKeeping your data lake fast, clean, and cost-efficient with a reusable script