PinnedInGoogle Cloud - CommunitybyYunus Durmuş·Aug 19A Practical Blueprint for Google Cloud’s Lakehouse with IcebergCatalog topology, namespace IAM gotchas, FinOps chargebacks, and making multi-tenant data mesh actually work on Google Cloud.
InGoogle Cloud - CommunitybyYunus Durmuş·Sep 12, 2025How to add modules and PIP libraries to Dataproc PySpark Serverless jobsIn this post I explain how to add pip libraries and python modules into Dataproc Spark Serverless jobs.
InGoogle Cloud - CommunitybyYunus Durmuş·Mar 24, 2025BigQuery User Defined Aggregate Functions (UDAF) to handle large precisionWhen you migrate data with precision and scale that is too large to store in BigQuery, you should store the number as string or reduce…
InGoogle Cloud - CommunitybyYunus Durmuş·Jan 2, 2025Monitor Apache Solr on DataprocI show you how to integrate Solr’s embedded Prometheus exporter with Google Cloud Monitoring and create beautiful dashboards.
InGoogle Cloud - CommunitybyYunus Durmuş·Nov 13, 2024Custom Dataproc Spark Monitoring Dashboard: Keep Your Spark Jobs HummingMonitoring Spark applications in a shared cluster is a big challenge. This dashboard helps platform teams in diagnosing the problems.A response icon1A response icon1
InGoogle Cloud - CommunitybyYunus Durmuş·Oct 15, 2024How to set Hadoop user identity in Apache Spark on Google Cloud DataprocRecently, I had an issue with setting the USER identity in Dataproc where that user identity will be used to set file owners in another…A response icon1A response icon1
InGoogle Cloud - CommunitybyYunus Durmuş·Jul 17, 2024Data Obfuscation for BigQuery Row Level Security Lookup TableBigQuery Row Level Security via lookup tables reveal the rules to all users. Data obfuscation via hashing can hide this information.A response icon2A response icon2
InGoogle Cloud - CommunitybyYunus Durmuş·Oct 16, 2023How GenAI saved days of development — an SQL query parser for BigQuery usage statisticsA 2.5 hours effort to create an SQL parser for usage statistics collection by using GenAI instead of days of coding.A response icon1A response icon1
InGoogle Cloud - CommunitybyYunus Durmuş·Aug 24, 2023How to chain and observe interdependent Data Products on Google Cloudauthors: Connor Lynch, Yunus Durmus
InGoogle Cloud - CommunitybyYunus Durmuş·Jul 20, 2023How to validate files uploaded to GCS via Transfer ApplianceWhen you upload files to GCS via Transfer Appliance, it verifies the upload of data in the background. Nevertheless, there may have issues…