Medium

Kuriko Iwai

Kuriko Iwai

·

Oct 27

PySpark Memory Management Explained: Architecture, OOM Causes & Troubleshooting

A deep dive into JVM heap, Python workers, shuffle, and diagnosing PySpark OOM error causes

PySpark Memory Management Explained: Architecture, OOM Causes & Troubleshooting

Kuriko Iwai

·

Oct 21

What Actually Happens When You Run a Spark Job?

A deep dive into RDDs, JVM, Stages, and Shuffle with a practical walk-through example

What Actually Happens When You Run a Spark Job?

Kuriko Iwai

·

Oct 13

How to Build Reliable RAG: A Deep Dive into 7 Failure Points and Evaluation Frameworks

Evaluate RAG pipelines and solve common failures using DeepEval, RAGAS, TruLens, and Phoenix

How to Build Reliable RAG: A Deep Dive into 7 Failure Points and Evaluation Frameworks

Kuriko Iwai

·

Oct 7

Demystifying Apache Spark — Core Mechanics & RDD Execution Guide

Master Apache Spark architecture from RDD lineage to Spark cluster deployment modes

Demystifying Apache Spark — Core Mechanics & RDD Execution Guide

Kuriko Iwai

·

Sep 29

How to Architect Semantic Chunking Pipelines for High-Performance RAG

Master critical chunking strategies for RAG to enhance retrieval accuracy and context retention

How to Architect Semantic Chunking Pipelines for High-Performance RAG

Kuriko Iwai

·

Sep 23

How to Solve Time-Varying Confounding & Mediator Bias in Longitudinal ML Pipelines

[Series] Causal Machine Learning — 3. De-Biasing Sequential Telemetry and Multi-Period Decisions

How to Solve Time-Varying Confounding & Mediator Bias in Longitudinal ML Pipelines

Kuriko Iwai

·

Jul 14

Fixing Selection Bias in Enterprise Data with Inverse Propensity Scoring (IPS)

[Series] Causal Machine Learning — 2. De-Biasing Historical Data Logs (without Expensive A/B Testing)

Fixing Selection Bias in Enterprise Data with Inverse Propensity Scoring (IPS)

Kuriko Iwai

·

Jun 25

Why Production Machine Learning Demands Causal Inference

[Series] Causal Machine Learning — 1. Rethinking the Predictive Paradigm

Why Production Machine Learning Demands Causal Inference

Kuriko Iwai

·

Mar 24

The Mathematical Memory of AI: Understanding Vector Databases and Embedding Pipelines

Explore the mechanics of vector databases, text embedding, and similarity metrics with coding examples

The Mathematical Memory of AI: Understanding Vector Databases and Embedding Pipelines

Kuriko Iwai

·

Mar 20

How to Design a Production-Ready RAG System (Architecture + Tradeoffs)

A practical guide to balancing cost, latency, and retrieval quality in modern RAG pipelines

How to Design a Production-Ready RAG System (Architecture + Tradeoffs)

Kuriko Iwai

Kuriko Iwai

Machine Learning Engineer | Building Agentic AI Pipeline | Ex-Meta | Join Newsletter => https://shorturl.at/2SXIW | Visit my website => https://kuriko-iwai.com

Help

Status

About

Careers

Press

Blog

Store

Privacy

Rules

Terms

Text to speech