Yair Morgenstern·Aug 2Game interfaces: data access and action validityGames have an API implicit in the UI; Making it explicit allows unifying logic and ensuring we retain coherent state
Yair Morgenstern·Jun 24Agent-ifying data enrichment pipelinesUsing LLMs to automate data gap detection and field discovery in enrichment pipelines — a research assistant, not a decision-maker.
Yair Morgenstern·May 20Migrating billions of files from s3 to SnowflakeS3 -> Snowflake cannot handle millions of files; merging thousands of jsons to larger parquets is fast, cheap and parallelizable
Yair Morgenstern·Feb 16LLMs democratize profiling… and other specialized rolesLLMs can now profile bottom-up and top-down; They can write code, by themselves, to analyze profiling data
Yair Morgenstern·Jan 25Backpressure from partitioned streams in FlinkPartitioning data streams by key allows for data locality (e.g. caching), but a single overloaded partition can backpressure the entire…
Yair Morgenstern·Jan 12Adding Python schemas with Claude and PydanticThis is a continuation of sorts to the previous two posts — Typifying Python with Claude and gradual data modeling.
Yair Morgenstern·Dec 23, 2025Data modelling as a series of trade-offsData modelling a wide spectrum; There is a clear path from schemaless to schemaful; By clarifying our goals we can make the right…
Yair Morgenstern·Dec 7, 2025Typifying 1000 Python files with Ruff + ClaudeStatic tools + Ruff + parallel(Claude-code) = Types
Yair Morgenstern·Nov 25, 2025This prompt could have been a functionMany parts of LLM prompts are deterministic steps that are better done by code.