Building an Agentic Knowledge Graph for Billions of Events

Rebuilding Netflix-inspired observability with AI

Read this story for free: link

When Mike Tyson fought Jake Paul in November 2024, Netflix delivered the fight to roughly 65 million concurrent streams, while logging over 38 million events a second. So what happens when the home screen renders 1.5 seconds slower? The alert fires, yet in Netflix’s own example the fix takes about four hours, mostly rebuilding facts the tools already knew, because one tool says application where another says service. Netflix’s answer, shown at QCon London, is ontology-driven observability: a knowledge graph on one shared agreement. We rebuild it on public data and test it:

The architecture we build, end to end (Created by Fareed Khan)

Our rebuild of Netflix’s architecture has seven parts:

• A telemetry lake: 1,922,290,480 real rows in 15.24 GB of Parquet.
• Identity and ontology: one name per machine, and shapes that refuse bad facts.
• A provenance store: a named graph per run, and questions asked as of a moment.
• Enrichers and a ladder: coverage records, and a model held to a closed vocabulary.
• Correlation: chains that follow the call graph.
• Root-cause agents: specialists with typed tools, an…