<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vladislav Pak on Medium]]></title>
        <description><![CDATA[Stories by Vladislav Pak on Medium]]></description>
        <link>https://medium.com/@vpakspace?source=rss-204422e57dbe------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*EjCQzCpVwbpPdM9E</url>
            <title>Stories by Vladislav Pak on Medium</title>
            <link>https://medium.com/@vpakspace?source=rss-204422e57dbe------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 01:27:33 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vpakspace/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How I Built a Graph RAG System With 96.7% Accuracy in 5 Days]]></title>
            <link>https://medium.com/@vpakspace/how-i-built-a-graph-rag-system-with-96-7-accuracy-in-5-days-c1c6f622487d?source=rss-204422e57dbe------2</link>
            <guid isPermaLink="false">https://medium.com/p/c1c6f622487d</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[neo4j-graph-database]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Vladislav Pak]]></dc:creator>
            <pubDate>Wed, 18 Feb 2026 10:45:33 GMT</pubDate>
            <atom:updated>2026-02-21T03:53:00.460Z</atom:updated>
            <content:encoded><![CDATA[<h3>TL;DR</h3><p>I built a Graph RAG system that combines 5 techniques from recent research papers into a single pipeline with a declarative reasoning engine, full provenance tracing, and a typed API. Result: <strong>174/180 (96.7%)</strong> on a bilingual benchmark of 30 questions evaluated across 6 retrieval modes. Three modes achieved 100%. Zero persistent failures.</p><p><strong>GitHub</strong>: <a href="https://github.com/vpakspace/agentic-graph-rag">vpakspace/agentic-graph-rag</a></p><h3>The Problem: Why Standard RAG Falls Short</h3><p>Classic RAG — “chunk the document, create embeddings, find similar ones” — works for simple factoid questions. But it breaks down on:</p><ul><li><strong>Relationship questions</strong>: “How is method X related to component Y?” — the answer is scattered across different chunks</li><li><strong>Multi-hop reasoning</strong>: “What happens if you change A, given that A affects B, and B affects C?”</li><li><strong>Global questions</strong>: “List all 7 architectural decisions” — the answer spans 7 different parts of the document</li><li><strong>Cross-language queries</strong>: a Russian question about concepts from an English document</li></ul><p>My goal was a system that handles <em>all</em> of these question types, not just the simple ones.</p><h3>Architecture: 5 Techniques from 2025</h3><h3>1. Skeleton Indexing (<a href="https://arxiv.org/abs/2502.09304">KET-RAG</a>, KDD 2025)</h3><p><strong>Problem</strong>: extracting entities from all chunks is expensive (O(n) LLM calls).</p><p><strong>Solution</strong>: build a KNN graph from chunk embeddings -&gt; <a href="https://en.wikipedia.org/wiki/PageRank">PageRank</a> -&gt; extract entities only from the top-25% “skeletal” chunks. Peripheral chunks are linked via keyword matching.</p><pre>Chunks -&gt; KNN Graph -&gt; PageRank -&gt; Top-beta Skeletal (full extraction)<br>                                -&gt; Peripheral (keyword linking only)</pre><p><strong>Result</strong>: 75% fewer LLM calls with comparable quality. This isn’t a hack — it’s math: PageRank identifies chunks that are most “central” in the document’s semantic space.</p><h3>2. Dual-Node Structure (<a href="https://arxiv.org/abs/2502.14802">HippoRAG 2</a>, ICML 2025)</h3><p><strong>Problem</strong>: standard GraphRAG loses context from full passages. Standard RAG loses relationships between entities.</p><p><strong>Solution</strong>: two node types in <a href="https://neo4j.com/">Neo4j</a>:</p><ul><li><strong>PhraseNode</strong> — entity (name, type, PageRank score, embedding)</li><li><strong>PassageNode</strong> — full chunk text (content, embedding)</li><li><strong>MENTIONED_IN</strong> — links entities to passages</li><li><strong>RELATED_TO</strong> — co-occurrences between entities</li></ul><p>This provides both graph navigation (via PhraseNode) and full context (via PassageNode).</p><h3>3. VectorCypher Retrieval</h3><p>Hybrid retrieval in three phases, inspired by <a href="https://neo4j.com/docs/neo4j-graphrag-python/current/user_guide_rag.html">VectorCypherRetriever</a> from Neo4j GraphRAG:</p><p>1. <strong>Vector Index</strong> -&gt; find nearest PhraseNodes via cosine similarity</p><p>2. <a href="https://neo4j.com/docs/cypher-manual/current/introduction/"><strong>Cypher</strong></a><strong> Traversal</strong> -&gt; expand through RELATED_TO (up to 3 hops)</p><p>3. <strong>PassageNode Collection</strong> -&gt; gather linked passages -&gt; GraphContext</p><p>Key insight: cosine re-ranking using actual PassageNode embeddings from Neo4j beats <a href="https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf">RRF fusion</a>.</p><h3>4. Agentic Router with Self-Correction</h3><p>Three routing tiers with cascading fallback:</p><p><strong>Tier | Method | Confidence | Description</strong><br>1 | <strong>Mangle</strong> (<a href="https://en.wikipedia.org/wiki/Datalog">Datalog</a>) | 0.7 | 65 bilingual keywords<br>2 | <strong>LLM</strong> (GPT-4o-mini) | 0.85 | Neural network classification<br>3 | <strong>Pattern</strong> (regex) | 0.5 | Regex patterns as fallback</p><p>If retrieval quality falls below a threshold (relevance &lt; 2.0 out of 5), the system escalates through a tool chain:</p><pre>vector_search -&gt; cypher_traverse -&gt; hybrid_search -&gt; comprehensive_search -&gt; full_document_read</pre><p>Each attempt rephrases the query via LLM. Best results are tracked across all attempts.</p><h3>5. PyMangle — A Datalog Engine in Python</h3><p>A full reimplementation of <a href="https://github.com/google/mangle">Google Mangle</a> (2,919 lines):</p><ul><li><a href="https://github.com/lark-parser/lark">Lark</a>-based parser with custom grammar</li><li>Semi-naive evaluation with <a href="https://en.wikipedia.org/wiki/Stratification_(mathematics">stratified negation</a>#In_logic)</li><li>35+ built-in functions (arithmetic, strings, lists, maps)</li><li>Temporal evaluation</li><li>Filter pushdown for external predicates</li></ul><p>Three rule files:</p><ul><li>routing.mg -- query routing (65 keywords)</li><li>access.mg -- RBAC (role inheritance + permit/deny)</li><li>graph.mg -- graph inference (reachable, common_neighbor, evidence)</li></ul><pre>% Transitive closure over the graph<br>reachable(X, Y, 1) :- edge(X, R, Y).<br>reachable(X, Z, D) :- reachable(X, Y, D1), edge(Y, R, Z),<br>    D = fn:plus(D1, 1), D &lt; 5.</pre><pre>% Common neighbors of two entities<br>common_neighbor(A, B, N) :- edge(A, R1, N), edge(B, R2, N), A != B.</pre><h3>Benchmark: From 38% to 96.7% in 10 Iterations</h3><h3>Benchmark Design</h3><ul><li><strong>30 questions</strong>: 7 simple, 7 relation, 6 multi_hop, 6 global, 4 temporal</li><li><strong>2 documents</strong>: Doc1 (Russian, knowledge graph) + Doc2 (English, SCL architecture)</li><li><strong>6 retrieval modes</strong>: vector, cypher, hybrid, agent_pattern, agent_llm, agent_mangle</li><li><strong>180 evaluations</strong> (30 x 6) via hybrid judge: embedding similarity + keyword overlap + <a href="https://arxiv.org/abs/2306.05685">LLM-as-judge</a></li></ul><h3>Results Evolution</h3><pre>v3:  38%  ████░░░░░░░░░░░░░░░░  Baseline (questions in EN, documents in RU)<br>v4:  67%  █████████░░░░░░░░░░░  +29pp  --  questions in RU (language match!)<br>v5:  73%  ██████████░░░░░░░░░░  +6pp   --  comprehensive_search for global<br>v10: 65%  █████████░░░░░░░░░░░  -8pp   --  added 15 new questions<br>v11: 80%  ████████████░░░░░░░░  +15pp  --  enumeration prompt<br>v12: 93%  ██████████████████░░  +13pp  --  hybrid judge<br>v14: 96.7%███████████████████░  +3.7pp  --  semantic judge</pre><h3>Final Results (v14)</h3><p><strong>Mode | Score</strong><br><strong>Vector</strong> | <strong>30/30 (100%)</strong> | Pure embedding search<br><strong>Hybrid</strong> | <strong>30/30 (100%)</strong> | Vector + Graph<br><strong>Agent (Mangle)</strong> | <strong>30/30 (100%)</strong> | Datalog rules<br>Agent (LLM) | 29/30 (96%) | GPT-4o-mini router<br>Agent (Pattern) | 28/30 (93%) | Regex patterns<br>Cypher | 27/30 (90%) | Graph traversal<br><strong>Total</strong> | <strong>174/180 (96.7%)</strong> | <strong>0 persistent failures</strong></p><h3>10 Optimization Lessons</h3><h3>1. Question Language = Document Language (+29pp)</h3><p>The single biggest improvement in the project’s history. English questions about a Russian document scored 38%. Switching to Russian questions — 67%. Embeddings handle cross-language search well, but the LLM generator loses context.</p><h3>2. Failures Are Not Retrieval — They’re Generation + Evaluation</h3><p>Key insight from v11: for global questions, ALL required keywords were present in the top-30 chunks. The problem was that the generator didn’t enumerate all items, and the judge truncated the answer to 500 characters.</p><h3>3. CoT Prompt for Judge — A Disaster</h3><p>Trying to make the judge “smarter” via <a href="https://arxiv.org/abs/2201.11903">Chain-of-Thought</a> (“list found keywords -&gt; count -&gt; give verdict”) caused a regression from 144/180 to 48/180. GPT-4o-mini literally searched for English strings in Russian text. A simpler “match CONCEPTS, not strings” prompt works 3x better.</p><h3>4. Cosine Re-ranking Beats RRF</h3><p>Hybrid search with <a href="https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf">Reciprocal Rank Fusion</a> produced worse results than cosine re-ranking using actual embeddings from Neo4j. RRF is great for combining different signals, but when both signals are embedding-based, direct cosine similarity is more accurate.</p><h3>5. Embedding Similarity for Judge (Threshold 0.65)</h3><p>For questions with a reference answer: cosine similarity between system answer and reference &gt;= 0.65 -&gt; auto-PASS. Calibration: correct answer ~0.677, incorrect answer (wrong document) ~0.570. The 0.65 threshold separates them perfectly.</p><h3>6. Cross-Language Routing</h3><p>A Russian question about concepts from an English document (Doc2/SCL) breaks vector_search — it returns Doc1. Solution: detect cross-language global query -&gt; route directly to full_document_read instead of vector_search.</p><h3>7. Comprehensive Search Dilutes Results</h3><p>comprehensive_search (multi-query fan-out) generates N sub-queries -&gt; each via vector_search -&gt; RRF merge. But if all sub-queries return Doc1, then the single full_document_read result for Doc2 drowns in the RRF merge.</p><h3>8. Self-Correction Loop Must Preserve the Best</h3><p>Early bug: each attempt overwrote previous results. If attempt 1 scored 2.5 and attempt 2 scored 1.8, the system returned 1.8. Fix: track best_results and best_score across all attempts.</p><h3>9. Enumeration Prompt — A Special Format</h3><p>For global questions (“list all…”), a regular prompt generates prose, not a list. A special enumeration prompt: “Output a numbered list. Scan ALL chunks. Do not stop early.”</p><h3>10. Judge Limit 500 -&gt; 2000 Characters</h3><p>Truncating the answer to 500 characters for the judge killed enumeration answers (7 items ~ 1500 characters). Increasing to 2000 — an instant +5pp.</p><h3>Typed API and Provenance</h3><p>Every query creates a PipelineTrace:</p><pre>{<br>  &quot;trace_id&quot;: &quot;tr_abc123def456&quot;,<br>  &quot;router_step&quot;: {<br>    &quot;method&quot;: &quot;mangle&quot;,<br>    &quot;decision&quot;: {&quot;query_type&quot;: &quot;simple&quot;, &quot;suggested_tool&quot;: &quot;vector_search&quot;}<br>  },<br>  &quot;tool_steps&quot;: [{<br>    &quot;tool_name&quot;: &quot;vector_search&quot;,<br>    &quot;results_count&quot;: 10,<br>    &quot;relevance_score&quot;: 3.2,<br>    &quot;duration_ms&quot;: 150<br>  }],<br>  &quot;escalation_steps&quot;: [],<br>  &quot;generator_step&quot;: {<br>    &quot;model&quot;: &quot;gpt-4o-mini&quot;,<br>    &quot;confidence&quot;: 0.82<br>  },<br>  &quot;total_duration_ms&quot;: 1800<br>}</pre><p>API: <a href="https://fastapi.tiangolo.com/">FastAPI</a> REST (/api/v1/) + <a href="https://github.com/jlowin/fastmcp">FastMCP</a> (SSE/MCP) -- for both REST clients and AI agents.</p><h3>Project Numbers</h3><p><strong>Metric | Value</strong><br>Python LOC | 16,206 (118 files)<br>Tests | 586 (320 core + 108 PyMangle + 158 rag-core)<br>Dependencies | 26 packages<br>Benchmark iterations | 10 (v2 -&gt; v14)<br>Result files | 15 JSON (~4.7 MB)<br>Mangle rules | 111 lines (3 files)<br>Classes | 36 (14 data models, 8 services, 6 config, 4 reasoning)</p><h3>What’s Next</h3><ul><li><a href="https://en.wikipedia.org/wiki/Personalized_PageRank"><strong>Personalized PageRank</strong></a> for query-focused graph traversal</li><li><strong>Human evaluation</strong> alongside LLM-as-judge</li><li><strong>Streaming responses</strong> in the <a href="https://streamlit.io/">Streamlit</a> UI</li><li><strong>More Mangle rules</strong> — temporal reasoning, conflict resolution</li></ul><h3>Tech Stack</h3><p><strong>Component | Technology</strong><br>LLM | <a href="https://platform.openai.com/docs/models">OpenAI GPT-4o / GPT-4o-mini</a><br>Embeddings | <a href="https://platform.openai.com/docs/guides/embeddings">text-embedding-3-small</a> (1536 dim)<br>Graph DB | <a href="https://neo4j.com/">Neo4j 5.x</a> (Vector Index + Cypher)<br>Reasoning | PyMangle (<a href="https://github.com/google/mangle">Datalog</a>, 2,919 LOC)<br>Doc Parsing | <a href="https://github.com/docling-project/docling">Docling</a> (PDF/DOCX/PPTX + GPU)<br>Graph Algorithms | <a href="https://networkx.org/">NetworkX</a> (PageRank, KNN, PPR)<br>API | <a href="https://fastapi.tiangolo.com/">FastAPI</a> (REST) + <a href="https://github.com/jlowin/fastmcp">FastMCP</a> (SSE/MCP)<br>UI | <a href="https://streamlit.io/">Streamlit</a> (7 tabs)<br>Testing | <a href="https://docs.pytest.org/">pytest</a> (586 tests) + <a href="https://github.com/astral-sh/ruff">ruff</a><br>CI/CD | <a href="https://github.com/features/actions">GitHub Actions</a></p><p><em>If you’re interested in implementation details or want to discuss Graph RAG — leave a comment or open an issue on </em><a href="https://github.com/vpakspace/agentic-graph-rag"><em>GitHub</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c1c6f622487d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How We Extract Hidden Ontologies from Millions of Power BI Dashboards and Make Them Accessible to…]]></title>
            <link>https://medium.com/@vpakspace/how-we-extract-hidden-ontologies-from-20-million-power-bi-dashboards-9a49c1635198?source=rss-204422e57dbe------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a49c1635198</guid>
            <dc:creator><![CDATA[Vladislav Pak]]></dc:creator>
            <pubDate>Sat, 07 Feb 2026 12:45:58 GMT</pubDate>
            <atom:updated>2026-02-07T14:24:01.137Z</atom:updated>
            <content:encoded><![CDATA[<h3>How We Extract Hidden Ontologies from Millions of Power BI Dashboards and Make Them Accessible to AI Agents</h3><blockquote><strong><em>TL;DR</em></strong><em>: We built an open-source tool that automatically extracts semantic models from Power BI .pbix files and transforms them into formal OWL ontologies. This enables AI agents to work safely with enterprise data by understanding business rules, relationships, and access constraints. </em><em>pip install powerbi-ontology-extractor â€” and in 10 minutes you have an ontology.</em></blockquote><h3>The Problem: Schema Drift Can Cost Millions</h3><blockquote><strong><em>Illustrative estimate.</em></strong><em> The $4.6M figure below is a modeled scenario, not a verified incident. Calculation: ~15,000 misrouted shipments x $300 avg rerouting cost = $4.5M + operational downtime. Real-world impact depends on industry, data volume, and detection speed.</em></blockquote><p>Consider a logistics company where a database administrator renames a column from Warehouse_Location to FacilityID. An AI agent managing routing doesn&#39;t know about the rename and starts shipping cargo to random addresses. In a modeled scenario with 15,000 daily shipments, this can accumulate to $4.6M in losses before the error is detected.</p><p>This case isn’t an exception â€” it’s a symptom of a systemic problem. There are <strong>millions of</strong> Power BI dashboards worldwide. Each one contains a semantic model â€” tables, relationships, measures, security rules. In essence, every Power BI dashboard is an <strong>informal ontology</strong>, locked inside a proprietary .pbix file.</p><p>AI agents can’t read .pbix files. They don’t know that Revenue in the sales department and Revenue in finance are different metrics with different formulas. They don&#39;t understand that an analyst has read-only access, while only an admin can delete records. Without this knowledge, an AI agent is a blind robot with access to a production database.</p><h3>The Scale of the Problem</h3><p>MetricValuePower BI dashboards worldwideMillionsAverage cost of manual ontology creation$50Kâ€“$200KPercentage of ontology that can be extracted automatically~70%Percentage requiring manual work by a business analyst~30%</p><p>Manual ontology creation takes months of expensive specialist work. Yet 70% of the effort is mechanical extraction of what’s already described in Power BI: tables become entities, columns become properties, foreign keys become relationships, DAX formulas become business rules.</p><h3>The Solution: 30 Minutes Instead of 3 Months</h3><p>We built <strong>PowerBI Ontology Extractor</strong> â€” an open-source Python tool that automates those 70% and provides a visual editor for the remaining 30%.</p><p>Solution architecture:</p><pre>Power BI (.pbix)  â†’  Ontology Extractor  â†’  OntoGuard  â†’  AI Agent<br>      10 min              10 min             5 min          5 min</pre><p>What happens at each stage:</p><ol><li><strong>Extraction</strong> (10 min): The binary .pbix file is unpacked, DataModel is parsed (via PBIXRay), extracting tables, columns, data types, relationships, DAX measures, and RLS rules</li><li><strong>Generation</strong> (10 min): Raw data is transformed into a formal ontology â€” classes, properties, relationships with cardinality, business rules from DAX, constraints</li><li><strong>Validation</strong> (5 min): The ontology passes through OntoGuard (a semantic firewall) that enforces role-based access control via OWL rules</li><li><strong>Deployment</strong> (5 min): The AI agent receives the ontology as a contract â€” and now knows what it can and cannot do</li></ol><h3>Installation</h3><pre>pip install powerbi-ontology-extractor</pre><p>That’s it. No Docker, no server configuration, no paid APIs (except optional OpenAI for chat). One command â€” and you have a CLI, Python API, and Streamlit UI.</p><h3>Conceptual Foundation</h3><p>PowerBI Ontology Extractor translates a Power BI semantic model into an explicit ontological artifact â€” something that can be queried, validated, versioned, and used as a “semantic contract” for AI agents. The starting idea here comes from a series of articles by Pankaj Kumar: at enterprise scale, BI models are effectively informal ontologies that can be formalized and turned into an infrastructure asset.</p><p>The practical value lies in reliability. When meaning is externalized, it becomes possible to perform “semantic checks” rather than just “schema checks” â€” for example, business logic validation and safer model changes during field and name drift. This aligns well with Kumar’s ideas on semantic validation (<a href="https://github.com/vpakspace/ontoguard-ai">OntoGuard</a>) and production infrastructure for agents at the intersection of MCP and ontologies (<a href="https://github.com/vpakspace/universal-agent-connector">Universal Agent Connector</a>).</p><h3>Under the Hood</h3><h3>Parsing .pbix Files</h3><p>Power BI stores data in a binary format. We use the <a href="https://github.com/pankajkumar/pbixray">PBIXRay</a> library to read the DataModel, then extract:</p><pre>from powerbi_ontology import PowerBIExtractor</pre><pre>extractor = PowerBIExtractor(&quot;sales_dashboard.pbix&quot;)<br>model = extractor.extract()</pre><pre>print(f&quot;Tables:        {len(model.entities)}&quot;)<br>print(f&quot;Relationships: {len(model.relationships)}&quot;)<br>print(f&quot;Measures:      {len(model.measures)}&quot;)<br>print(f&quot;RLS Rules:     {len(model.rls_rules)}&quot;)</pre><p>For the real Sales_Returns_Sample.pbix file from Microsoft, this gives:</p><pre>Tables:        15<br>Relationships: 9<br>Measures:      58<br>RLS Rules:     0</pre><p>Each table becomes an Entity with typed properties, each relationship becomes a Relationship with cardinality (one-to-many, many-to-many) and cross-filter direction.</p><h3>DAX â†’ Business Rules</h3><p>The most interesting part is automatic DAX formula parsing. DAX is the Power BI query language where business logic is encoded:</p><pre>Total Revenue = SUMX(Sales, Sales[Quantity] * Sales[Unit Price])</pre><pre>Revenue YoY% =<br>DIVIDE(<br>    [Total Revenue] - CALCULATE([Total Revenue], SAMEPERIODLASTYEAR(&#39;Date&#39;[Date])),<br>    CALCULATE([Total Revenue], SAMEPERIODLASTYEAR(&#39;Date&#39;[Date]))<br>)</pre><p>Our DAX <strong>subset</strong> parser uses regex to recognise four pattern families (not the full DAX grammar): — <strong>Conditional logic</strong>: IF, SWITCH, CALCULATE with filters â€” transformed into business rules — <strong>Simple thresholds</strong>: field &gt; value comparisons â€” transformed into threshold rules - <strong>Aggregations</strong>: SUM, AVERAGE, COUNT â€” dependency tracking (not parsed into rules) - <strong>Dependencies</strong>: Which tables and columns each measure references</p><p><strong>Not supported</strong>: nested CALCULATE (inner level), VAR/RETURN blocks, row-context iterators (SUMX, FILTER), table constructors, time-intelligence functions (SAMEPERIODLASTYEAR, DATEADD), SELECTEDVALUE, HASONEVALUE.</p><p>This is critically important: an AI agent that knows the Revenue formula won’t sum the Price column directly â€” it will use the correct metric.</p><h3>OWL Ontology Generation</h3><p>The extracted model is transformed into a formal OWL ontology:</p><pre>from powerbi_ontology import OntologyGenerator<br>from powerbi_ontology.export.owl import OWLExporter</pre><pre># Generate ontology<br>ontology = OntologyGenerator(model).generate()</pre><pre># Export to OWL<br>exporter = OWLExporter(<br>    ontology,<br>    default_roles=[&quot;Admin&quot;, &quot;Analyst&quot;, &quot;Viewer&quot;]<br>)<br>exporter.save(&quot;sales_ontology.owl&quot;)</pre><pre>summary = exporter.get_export_summary()<br>print(f&quot;OWL Classes:        {summary[&#39;classes&#39;]}&quot;)<br>print(f&quot;Properties:         {summary[&#39;datatype_properties&#39;]}&quot;)<br>print(f&quot;Action Rules:       {summary[&#39;action_rules&#39;]}&quot;)</pre><p>The OWL file contains: — <strong>owl:Class</strong> for each entity (Customer, Sales, Product) — <strong>owl:DatatypeProperty</strong> for each property with XSD types — <strong>owl:ObjectProperty</strong> for relationships between entities — <strong>Action Rules</strong> â€” who (role) can do what (create/read/update/delete) with which entity — <strong>Business rules</strong> from DAX measures (as OWL annotations) — <strong>Constraints</strong> (required, unique, range, enum)</p><p>For Sales_Returns_Sample, this generates <strong>1,734 RDF triples</strong> â€” a complete formal description of the dashboard’s semantics.</p><h3>What’s New</h3><h3>1. Automatic Extraction, Not Manual Creation</h3><p>Existing tools (Protege, WebVOWL, TopBraid) assume you <strong>manually</strong> create ontologies. That’s expensive ($50K+) and slow (months). We extract 70% of the ontology <strong>automatically</strong> from what already exists in Power BI.</p><h3>2. DAX â†’ Business Rules (Not Just Schema)</h3><p>Tools like pbi-tools or Tabular Editor can export Power BI schemas. But a schema is not an ontology. We go further: we parse DAX formulas and transform them into semantic business rules. The AI agent gets not just &quot;a Sales table with a Revenue column,&quot; but &quot;Revenue is calculated as SUMX(Sales, Quantity * UnitPrice) and is accessible to Admin and Analyst roles for reading.&quot;</p><h3>3. A Bridge Between BI and AI</h3><p>Power BI is a tool for humans. OWL is a standard for machines. Our extractor is a bridge: humans work in familiar Power BI, while AI agents receive a formal ontology with access rules.</p><h3>4. Schema Drift Detection</h3><p>Remember the schema drift scenario above? We solve this problem with Schema Drift Detection:</p><pre>from powerbi_ontology import SchemaMapper</pre><pre>mapper = SchemaMapper()<br># Bind ontology to actual DB schema<br>drift = mapper.check_drift(ontology, actual_schema)</pre><pre>for issue in drift:<br>    print(f&quot;[{issue.severity}] {issue.entity}: {issue.description}&quot;)<br>    # [CRITICAL] Sales: Column &#39;Warehouse_Location&#39; not found (renamed to &#39;FacilityID&#39;?)</pre><p>The system normalizes types (varchar(255) â†’ text, int â†’ integer), detects renames through similarity matching (&gt;70% match), and classifies issues by severity: CRITICAL blocks the agent, WARNING logs, INFO informs.</p><h3>5. Semantic Debt Analysis</h3><p>When an organization has 50+ dashboards, “semantic debt” inevitably emerges â€” the same metric defined differently across dashboards:</p><pre>from powerbi_ontology import analyze_ontologies</pre><pre>report = analyze_ontologies([<br>    &quot;sales_ontology.json&quot;,<br>    &quot;finance_ontology.json&quot;,<br>    &quot;marketing_ontology.json&quot;<br>])</pre><pre>for conflict in report.conflicts:<br>    print(f&quot;Conflict: {conflict.entity}.{conflict.property}&quot;)<br>    print(f&quot;  Dashboard A: {conflict.definition_a}&quot;)<br>    print(f&quot;  Dashboard B: {conflict.definition_b}&quot;)<br>    # Conflict: Revenue<br>    #   Sales dashboard: SUMX(Sales, Quantity * UnitPrice)<br>    #   Finance dashboard: SUM(Invoices[Amount]) - SUM(Refunds[Amount])</pre><p>This lets you detect conflicts <strong>before</strong> the AI agent starts using contradictory data.</p><h3>How to Use It</h3><h3>Option 1: Python API (for developers)</h3><pre>pip install powerbi-ontology-extractor</pre><pre>from powerbi_ontology import PowerBIExtractor, OntologyGenerator<br>from powerbi_ontology.export.owl import OWLExporter</pre><pre># 1. Extract model from .pbix<br>extractor = PowerBIExtractor(&quot;my_dashboard.pbix&quot;)<br>model = extractor.extract()</pre><pre># 2. Generate ontology<br>ontology = OntologyGenerator(model).generate()</pre><pre># 3. Export<br># To OWL (for OntoGuard / triple stores)<br>OWLExporter(ontology).save(&quot;ontology.owl&quot;)</pre><pre># To JSON (for API / AI agents)<br>import json<br>with open(&quot;ontology.json&quot;, &quot;w&quot;) as f:<br>    json.dump(ontology.to_dict(), f, indent=2)</pre><h3>Option 2: CLI (for DevOps and automation)</h3><pre># Extract a single file<br>pbix2owl extract -i dashboard.pbix -o ontology.owl</pre><pre># Batch process a directory (8 parallel workers)<br>pbix2owl batch -i ./dashboards/ -o ./ontologies/ -w 8 --recursive</pre><pre># Semantic debt analysis<br>pbix2owl analyze -i ./ontologies/ -o report.md</pre><pre># Version comparison<br>pbix2owl diff -s v1.json -t v2.json -o changelog.md</pre><h3>Option 3: Streamlit UI (for business analysts)</h3><pre>pip install powerbi-ontology-extractor streamlit<br>streamlit run ontology_editor.py --server.port 8503</pre><p>A visual editor with 8 tabs:</p><p>TabFunctionLoad/CreateUpload .pbix or create from scratchEntitiesEdit entities and propertiesRelationshipsManage relationshipsPermissionsRBAC matrix (role x entity x action)Business RulesBusiness rules with classificationOWL PreviewPreview and export OWLDiff &amp; MergeCompare and merge versionsAI ChatAsk questions about the ontology in natural language</p><p>An analyst can upload a .pbix file, edit the automatically extracted ontology (those 30%), ask an AI question (“Which entities are related to Customer?”), and export the result â€” all without writing a single line of code.</p><h3>Option 4: MCP Server (for Claude Code)</h3><p>PowerBI Ontology Extractor works as an MCP server that you can connect to Claude Code:</p><pre>// ~/.claude.json<br>{<br>  &quot;mcpServers&quot;: {<br>    &quot;powerbi-ontology&quot;: {<br>      &quot;command&quot;: &quot;python&quot;,<br>      &quot;args&quot;: [&quot;-m&quot;, &quot;powerbi_ontology.mcp_server&quot;]<br>    }<br>  }<br>}</pre><p>After this, 8 tools become available in Claude Code: pbix_extract, ontology_generate, export_owl, export_json, analyze_debt, ontology_diff, ontology_merge, ontology_chat_ask.</p><p>You can simply tell Claude: <em>“Extract the ontology from sales.pbix and export it to OWL”</em> â€” and it will do it.</p><h3>Ecosystem Integration</h3><p>PowerBI Ontology Extractor is the first element in a chain for safe AI agent interaction with data:</p><pre>â”Œâ”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”     â”Œâ”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”     â”Œâ”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”<br>â”‚  Ontology Extractor  â”‚â”€â”€â”€â”€â–¶â”‚      OntoGuard       â”‚â”€â”€â”€â”€â–¶â”‚ Universal Agent Connector  â”‚<br>â”‚  (this project)      â”‚     â”‚  Semantic Firewall   â”‚     â”‚    MCP Infrastructure     â”‚<br>â”‚  Extraction          â”‚     â”‚  Validation          â”‚     â”‚    Agent Connection        â”‚<br>â””â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”˜     â””â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”˜     â””â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”€â”˜</pre><ul><li><a href="https://github.com/vpakspace/ontoguard-ai"><strong>OntoGuard</strong></a> â€” a semantic firewall. It takes the OWL ontology extracted by our tool and checks every AI agent action: “Can the Analyst role delete a record in the Patients table?” If the OWL rule prohibits it â€” the action is blocked before it reaches the database.</li><li><a href="https://github.com/vpakspace/universal-agent-connector"><strong>Universal Agent Connector</strong></a> â€” MCP infrastructure for connecting AI agents to databases. Uses OntoGuard as middleware: NL query â†’ SQL â†’ OntoGuard check â†’ execution.</li></ul><p>Together they implement a <strong>semantic contract</strong>: the AI agent gets not just data access, but access with understanding of what the data means, how it’s connected, and what can be done with it.</p><h3>Evaluation</h3><p>We evaluate extraction accuracy on <strong>real Microsoft .pbix samples</strong>, comparing extracted data against manually verified ground truth.</p><h3>Entity Extraction Accuracy</h3><p>DatasetPrecisionRecallF1EntitiesSales_Returns_Sample87%87%87%15Adventure_Works_DW_2020100%100%100%11</p><h3>Relationship Extraction</h3><p>DatasetPrecisionRecallF1RelationshipsSales_Returns_Sample100%100%100%9Adventure_Works_DW_2020100%100%100%13</p><h3>DAX Subset Parser â€” Pattern Coverage</h3><p><strong>8/8 patterns</strong> handled correctly (100%). The regex-based parser handles: CALCULATE, IF, SWITCH, and simple thresholds. Patterns without conditions (plain SUM, SUMX iterators) are correctly ignored.</p><h3>OWL Export</h3><p>DatasetOWL TriplesBusiness RulesExtraction TimeSales_Returns_Sample1,65632117 msAdventure_Works_DW_20201,083073 ms</p><p><strong>Limitations</strong>: The regex-based DAX parser does not support nested CALCULATE (inner level), row context iterators (SUMX/FILTER), table constructors, or advanced DAX patterns like SELECTEDVALUE/HASONEVALUE. The evaluation script and ground truth data are available in evaluation/run_evaluation.py.</p><h3>Testing and Security</h3><h3>370 Tests, 81% Coverage</h3><pre>$ pytest<br>========================= 370 passed in 4.56s =========================</pre><p>We test on real .pbix files from Microsoft (Sales_Returns_Sample, Adventure_Works_DW_2020), not synthetic data.</p><h3>Security Hardening (v0.1.1)</h3><p>In version 0.1.1, 14 security issues discovered during code review were fixed:</p><p>SeverityCountExamplesCRITICAL3Path traversal in file writing, XSS in chat, unsafe YAMLHIGH2API key validation, upload size limit (50 MB)MEDIUM4Audit logging, DAX parser hardening, chat history limitLOW5Type hints, deterministic hashing, rate limiting</p><p>All mutating operations (file uploads, exports, edits) are logged in data/audit.log.</p><h3>Authorship and Roadmap</h3><p>The framework has been significantly rearchitected from the original concept by Kumar, but I deliberately kept entity names and semantic blocks “as the author had them” â€” so that readers can more easily cross-reference with the original sources without losing the thread of context.</p><p>The project is actively evolving. Near-term steps:</p><ul><li><strong>Richer constraint extraction</strong> â€” in the spirit of SHACL validations, to express not just types but business invariants (ranges, field dependencies)</li><li><strong>Change impact analysis</strong> â€” diffs between model versions with assessment of which agents and pipelines are affected</li><li><strong>Ontology packaging standardization</strong> â€” for agent runtime systems (MCP-compatible format, artifact versioning)</li><li><strong>v0.2.0</strong>: Power BI Semantic Link support (direct connection to Fabric)</li><li><strong>v0.3.0</strong>: Automatic Semantic Contract generation with property-level permissions</li><li><strong>v1.0.0</strong>: Production-ready with CI/CD, Docker image, Helm chart</li></ul><h3>Try It Now</h3><pre># Install<br>pip install powerbi-ontology-extractor</pre><pre># CLI<br>pbix2owl extract -i your_dashboard.pbix -o ontology.owl</pre><pre># Or the visual editor<br>pip install streamlit<br>streamlit run ontology_editor.py</pre><ul><li><strong>PyPI</strong>: <a href="https://pypi.org/project/powerbi-ontology-extractor/">powerbi-ontology-extractor</a></li><li><strong>GitHub</strong>: <a href="https://github.com/vpakspace/powerbi-ontology-extractor">vpakspace/powerbi-ontology-extractor</a></li><li><strong>Release</strong>: <a href="https://github.com/vpakspace/powerbi-ontology-extractor/releases/tag/v0.1.1">v0.1.1</a></li><li><strong>License</strong>: MIT</li></ul><p><em>If this project is useful â€” give it a star on GitHub. If you have questions or ideas â€” open an Issue, we respond.</em></p><h3>References</h3><h3>Pankaj Kumar’s Article Series (conceptual foundation)</h3><ol><li><a href="https://medium.com/@cloudpankaj/microsoft-vs-palantir-two-paths-to-enterprise-ontology-and-why-microsofts-bet-on-semantic-6e72265dce21">Microsoft vs Palantir: Two Paths to Enterprise Ontology</a></li><li><a href="https://medium.com/@cloudpankaj/the-power-bi-ontology-paradox-how-20-million-dashboards-became-microsofts-secret-weapon-for-5585e7d18c01">The Power BI Ontology Paradox</a></li><li><a href="https://medium.com/@cloudpankaj/from-power-bi-dashboard-to-ai-agent-in-30-minutes-i-built-the-tool-that-unlocks-20-million-hidden-500e59bd91df">From Power BI Dashboard to AI Agent in 30 Minutes</a></li><li><a href="https://medium.com/@cloudpankaj/universal-agent-connector-mcp-ontology-production-ready-ai-infrastructure-0b4e35f22942">Universal Agent Connector: MCP + Ontology</a></li><li><a href="https://medium.com/@cloudpankaj/ontoguard-i-built-an-ontology-firewall-for-ai-agents-in-48-hours-using-cursor-ai-be4208c405e7">OntoGuard: Ontology Firewall for AI Agents</a></li></ol><h3>Projects</h3><ul><li><a href="https://github.com/vpakspace/powerbi-ontology-extractor">PowerBI Ontology Extractor</a> â€” this project</li><li><a href="https://github.com/vpakspace/ontoguard-ai">OntoGuard AI</a> â€” semantic firewall for AI agents</li><li><a href="https://github.com/vpakspace/universal-agent-connector">Universal Agent Connector</a> â€” MCP infrastructure for connecting agents to databases</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a49c1635198" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How We Extract Hidden Ontologies from 20 Million Power BI Dashboards and Make Them Accessible to…]]></title>
            <link>https://medium.com/@vpakspace/how-we-extract-hidden-ontologies-from-20-million-power-bi-dashboards-and-make-them-accessible-to-6f2128789c34?source=rss-204422e57dbe------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f2128789c34</guid>
            <category><![CDATA[power-bi]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[ontology]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Vladislav Pak]]></dc:creator>
            <pubDate>Fri, 06 Feb 2026 06:07:08 GMT</pubDate>
            <atom:updated>2026-02-06T06:07:08.324Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PMQQd2R0AYAum-Tv74YKsw.png" /></figure><h3>How We Extract Hidden Ontologies from 20 Million Power BI Dashboards</h3><h3>and Make Them Accessible to AI Agents</h3><blockquote><strong><em>TL;DR</em></strong><em>: We built an open-source tool that automatically extracts semantic models from Power BI .pbix files and transforms them into formal OWL ontologies. This enables AI agents to work safely with enterprise data by understanding business rules, relationships, and access constraints. </em><em>pip install powerbi-ontology-extractor — and in 10 minutes you have an ontology.</em></blockquote><h3>The Problem: $4.6M Lost Due to a Renamed Column</h3><p>In 2024, a major logistics company lost $4.6M overnight. The cause was absurdly simple: a database administrator renamed a column from Warehouse_Location to FacilityID. The AI agent managing routing didn&#39;t know about the rename and started shipping cargo to random addresses.</p><p>This case isn’t an exception — it’s a symptom of a systemic problem. There are <strong>over 20 million</strong> Power BI dashboards worldwide (Microsoft, 2024). Each one contains a semantic model — tables, relationships, measures, security rules. In essence, every Power BI dashboard is an <strong>informal ontology</strong>, locked inside a proprietary .pbix file.</p><p>AI agents can’t read .pbix files. They don’t know that Revenue in the sales department and Revenue in finance are different metrics with different formulas. They don&#39;t understand that an analyst has read-only access, while only an admin can delete records. Without this knowledge, an AI agent is a blind robot with access to a production database.</p><h3>The Scale of the Problem</h3><p>Metric Value Power BI dashboards worldwide 20M+ Average cost of manual ontology creation $50K–$200K Percentage of ontology that can be extracted automatically ~70% Percentage requiring manual work by a business analyst ~30%</p><p>Manual ontology creation takes months of expensive specialist work. Yet 70% of the effort is mechanical extraction of what’s already described in Power BI: tables become entities, columns become properties, foreign keys become relationships, DAX formulas become business rules.</p><h3>The Solution: 30 Minutes Instead of 3 Months</h3><p>We built <strong>PowerBI Ontology Extractor</strong> — an open-source Python tool that automates those 70% and provides a visual editor for the remaining 30%.</p><p>Solution architecture:</p><pre>Power BI (.pbix)  →  Ontology Extractor  →  OntoGuard  →  AI Agent<br>      10 min              10 min             5 min          5 min</pre><p>What happens at each stage:</p><ol><li><strong>Extraction</strong> (10 min): The binary .pbix file is unpacked, DataModel is parsed (via PBIXRay), extracting tables, columns, data types, relationships, DAX measures, and RLS rules</li><li><strong>Generation</strong> (10 min): Raw data is transformed into a formal ontology — classes, properties, relationships with cardinality, business rules from DAX, constraints</li><li><strong>Validation</strong> (5 min): The ontology passes through OntoGuard (a semantic firewall) that enforces role-based access control via OWL rules</li><li><strong>Deployment</strong> (5 min): The AI agent receives the ontology as a contract — and now knows what it can and cannot do</li></ol><h3>Installation</h3><pre>pip install powerbi-ontology-extractor</pre><p>That’s it. No Docker, no server configuration, no paid APIs (except optional OpenAI for chat). One command — and you have a CLI, Python API, and Streamlit UI.</p><h3>Conceptual Foundation</h3><p>PowerBI Ontology Extractor translates a Power BI semantic model into an explicit ontological artifact — something that can be queried, validated, versioned, and used as a “semantic contract” for AI agents. The starting idea here comes from a series of articles by <a href="#references">Pankaj Kumar</a>: at enterprise scale, BI models are effectively informal ontologies that can be formalized and turned into an infrastructure asset.</p><p>The practical value lies in reliability. When meaning is externalized, it becomes possible to perform “semantic checks” rather than just “schema checks” — for example, business logic validation and safer model changes during field and name drift. This aligns well with Kumar’s ideas on semantic validation (<a href="https://github.com/vpakspace/ontoguard-ai">OntoGuard</a>) and production infrastructure for agents at the intersection of MCP and ontologies (<a href="https://github.com/vpakspace/universal-agent-connector">Universal Agent Connector</a>).</p><h3>Under the Hood</h3><h3>Parsing .pbix Files</h3><p>Power BI stores data in a binary format. We use the <a href="https://github.com/pankajkumar/pbixray">PBIXRay</a> library to read the DataModel, then extract:</p><pre>from powerbi_ontology import PowerBIExtractor</pre><pre>extractor = PowerBIExtractor(&quot;sales_dashboard.pbix&quot;)<br>model = extractor.extract()</pre><pre>print(f&quot;Tables:        {len(model.entities)}&quot;)<br>print(f&quot;Relationships: {len(model.relationships)}&quot;)<br>print(f&quot;Measures:      {len(model.measures)}&quot;)<br>print(f&quot;RLS Rules:     {len(model.rls_rules)}&quot;)</pre><p>For the real Sales_Returns_Sample.pbix file from Microsoft, this gives:</p><pre>Tables:        15<br>Relationships: 9<br>Measures:      58<br>RLS Rules:     0</pre><p>Each table becomes an Entity with typed properties, each relationship becomes a Relationship with cardinality (one-to-many, many-to-many) and cross-filter direction.</p><h3>DAX → Business Rules</h3><p>The most interesting part is automatic DAX formula parsing. DAX is the Power BI query language where business logic is encoded:</p><pre>Total Revenue = SUMX(Sales, Sales[Quantity] * Sales[Unit Price])</pre><pre>Revenue YoY% =<br>DIVIDE(<br>    [Total Revenue] - CALCULATE([Total Revenue], SAMEPERIODLASTYEAR(&#39;Date&#39;[Date])),<br>    CALCULATE([Total Revenue], SAMEPERIODLASTYEAR(&#39;Date&#39;[Date]))<br>)</pre><p>Our DAX parser recognizes:</p><ul><li><strong>Aggregations</strong>: SUM, AVERAGE, COUNT, SUMX — transformed into calculation rules</li><li><strong>Conditional logic</strong>: IF, SWITCH, CALCULATE with filters — transformed into business rules</li><li><strong>Time Intelligence</strong>: SAMEPERIODLASTYEAR, DATEADD — marked as temporal metrics</li><li><strong>Dependencies</strong>: Which tables and columns each measure uses</li></ul><p>This is critically important: an AI agent that knows the Revenue formula won’t sum the Price column directly — it will use the correct metric.</p><h3>OWL Ontology Generation</h3><p>The extracted model is transformed into a formal OWL ontology:</p><pre>from powerbi_ontology import OntologyGenerator<br>from powerbi_ontology.export.owl import OWLExporter</pre><pre># Generate ontology<br>ontology = OntologyGenerator(model).generate()</pre><pre># Export to OWL<br>exporter = OWLExporter(<br>    ontology,<br>    default_roles=[&quot;Admin&quot;, &quot;Analyst&quot;, &quot;Viewer&quot;]<br>)<br>exporter.save(&quot;sales_ontology.owl&quot;)</pre><pre>summary = exporter.get_export_summary()<br>print(f&quot;OWL Classes:        {summary[&#39;classes&#39;]}&quot;)<br>print(f&quot;Properties:         {summary[&#39;datatype_properties&#39;]}&quot;)<br>print(f&quot;Action Rules:       {summary[&#39;action_rules&#39;]}&quot;)</pre><p>The OWL file contains:</p><ul><li><strong>owl:Class</strong> for each entity (Customer, Sales, Product)</li><li><strong>owl:DatatypeProperty</strong> for each property with XSD types</li><li><strong>owl:ObjectProperty</strong> for relationships between entities</li><li><strong>Action Rules</strong> — who (role) can do what (create/read/update/delete) with which entity</li><li><strong>SWRL rules</strong> from DAX measures</li><li><strong>Constraints</strong> (required, unique, range, enum)</li></ul><p>For Sales_Returns_Sample, this generates <strong>1,734 RDF triples</strong> — a complete formal description of the dashboard’s semantics.</p><h3>What’s New</h3><h3>1. Automatic Extraction, Not Manual Creation</h3><p>Existing tools (Protege, WebVOWL, TopBraid) assume you <strong>manually</strong> create ontologies. That’s expensive ($50K+) and slow (months). We extract 70% of the ontology <strong>automatically</strong> from what already exists in Power BI.</p><h3>2. DAX → Business Rules (Not Just Schema)</h3><p>Tools like pbi-tools or Tabular Editor can export Power BI schemas. But a schema is not an ontology. We go further: we parse DAX formulas and transform them into semantic business rules. The AI agent gets not just &quot;a Sales table with a Revenue column,&quot; but &quot;Revenue is calculated as SUMX(Sales, Quantity * UnitPrice) and is accessible to Admin and Analyst roles for reading.&quot;</p><h3>3. A Bridge Between BI and AI</h3><p>Power BI is a tool for humans. OWL is a standard for machines. Our extractor is a bridge: humans work in familiar Power BI, while AI agents receive a formal ontology with access rules.</p><h3>4. Schema Drift Detection</h3><p>Remember the $4.6M story? We solve this problem with Schema Drift Detection:</p><pre>from powerbi_ontology import SchemaMapper</pre><pre>mapper = SchemaMapper()<br># Bind ontology to actual DB schema<br>drift = mapper.check_drift(ontology, actual_schema)</pre><pre>for issue in drift:<br>    print(f&quot;[{issue.severity}] {issue.entity}: {issue.description}&quot;)<br>    # [CRITICAL] Sales: Column &#39;Warehouse_Location&#39; not found (renamed to &#39;FacilityID&#39;?)</pre><p>The system normalizes types (varchar(255) → text, int → integer), detects renames through similarity matching (&gt;70% match), and classifies issues by severity: CRITICAL blocks the agent, WARNING logs, INFO informs.</p><h3>5. Semantic Debt Analysis</h3><p>When an organization has 50+ dashboards, “semantic debt” inevitably emerges — the same metric defined differently across dashboards:</p><pre>from powerbi_ontology import analyze_ontologies</pre><pre>report = analyze_ontologies([<br>    &quot;sales_ontology.json&quot;,<br>    &quot;finance_ontology.json&quot;,<br>    &quot;marketing_ontology.json&quot;<br>])</pre><pre>for conflict in report.conflicts:<br>    print(f&quot;Conflict: {conflict.entity}.{conflict.property}&quot;)<br>    print(f&quot;  Dashboard A: {conflict.definition_a}&quot;)<br>    print(f&quot;  Dashboard B: {conflict.definition_b}&quot;)<br>    # Conflict: Revenue<br>    #   Sales dashboard: SUMX(Sales, Quantity * UnitPrice)<br>    #   Finance dashboard: SUM(Invoices[Amount]) - SUM(Refunds[Amount])</pre><p>This lets you detect conflicts <strong>before</strong> the AI agent starts using contradictory data.</p><h3>How to Use It</h3><h3>Option 1: Python API (for developers)</h3><pre>pip install powerbi-ontology-extractor</pre><pre>from powerbi_ontology import PowerBIExtractor, OntologyGenerator<br>from powerbi_ontology.export.owl import OWLExporter</pre><pre># 1. Extract model from .pbix<br>extractor = PowerBIExtractor(&quot;my_dashboard.pbix&quot;)<br>model = extractor.extract()</pre><pre># 2. Generate ontology<br>ontology = OntologyGenerator(model).generate()</pre><pre># 3. Export<br># To OWL (for OntoGuard / triple stores)<br>OWLExporter(ontology).save(&quot;ontology.owl&quot;)</pre><pre># To JSON (for API / AI agents)<br>import json<br>with open(&quot;ontology.json&quot;, &quot;w&quot;) as f:<br>    json.dump(ontology.to_dict(), f, indent=2)</pre><h3>Option 2: CLI (for DevOps and automation)</h3><pre># Extract a single file<br>pbix2owl extract -i dashboard.pbix -o ontology.owl</pre><pre># Batch process a directory (8 parallel workers)<br>pbix2owl batch -i ./dashboards/ -o ./ontologies/ -w 8 --recursive</pre><pre># Semantic debt analysis<br>pbix2owl analyze -i ./ontologies/ -o report.md</pre><pre># Version comparison<br>pbix2owl diff -s v1.json -t v2.json -o changelog.md</pre><h3>Option 3: Streamlit UI (for business analysts)</h3><pre>pip install powerbi-ontology-extractor streamlit<br>streamlit run ontology_editor.py --server.port 8503</pre><p>A visual editor with 8 tabs:</p><p>Tab Function Load/Create Upload .pbix or create from scratch Entities Edit entities and properties Relationships Manage relationships Permissions RBAC matrix (role x entity x action) Business Rules Business rules with classification OWL Preview Preview and export OWL Diff &amp; Merge Compare and merge versions AI Chat Ask questions about the ontology in natural language</p><p>An analyst can upload a .pbix file, edit the automatically extracted ontology (those 30%), ask an AI question (“Which entities are related to Customer?”), and export the result — all without writing a single line of code.</p><h3>Option 4: MCP Server (for Claude Code)</h3><p>PowerBI Ontology Extractor works as an MCP server that you can connect to Claude Code:</p><pre>// ~/.claude.json<br>{<br>  &quot;mcpServers&quot;: {<br>    &quot;powerbi-ontology&quot;: {<br>      &quot;command&quot;: &quot;python&quot;,<br>      &quot;args&quot;: [&quot;-m&quot;, &quot;powerbi_ontology.mcp_server&quot;]<br>    }<br>  }<br>}</pre><p>After this, 8 tools become available in Claude Code: pbix_extract, ontology_generate, export_owl, export_json, analyze_debt, ontology_diff, ontology_merge, ontology_chat_ask.</p><p>You can simply tell Claude: <em>“Extract the ontology from sales.pbix and export it to OWL”</em> — and it will do it.</p><h3>Ecosystem Integration</h3><p>PowerBI Ontology Extractor is the first element in a chain for safe AI agent interaction with data:</p><pre>┌──────────────────────┐     ┌──────────────────────┐     ┌────────────────────────────┐<br>│  Ontology Extractor  │────▶│      OntoGuard       │────▶│ Universal Agent Connector  │<br>│  (this project)      │     │  Semantic Firewall   │     │    MCP Infrastructure     │<br>│  Extraction          │     │  Validation          │     │    Agent Connection        │<br>└──────────────────────┘     └──────────────────────┘     └────────────────────────────┘</pre><ul><li><a href="https://github.com/vpakspace/ontoguard-ai"><strong>OntoGuard</strong></a> — a semantic firewall. It takes the OWL ontology extracted by our tool and checks every AI agent action: “Can the Analyst role delete a record in the Patients table?” If the OWL rule prohibits it — the action is blocked before it reaches the database.</li><li><a href="https://github.com/vpakspace/universal-agent-connector"><strong>Universal Agent Connector</strong></a> — MCP infrastructure for connecting AI agents to databases. Uses OntoGuard as middleware: NL query → SQL → OntoGuard check → execution.</li></ul><p>Together they implement a <strong>semantic contract</strong>: the AI agent gets not just data access, but access with understanding of what the data means, how it’s connected, and what can be done with it.</p><h3>Testing and Security</h3><h3>370 Tests, 81% Coverage</h3><pre>$ pytest<br>========================= 370 passed in 4.56s =========================</pre><p>We test on real .pbix files from Microsoft (Sales_Returns_Sample, Adventure_Works_DW_2020), not synthetic data.</p><h3>Security Hardening (v0.1.1)</h3><p>In version 0.1.1, 14 security issues discovered during code review were fixed:</p><p>Severity Count Examples CRITICAL 3 Path traversal in file writing, XSS in chat, unsafe YAML HIGH 2 API key validation, upload size limit (50 MB) MEDIUM 4 Audit logging, DAX parser hardening, chat history limit LOW 5 Type hints, deterministic hashing, rate limiting</p><p>All mutating operations (file uploads, exports, edits) are logged in data/audit.log.</p><h3>Authorship and Roadmap</h3><p>The framework has been significantly rearchitected from the original concept by Kumar, but I deliberately kept entity names and semantic blocks “as the author had them” — so that readers can more easily cross-reference with the original sources without losing the thread of context.</p><p>The project is actively evolving. Near-term steps:</p><ul><li><strong>Richer constraint extraction</strong> — in the spirit of SHACL validations, to express not just types but business invariants (ranges, field dependencies)</li><li><strong>Change impact analysis</strong> — diffs between model versions with assessment of which agents and pipelines are affected</li><li><strong>Ontology packaging standardization</strong> — for agent runtime systems (MCP-compatible format, artifact versioning)</li><li><strong>v0.2.0</strong>: Power BI Semantic Link support (direct connection to Fabric)</li><li><strong>v0.3.0</strong>: Automatic Semantic Contract generation with property-level permissions</li><li><strong>v1.0.0</strong>: Production-ready with CI/CD, Docker image, Helm chart</li></ul><h3>Try It Now</h3><pre># Install<br>pip install powerbi-ontology-extractor</pre><pre># CLI<br>pbix2owl extract -i your_dashboard.pbix -o ontology.owl</pre><pre># Or the visual editor<br>pip install streamlit<br>streamlit run ontology_editor.py</pre><ul><li><strong>PyPI</strong>: <a href="https://pypi.org/project/powerbi-ontology-extractor/">powerbi-ontology-extractor</a></li><li><strong>GitHub</strong>: <a href="https://github.com/vpakspace/powerbi-ontology-extractor">vpakspace/powerbi-ontology-extractor</a></li><li><strong>Release</strong>: <a href="https://github.com/vpakspace/powerbi-ontology-extractor/releases/tag/v0.1.1">v0.1.1</a></li><li><strong>License</strong>: MIT</li></ul><p><em>If this project is useful — give it a star on GitHub. If you have questions or ideas — open an Issue, we respond.</em></p><h3>References</h3><h3>Pankaj Kumar’s Article Series (conceptual foundation)</h3><ol><li><a href="https://medium.com/@cloudpankaj/microsoft-vs-palantir-two-paths-to-enterprise-ontology-and-why-microsofts-bet-on-semantic-6e72265dce21">Microsoft vs Palantir: Two Paths to Enterprise Ontology</a></li><li><a href="https://medium.com/@cloudpankaj/the-power-bi-ontology-paradox-how-20-million-dashboards-became-microsofts-secret-weapon-for-5585e7d18c01">The Power BI Ontology Paradox</a></li><li><a href="https://medium.com/@cloudpankaj/from-power-bi-dashboard-to-ai-agent-in-30-minutes-i-built-the-tool-that-unlocks-20-million-hidden-500e59bd91df">From Power BI Dashboard to AI Agent in 30 Minutes</a></li><li><a href="https://medium.com/@cloudpankaj/universal-agent-connector-mcp-ontology-production-ready-ai-infrastructure-0b4e35f22942">Universal Agent Connector: MCP + Ontology</a></li><li><a href="https://medium.com/@cloudpankaj/ontoguard-i-built-an-ontology-firewall-for-ai-agents-in-48-hours-using-cursor-ai-be4208c405e7">OntoGuard: Ontology Firewall for AI Agents</a></li></ol><h3>Projects</h3><ul><li><a href="https://github.com/vpakspace/powerbi-ontology-extractor">PowerBI Ontology Extractor</a> — this project</li><li><a href="https://github.com/vpakspace/ontoguard-ai">OntoGuard AI</a> — semantic firewall for AI agents</li><li><a href="https://github.com/vpakspace/universal-agent-connector">Universal Agent Connector</a> — MCP infrastructure for connecting agents to databases</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6f2128789c34" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>