<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Weidong Yang on Medium]]></title>
        <description><![CDATA[Stories by Weidong Yang on Medium]]></description>
        <link>https://medium.com/@weidong_yang?source=rss-ab264147a186------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*DuGTccZLvzUtUh4zAFr9HA.jpeg</url>
            <title>Stories by Weidong Yang on Medium</title>
            <link>https://medium.com/@weidong_yang?source=rss-ab264147a186------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 08:28:14 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@weidong_yang/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[From Spanner Graph to Operational Insight: Visual Telco Analysis with Kineviz GraphXR]]></title>
            <link>https://medium.com/kineviz/from-spanner-graph-to-operational-insight-visual-telco-analysis-with-kineviz-graphxr-8b281dceeb6b?source=rss-ab264147a186------2</link>
            <guid isPermaLink="false">https://medium.com/p/8b281dceeb6b</guid>
            <category><![CDATA[graph]]></category>
            <category><![CDATA[telco]]></category>
            <category><![CDATA[data-visualization]]></category>
            <dc:creator><![CDATA[Weidong Yang]]></dc:creator>
            <pubDate>Thu, 16 Apr 2026 05:22:36 GMT</pubDate>
            <atom:updated>2026-04-16T05:23:22.782Z</atom:updated>
            <content:encoded><![CDATA[<p>Communication Service Providers (CSPs) are building autonomous networks while tackling the exploding complexity of modern telecommunications. This shift requires more than just automation alone: it demands a responsive human oversight layer paired with a dynamic visualization system that can detect emerging incident patterns, retrieve relevant data, generate intuitive views, and feed insights back into the network.</p><p>In this context, we demonstrate how Kineviz GraphXR, an agent-assisted graph visual analytics platform, working with Spanner Graph, enables field operators to monitor incidents and identify root causes more effectively. Within GraphXR, an AI agent collaborates with the analyst by interpreting intent, generating queries, and constructing visualizations, allowing operators to focus on reasoning rather than mechanics.</p><p>Effective analysis depends on visualizing relationships within the data. GraphXR provides a no-code, interactive environment where both the analyst and the AI agent operate on the same visual context, iteratively refining the investigation.</p><p>Spanner Graph offers an efficient high-performance storage solution for the diverse, multi-layered telecommunications network data. This data spans from low-level individual event logs to time-windowed KPIs and high-level aggregated incidents, supporting both OLTP (Online Transaction Processing) and analytical use cases. Crucially, the platform enables explicit modeling and querying of relationships using GQL, which enables immediate visual analysis in GraphXR.</p><p>In practice, field operators often lack GQL expertise, and even correct queries return tabular results that require interpretation. Recognizing patterns — such as a single device causing repeated failures or groups of devices failing together — can be difficult.</p><p>An analyst–agent co-reasoning approach addresses this gap. The agent surfaces patterns and generates visual structures, while the analyst validates and guides the process. Insight emerges through this iterative process, enabling faster and more intuitive root cause analysis.</p><h3>The data behind the alerts</h3><p>We use data from the <a href="https://github.com/GoogleCloudPlatform/telco-autonomous-networks-data-demo">Google Telco Autonomous Network Data Demo</a>. It is organized into three layers:</p><p><strong>Event logs: </strong>Detailed records of what happens during individual network procedures, such as when a phone tries to connect to the network. Each record shows the steps taken during the connection process, the result of each step, and the session involved. These logs contain the fine-grained signals that can indicate when something has gone wrong.</p><p><strong>Aggregated KPI snapshots: </strong>Every 15 minutes, the network aggregates key performance indicators (KPIs) for each base station (eNodeB) and its cells, including eRAB success rate, RRC connection attempts, retainability, and uplink signal strength. When any metric falls below a predefined threshold, an alert is triggered.</p><p>Among these, the eRAB success rate is a primary indicator of the health of a Long-Term Evolution (LTE) network, measuring the percentage of successful data session setups. While typically above 99%, sustained drops across consecutive intervals signal an incident that requires investigation.</p><p><strong>Incidents: </strong>An incident is triggered when failures persist over time or begin to affect neighboring cells. At this point, the issue is no longer just a single anomaly but a network problem that requires investigation.</p><p>We ported the ANO data from BigQuery into Spanner Graph. (BigQuery Graph, just released for public preview, also supports this workflow). From the Cloud Marketplace we connected SpannerGraph to GraphXR.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2vS0cm23NyiMNs3In8Cd_w.png" /></figure><h3>What the graph shows you</h3><p>Investigators recognize that the investigative process is iterative, involving both answering established questions and uncovering new ones. Visualizing query results as a graph supports this process by illuminating insights and guiding subsequent steps.</p><p><strong>Initial no-code query and graph visualization of base stations and their connected cells</strong></p><p>In the GraphXR search bar, we entered the no-code point-and-click graph query:</p><p>(:eNodeB) — [:HAS_CELL] -&gt; (:Cell)</p><p>This pulls in the base station (<strong>eNodeB)</strong> nodes and their connected <strong>Cells</strong> as a connected graph.</p><p>To better visualize the hierarchical structure, we can right-click and apply a <strong>Tree</strong> layout to the graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*hUYlMy5_5BZ2gtGHb8lBeg.gif" /></figure><p><strong>Incident analysis</strong></p><p>The Google Automated Network Operations (ANO) demonstration features a component called an “incident detector agent.” This agent is designed to automatically monitor two important performance metrics, or Key Performance Indicators (KPIs), to detect problems in the telecom network:</p><ol><li><strong>eRAB success rate:</strong> This measures how often the network successfully sets up a connection for a user. The agent flags an issue if this rate drops <strong>below 97%</strong>.</li><li><strong>Active eRAB release rate:</strong> This measures how frequently active connections are being dropped or disconnected. The agent flags an issue if this rate goes <strong>above 3 times per hour</strong>.</li></ol><p>For the purpose of this study, each <strong>incident</strong> groups the flagged KPIs for each cell over time. We created a simple Grovebook, GraphXR’s built-in low-code notebook and dashboard, to list <strong>incidents</strong> in the data set, together with the associated starting, ending time, cell ID, and number of associated KPISnapshots. This lets a user select an <strong>incident</strong> and load the relevant data into GraphXR.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zu8PbTGLTyETlsTYUoSXyw.png" /></figure><p>Clicking on the Load Incident button dynamically generates the following GQL query using the selected incident_id, sends the query to Spanner Graph, and returns the graph.</p><pre>MATCH (i:Incident)-[a:AFFECTS_CELL]-&gt;(c:Cell)-[h:HAS_KPI]-&gt;(k:KPISnapshot)<br>WHERE i.incident_id = &#39;${id}&#39;<br>AND k.measurement_end &gt;= i.start_ts<br>AND k.measurement_end &lt;= i.end_ts<br>AND k.erab_success_rate &lt; 97.0<br>return *</pre><p>The incident, the affected cells and their KPI snapshots now appear together as a connected graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*npSVhaZQCf3GVPa5XKNcRg.png" /></figure><p><strong>Visualizing KPIs over time.</strong></p><p>From the dashboard, we loaded the KPI snapshots associated with the selected incident and plotted them with time on the x-axis and eRAB success rate on the y-axis. Nodes are scaled in size according to connection volume.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kWRCdg4KhZ0b7YZUwu4UZQ.png" /></figure><p>A daily rhythm shows up immediately: high-volume connections during the day and lower-volume connections during the evening.</p><p>The y-axis showing the eRAB success rate revealed something unexpected: the worst success rates were during quiet hours, when volume was lowest.</p><p>GraphXR includes an agent that understands both the database schema and the analysis workflow. Using a natural language input, it can generate Grovebooks directly from a description of the task. For example, you might say, <em>“show all failed security procedures by device.”</em> Grove will create an interface element — such as a button — that runs the query, applies the layout, and presents the result. The agent can also examine the visual pattern together with the data in the graph, and perform assessment, as shown in this screen capture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LzamGPm_rCjSfqOrgjQP7Q.png" /></figure><p><strong>Plotting KPI properties to explore correlation in 3D</strong></p><p>In GraphXR, we can plot success rate, retainability, and signal strength in a 3-dimensional parametric layout, again scaling node size according to traffic volume. This enables us to observe the correlations behind the failures hidden in multi-dimensional structure, as shown in the two examples below.</p><p><strong>Correlation between traffic volume, success rate and retainability:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WpTOmrVWtMMlFE3_nEQ88Q.png" /></figure><p><strong>Correlation to signal strength:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m_1ErceZQdNJEH4rwKHUzg.png" /></figure><p><strong>Visualizing Failed Procedures and their Devices</strong></p><p>Graph visualization made it much easier to spotlight and follow up on failed procedures.</p><p>A simple GQL query in GraphXR’s query window loads all the failed Procedures:</p><pre>MATCH (p:Procedure) WHERE p.success = false RETURN p</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DxzaZ9LKq4mMloyVAedIUA.png" /></figure><p>We then select the Procedures, right mouse click and <strong>Expand</strong> on the [:PERFORMS] relationship.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DaYuHRr3pGbo-CUfecAPmA.png" /></figure><p>Now the graph shows which devices contributed to the failures. Coloring the devices by the <em>procedure_type</em> property value immediately reveals an interesting situation:</p><ul><li>One device accounted for all 21 INITIAL_CTXT_SETUP_FAILURE (security context setup) failures.</li><li>Three other devices always failed together in the same three procedure types: ERAB_SETUP, INITIAL_ATTACH AND SERVICE_REQUEST, suggesting those three procedure types are related to the same root cause of the failure.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uiKJ2jYzbJTAmHJzqnuglA.png" /></figure><h3>From investigation to ongoing operation</h3><p>In GraphXR, when an analyst completes an investigation and highlights useful patterns, those insights can quickly be turned into a customized low-code <strong>Grovebook </strong>dashboard, as we show. With a Grovebook, users can run queries automatically and repeat data fetching, transformation, and layout steps.</p><p>This makes an investigative workflow accessible to field technicians and operations staff, since they would not need to know GQL or manage database queries directly. With SSO and role-based access controls, these dashboards can also be deployed safely across teams.</p><h3>Getting started</h3><ul><li><strong>Google Cloud Marketplace:</strong><a href="https://console.cloud.google.com/marketplace/product/kineviz-public/graphxr-explorer-spanner"> GraphXR Explorer for Spanner</a>, self-service; listed as an official Spanner Graph visualization partner</li><li><strong>Source data and agent framework:</strong><a href="https://github.com/GoogleCloudPlatform/telco-autonomous-networks-data-demo"> Telco ANO data demo</a> on GitHub</li><li><strong>Free trial:</strong><a href="https://graphxr.kineviz.com/"> graphxr.kineviz.com</a> with your own Spanner Graph endpoint</li><li><strong>Onboarding:</strong><a href="https://www.kineviz.com/googlecloud"> kineviz.com/googlecloud</a> for hands-on setup with the ANO dataset</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8b281dceeb6b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/kineviz/from-spanner-graph-to-operational-insight-visual-telco-analysis-with-kineviz-graphxr-8b281dceeb6b">From Spanner Graph to Operational Insight: Visual Telco Analysis with Kineviz GraphXR</a> was originally published in <a href="https://medium.com/kineviz">Kineviz</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Knowledge Maps — Evidence-First, Context-Preserving Structure for Real-World Knowledge]]></title>
            <link>https://medium.com/kineviz/knowledge-maps-evidence-first-context-preserving-structure-for-real-world-knowledge-31995f86908e?source=rss-ab264147a186------2</link>
            <guid isPermaLink="false">https://medium.com/p/31995f86908e</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[knowledge-graph]]></category>
            <category><![CDATA[graphrag]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <dc:creator><![CDATA[Weidong Yang]]></dc:creator>
            <pubDate>Sun, 25 Jan 2026 00:23:11 GMT</pubDate>
            <atom:updated>2026-01-25T00:40:14.945Z</atom:updated>
            <content:encoded><![CDATA[<h3>Knowledge Maps — Evidence-First, Context-Preserving Structure for Real-World Knowledge</h3><p><strong><em>How to turn messy text into useful structure for analytics and decision-making — while keeping context, time, and provenance intact.</em></strong></p><h3>Introduction</h3><p>Teams are sitting on mountains of unstructured text: filings, emails, meeting notes, tickets, research docs, and chat logs. We want to turn that into something we can query and analyze: entities, relationships, timelines, and “facts.” The risk is that the structure starts to feel like truth — especially when the world changes, sources disagree, and meaning depends on context.</p><p>In this article, I’m proposing a different stance: a <strong>Knowledge Map</strong>. Store evidence first (observations with provenance), use entities and relationships as navigational handles back to that evidence, and compute a “best understanding” at query time based on what matters for the current question. It’s not anti-structure — it’s structure that stays accountable to narrative.</p><p>At Kineviz, we’ve implemented this concept in <strong>SightXR</strong>: an integrated unstructured data processing pipeline, with <strong>GraphXR</strong> as the user interface for exploration.</p><h3>Why text doesn’t want to be “facts”</h3><p>Human knowledge is often recorded as narrative: what someone said, what a document claims, what a meeting concluded, what a report summarizes. This is powerful because it preserves scope, intent, uncertainty, and time. But that same richness makes it hard to flatten into “facts” without losing something important.</p><p>The hard parts aren’t just technical (parsing). They’re real-world: things change, sources disagree, and words are overloaded and context-dependent.</p><p>If structure is produced by aggressively collapsing narrative into fixed “facts,” the resulting system becomes brittle: it will be wrong in precisely the situations where you need it most (edge cases, exceptions, evolving realities).</p><h3>Names, context, and the point of structure</h3><p>Over the last decade, compute has been shifting from brittle, rule-heavy systems toward systems that learn representations from data. A simple way to remember the arc is Deep Blue → AlphaGo: one leaned on explicit rules and search, the other leaned on learned intuition plus search. In knowledge work, that same shift shows up as a move from “hand-built facts” toward systems that can read messy language and still extract useful signals.</p><p>It’s tempting to say we’re moving from “symbolic” systems to “semantic” systems. In practice, you almost always end up with a hybrid. We still need structure for indexing and querying. We also need context for meaning. The question isn’t which to pick — it’s what you allow each layer to claim.</p><p>The Tao Te Ching captures a tension that shows up in every knowledge system:</p><p>“道可道，非常道；名可名，非常名。无名天地之始，有名万物之母。”</p><p><em>(The Tao that can be told is not the eternal Tao; The name that can be named is not the eternal name. The nameless is the beginning of heaven and earth; The named is the mother of all things.)</em></p><p>In plain terms, the unnamed is the full messiness of reality and language, and the named is what we do to make it usable — categories, labels, abstractions. The danger isn’t naming; it’s <strong>**reification**</strong>: treating the label as the thing. When that happens, edge cases and time get flattened, and “facts” drift away from where they came from.</p><h3>Knowledge Graph vs Knowledge Map (what changes in practice)</h3><p>Knowledge Graphs are powerful and widely used. They tend to make structure the primary artifact: entities and predicates become the center, triples become the canonical unit, and “truth” is implied by membership in the graph.</p><p>In practice, many KGs do model provenance, uncertainty, and temporal validity. The issue is that these often become <strong>optional</strong>, <strong>uneven</strong>, or <strong>dropped</strong> — especially when the graph is produced automatically from text.</p><p>A <strong>Knowledge Map</strong> reverses the priority:</p><p>evidence is primary, structure is an index into evidence, and the system does not pretend the structure is a final ontology of reality.</p><p>One immediate implication is <strong>entity resolution</strong>. If evidence is the anchor, resolution can’t be an aggressive “merge everything into one canonical identity” step. It has to be conservative and scoped: resolve within a document when it’s clear (pronouns, implicit subjects), let users specify local disambiguation when needed, and use lightweight global aliases for consistency — while always keeping the original quotes and sources intact. I’ll discuss in detail in a follow-up post on practical resolution strategies and tradeoffs.</p><h3>How a Knowledge Map works: observations + query-time views</h3><p>An <strong>Observation</strong> is a minimal unit of knowledge you can audit: an excerpt (quote), its provenance (source, location, time), and optional structure (entities and relationships) that points back to the excerpt.</p><p>The key principle:</p><h4>The quote is the anchor</h4><p>This doesn’t mean quotes are always true. It means the quote is the <strong>auditable anchor</strong> for what was recorded, and any structure derived from it is a <strong>helpful handle</strong>, not an eternal truth.</p><p>“<strong>Query-time meaning</strong>” is just a practical rule: don’t store one final answer. Store evidence. Compute the view when you ask.</p><p>Most teams want the system to answer questions like “What’s the current status of Project X?”, “Who is the customer’s champion right now?”, or “Is Jerry still at Acme?”</p><p>If you force the system to store a single “current truth,” you immediately run into the real world: status changes, different docs disagree, and “current” depends on what you mean (time window, scope, trust).</p><p>In a Knowledge Map, you do this instead:</p><p>1. <strong>Pull the evidence</strong> relevant to the question (the quotes + where they came from).</p><p>2. <strong>Apply a view</strong> (a small set of rules) to turn evidence into an answer.</p><p>3. <strong>Show your work</strong> (citations back to the quotes).</p><h4>A concrete example</h4><p>Question: “Is Jerry still at Acme?”</p><p>You might have quotes like “Jerry joined Acme in 2018.” (source A) and “Jerry left Acme in late 2024 due to restructuring.” (source B).</p><p>A reasonable view could prefer the newest evidence within the time window you care about; if there’s a later “left” quote, show “not currently.”</p><p>Result:</p><p><strong>Best-effort answer</strong>: likely not at Acme (based on the “left” quote), with citations linking to the two quotes.</p><p>The key point: the answer is a <strong>derived view</strong>, not a permanent fact stored in the database.</p><p>Different views (or different scope) can yield different answers — and that’s okay — because the system stays grounded in evidence.</p><p>Modern language models make it feasible to propose structure from text at scale. The Knowledge Map claim is not “models will finally extract truth.” It’s narrower: models can propose structure, and provenance keeps it honest. When structure is always tethered to evidence, the system becomes auditable (show the source), correctable (improve prompts and re-run extraction), and robust to change (add new observations instead of rewriting history).</p><h3>Why this matters (analytics + decision support)</h3><p>Evidence-linked structure unlocks a more trustworthy pipeline. In practice, this shows up in a few ways. First, analytics: you can count relationship occurrences over time, detect narrative shifts, and build dashboards that remain traceable to sources. Second, decision support: you can generate summaries with citations (“based on these sources…”), surface contradictions (multiple narratives about the same subject), and maintain decision trails (why the system recommended something).</p><p>The defining property isn’t “having a graph.” It’s having an <strong>evidentiary chain </strong>from structure back to language.</p><h3>Practical notes if you’re building one</h3><p>Here are a few rules of thumb. Treat prompts as part of the system: version them, review them, and assume they will evolve. Make provenance non-optional so every extracted item carries source, location, and time (or the best available proxy). Keep structure lightweight with a small set of stable labels, and capture nuance in text and metadata. Store evidence and derive views (“current status,” “best answer”) instead of writing them back as permanent facts. Finally, design for re-processing and show citations in the product — trust comes from letting users see “why” in one click.</p><p>No system escapes error. Extraction errors happen, but citations and re-processing keep them from becoming invisible truth. Naming is never perfectly consistent, but normalization policies help — and names remain handles, not essences. Ambiguity is unavoidable, so preserving context and allowing multiple narratives to coexist is a feature, not a bug. And to avoid false certainty, always show evidence and treat derived views as scoped.</p><h3>Conclusion</h3><p>The point of a Knowledge Map is not to replace your documents with a database. It’s to make unstructured text easier to work with: you can start from an entity or a relationship, immediately see the supporting observations, and understand the story in the original context.</p><p>This is also why we treat “naming” as a tool instead of truth. Labels make search and analytics possible, but the thing you ultimately trust is the evidence: the quote, its source, and when it was written.</p><p>In a future post, I’ll go deeper on entity resolution in this setting: when merging is helpful, when it’s risky, and how to stay conservative without losing usefulness.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=31995f86908e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/kineviz/knowledge-maps-evidence-first-context-preserving-structure-for-real-world-knowledge-31995f86908e">Knowledge Maps — Evidence-First, Context-Preserving Structure for Real-World Knowledge</a> was originally published in <a href="https://medium.com/kineviz">Kineviz</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Primitive ReCall: Part 4of Towards Semantic Interpretation of Dance]]></title>
            <link>https://medium.com/@weidong_yang/primitive-recall-part-4of-towards-semantic-interpretation-of-dance-c60172b1ea40?source=rss-ab264147a186------2</link>
            <guid isPermaLink="false">https://medium.com/p/c60172b1ea40</guid>
            <category><![CDATA[semantic-search]]></category>
            <category><![CDATA[dance]]></category>
            <category><![CDATA[dance-and-technology]]></category>
            <category><![CDATA[vector-database]]></category>
            <dc:creator><![CDATA[Weidong Yang]]></dc:creator>
            <pubDate>Sun, 27 Jul 2025 17:38:51 GMT</pubDate>
            <atom:updated>2025-08-08T05:33:52.720Z</atom:updated>
            <content:encoded><![CDATA[<p>(<em>Images below are from </em><a href="https://kinetecharts.org/open-lab"><em>Kinetech Arts Open Lab</em></a><em>, recorded on 2025–07–23</em>)</p><p>Time to attempt some simple memory recall using the data we’ve collected so far. In this experiment, we asked the AI to live-estimate a dancer’s pose, and then search through a set of stored videos — each with pose data already analyzed — to find a close match.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4y6CddJgswtLMNCKYvxbUw.png" /></figure><p>Some technical details:</p><p>First, we normalize each pose to minimize the effects of scale and orientation. This is applied both to the live pose and to the stored memory data.</p><p><strong>Then comes the recall.</strong> We started with a brute-force approach. Each pose consists of 33 joints, each represented as a [x, y, z] vector, resulting in 99 numerical values per frame. With about 20 minutes of video footage as our memory bank, and assuming 30 FPS, that gives us 36,000 frames to compare against. Comparing a live pose to each of these 36,000 frames takes about 1 second on a modern Apple Silicon CPU. That’s roughly 100 million floating-point operations per second (MFLOPS).</p><p>To put that in perspective: we’re barely scratching the surface of the M4 Pro’s capabilities, which can handle trillions (TFLOPS) of such operations. Clearly, raw Python loops are nowhere near optimized. We’re getting maybe 0.01% of the hardware’s potential.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F1FpKq__7GXP699wA06G2Q.png" /></figure><p>Fortunately, we don’t have to reinvent the wheel. Enter the vector database.</p><p>For this experiment, we chose <strong>LanceDB</strong>: a high-performance, multi-modal, open-source, embedded vector database that runs smoothly on a local machine (sorry for the mouthful of tech jargon). Vector DBs are usually optimized for long vectors — often more than 1000 dimensions — with extremely efficient similarity search. But our vectors only have 99 dimensions. Padding them up to 1000 doesn’t make sense.</p><p>So, we rolled our own solution: we compressed the 99-dimensional vectors into 32 dimensions. This step still needs refining to preserve important structure while removing bias from magnitude differences. For instance, we want to capture both fine arm-hand relationships and the broader head-to-foot spatial distribution. That’s a task for future iterations.</p><p>For reasons we haven’t yet understood, LanceDB’s built-in cosine similarity didn’t work well with our tiny 32-dim vectors, so we implemented our own similarity function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nqu0Fo_5VYEiNhmpnlsB3w.png" /></figure><p><strong>The result?</strong> Lookup time dropped to under 100 milliseconds — a more than 10x speedup over brute-force. As we scale up the memory bank, brute-force lookup time grows linearly. Not so for vector search: its increase in time is much more graceful. And I’m confident there’s room to push another 10x improvement, potentially bringing lookup time down to 1ms.</p><p>Reducing the vector from 99 to 32 dimensions introduces some fuzziness in the matching. We also deliberately introduce a degree of randomness to avoid always retrieving the same so-called “best” pose. From our lab experience, we’ve learned that the “best” match is rarely the most interesting. A touch of unpredictability tends to spark more curiosity and engagement.</p><p>(<em>Side note: platforms like YouTube and Facebook relentlessly optimize for the “best” match, leading to echo chambers — attention-grabbing, yes, but often monotonous and mentally stifling.</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-ikaaqj15irh6QUeod1RNQ.png" /></figure><p>This is a collaborative project. Jason Grafft just posted his work on <a href="https://medium.com/@jgrafft/cheaply-but-not-easily-aligning-multiple-high-speed-digital-video-streams-6541a84edb66">aligning multiple high-speed video streams</a>.</p><p>And Natalie Marie wrote about <a href="https://medium.com/@nataliemariesober/part-1-dance-ai-a-dancers-perspective-5c19f5da8162">Part 1 Dance+AI: A Dancer’s Perspective</a></p><p>Interested in joining us in the lab? You can find us <a href="https://kinetecharts.org/open-lab">here</a>. And you can find my last post here:</p><p><a href="https://medium.com/@weidong_yang/forcing-ai-to-hallucinate-part-3-of-towards-semantic-interpretation-of-dance-341e0577e529">Part 3</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c60172b1ea40" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Forcing AI to Hallucinate: Part 3 of Towards Semantic Interpretation of Dance]]></title>
            <link>https://medium.com/@weidong_yang/forcing-ai-to-hallucinate-part-3-of-towards-semantic-interpretation-of-dance-341e0577e529?source=rss-ab264147a186------2</link>
            <guid isPermaLink="false">https://medium.com/p/341e0577e529</guid>
            <category><![CDATA[dance-and-technology]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[dance]]></category>
            <dc:creator><![CDATA[Weidong Yang]]></dc:creator>
            <pubDate>Fri, 11 Jul 2025 04:34:33 GMT</pubDate>
            <atom:updated>2025-07-27T17:40:30.835Z</atom:updated>
            <content:encoded><![CDATA[<p>This is a common theme in the <a href="https://kinetecharts.org/open-lab">Kinetech Arts Open Lab</a>, we often interested as much in what breaks, as in what work. So naturally, just as we were planning to build on the progress from our last lab with clean, choreographed, and controlled data collection, a new idea began to emerge: what if we challenge the AI and intentionally push it to fail — spectacularly?</p><p>Natalie posed an intriguing test: could the AI recognize a dancer upside down, especially when the body is folded in complex ways? Sometimes it managed, but often it failed in fascinating fashion. In one image, the AI mistook the shadow of a leg for an actual leg. In another, despite Natalie clearly being upside down, the AI insisted she was upright.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GOrAy3fLRSbhHAYT06o01A.png" /><figcaption>Natalie: Left, mistaken shadow as leg. Right, AI think she is upright</figcaption></figure><p>Then we raised the stakes: what if one dancer stood directly behind another? As expected, the AI struggled. It began picking limbs from each person, seemingly randomly, combining them into a composite body that defied human anatomy — an accidental surrealist sculpture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*avtEOjwWdDc5jtwO74asvg.png" /><figcaption>Sara (front) and Sheryl (back). AI pick and choose from two persons and compose a figure that is pushing the boundry of human anatomy</figcaption></figure><p>Finally, we tested the ultimate case: two dancers entangled. The result was a delightful mess. The AI sensed a human presence but lost the ability to distinguish arms from legs. The body parts became interchangeable in its reconstruction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T6jTQFd2CSItyyDtyfLq8w.png" /><figcaption>Daiane and Sonia Really pushing the boundary</figcaption></figure><p>To be fair, we’re pushing the model far beyond the scope of its training. It’s no surprise it can’t generalize well under these conditions. What we’re seeing are examples of the model extrapolating with weak constraints, leading to physically impossible and unstable outputs.</p><p>Today’s AI models can perform quite well within the boundaries of their training datasets. That’s why high-quality, diverse datasets are so important. While the exact datasets used to train <a href="https://ai.google.dev/edge/mediapipe/solutions/vision/pose_landmarker">MediaPipe Pose</a> (the model we used) aren’t fully public, ChatGPT helped us trace likely sources: the <a href="https://cocodataset.org/">COCO dataset</a>, the <a href="http://human-pose.mpi-inf.mpg.de/">MPII Human Pose Dataset</a>, the <a href="http://vision.imar.ro/human3.6m/description.php">Human3.6M</a>, and Google’s own internal 33-landmark dataset.</p><p>According to the original <a href="https://arxiv.org/abs/2006.10204">BlasePose paper</a>, the training pipeline included three stages:</p><p><strong>Stage 1</strong>: 2D landmark detection, initially trained on COCO/MPII, then refined with Google’s proprietary dataset.</p><p><strong>Stage 2</strong>: 3D estimation, trained using synthetic and motion capture data with multi-camera depth.</p><p><strong>Stage 3</strong>: Tracking and temporal smoothing to ensure stable predictions over time.</p><p>By putting the AI in extreme unfamiliar scenarios, we are in fact, forcing AI to hallucinate.</p><h3><strong><em>July 16</em></strong></h3><p>In this lab, we pushed the idea of forcing AI to hallucinate even further. We deliberately created scenarios designed to trick the AI into perceiving two people as one — an experiment that turned out to be more challenging than we anticipated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3Ks5M9F7iNEdkBXwSLkaKg.png" /><figcaption>AI was able to track a specific person even in side by side, or blocking situations. Albeit with reduced confidence. Dancers: Latiece and Riva</figcaption></figure><p>It took some effort to find the “right” composition to fool AI:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jxPQRkXx59MPE1u8WQz0tg.png" /><figcaption>Some success attempts. Dancers: Latiece and Riva</figcaption></figure><p>In next week’s lab, we’ll be even more deliberate in getting the AI to “confidently” make the wrong decisions. Stay tuned.</p><p>Interested in joining us in the lab? You can find us <a href="https://kinetecharts.org/open-lab">here</a>.</p><p><a href="https://medium.com/@weidong_yang/semantic-interpretation-of-dance-sequences-part-2-ec201b2b8c6d">Part 2</a>, <a href="https://medium.com/@weidong_yang/primitive-recall-part-4of-towards-semantic-interpretation-of-dance-c60172b1ea40">Part 4</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=341e0577e529" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Semantic Interpretation of Dance Sequences — Part 2]]></title>
            <link>https://medium.com/@weidong_yang/semantic-interpretation-of-dance-sequences-part-2-ec201b2b8c6d?source=rss-ab264147a186------2</link>
            <guid isPermaLink="false">https://medium.com/p/ec201b2b8c6d</guid>
            <category><![CDATA[dance]]></category>
            <category><![CDATA[visualization]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Weidong Yang]]></dc:creator>
            <pubDate>Sat, 05 Jul 2025 03:07:56 GMT</pubDate>
            <atom:updated>2025-07-11T04:37:18.324Z</atom:updated>
            <content:encoded><![CDATA[<h3>Semantic Interpretation of Dance Sequences — Part 2</h3><h3>Understanding the Limit of MediaPipe Pose Tracking and Exploring PCA, t-SNE and UMAP</h3><p>In this week’s Open Lab (Wednesday, July 2nd at ODC Annex), we improved lighting and camera angles to get higher quality video for pose analysis using MediaPipe. Here are a few key observations:</p><ul><li>High contrast with the background is good. Shadow, surprisingly, may not be a problem. We used a plain white wall and had dancers wear dark clothing. This helped separate their bodies from the background, improving tracking accuracy.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*94T-P8zBNpWvYbMdvIUogQ.png" /><figcaption>Left Jawa, Right Daiane. Lighting and contrast with background helps</figcaption></figure><ul><li><strong>Baggy clothes are a problem.</strong> Loose tops interfered with arm detection. Also, red clothing sometimes confused the model.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y9zHK_e0VjWQtlj247tnCA.png" /><figcaption>The red pants causing difficulty for pose extraction. Also the baggy top caused difficulty in arm recognition</figcaption></figure><ul><li><strong>Surprisingly good at fast motion.</strong> MediaPipe seems to “remember” previous frames, which helps it handle fast movement and short-term occlusion.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c46LWH2WxaOrP5a3rVeSmQ.png" /><figcaption>Natalie spinning. 3D pose rotated to show that, even with body side way to the camera, the pose is still accurate.</figcaption></figure><ul><li>It can track poses not common in daily live.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LI2GwwL1guE4HuafulIpew.png" /><figcaption>Clockwise: Kurt, Jawa, Daiane,Kurt</figcaption></figure><ul><li><strong>Unfamiliar poses are harder.</strong> Common dance poses — especially on the floor or sideways to the camera — can break the model. This is likely because the training data didn’t include enough examples from dance contexts.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yq-EQ4--7-AAR_A9UoD8vA.png" /><figcaption>Pushing the limit. Left: Daiane, Right: Natalie</figcaption></figure><p>Despite its limitations, MediaPipe’s pose tracking handled many dynamic dance moves with impressive robustness.</p><h3>Visualizing Dance with PCA, t-SNE, and UMAP</h3><p>Once we captured the pose sequences, we explored three techniques to simplify and visualize the movement data: PCA, t-SNE, and UMAP. These tools reduce high-dimensional pose data into 2D or 3D so we can better see patterns.</p><ul><li><strong>PCA (Principal Component Analysis)</strong> finds the main axes of variation. It captures large-scale patterns like walking vs. jumping. However, it tends to miss small, subtle movements like a wrist flick or a shoulder tilt. The upside is that PCA is consistent and transferable across datasets.</li><li><strong>t-SNE (t-distributed Stochastic Neighbor Embedding)</strong> focuses on local similarity. It groups similar poses together, revealing clusters of similar shapes or transitions. It’s great for spotting fine details, but its results vary between runs and often appear chaotic.</li><li><strong>UMAP (Uniform Manifold Approximation and Projection)</strong> gives us a balance. Like t-SNE, it preserves local structure, but it also maintains more of the global relationships in the data. It’s faster, more stable, and better at mapping entire sequences across sessions.</li></ul><p>In short: PCA shows the big picture, t-SNE reveals fine texture, and UMAP does both. Together, they help us turn dance sequences into visible structures — revealing the flow, rhythm, and transformations in movement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c9FLJvL-_SiJvcr6j0k9wQ.png" /><figcaption>1:39 sequence of dance by Jawa visualized in PCA, t-SNE and UMAP</figcaption></figure><p>In the figure above, we see the same 1:39 movement sequence by Jawa visualized in all three methods: <strong>PCA</strong> shows a smooth trajectory, dominated by large body movements. Subtle gestures are hidden. <strong>t-SNE</strong> has more spread and shows fine-grained variation, but the path is fragmented, making it hard to interpret over time. <strong>UMAP</strong> captures both detail and order. It shows a more coherent structure while still preserving nuanced differences.</p><p>Building on these insights, our next experiments will combine PCA and UMAP to prepare us for training vector embedding models. These models will help us better capture and describe the semantic richness of complex dance sequences.</p><p><a href="https://medium.com/@weidong_yang/semantic-interpretation-of-dance-sequences-part-1-ba1f26aac5cd">Part 1</a>, <a href="https://medium.com/@weidong_yang/forcing-ai-to-hallucinate-part-3-of-towards-semantic-interpretation-of-dance-341e0577e529">Part 3</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ec201b2b8c6d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Semantic Interpretation of Dance Sequences — Part 1]]></title>
            <link>https://medium.com/@weidong_yang/semantic-interpretation-of-dance-sequences-part-1-ba1f26aac5cd?source=rss-ab264147a186------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba1f26aac5cd</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mls]]></category>
            <category><![CDATA[dance]]></category>
            <dc:creator><![CDATA[Weidong Yang]]></dc:creator>
            <pubDate>Sun, 29 Jun 2025 17:11:12 GMT</pubDate>
            <atom:updated>2025-07-05T03:08:45.010Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Semantic Interpretation of Dance Sequences — Part 1</strong></h3><h3><strong>Dimensionality Reduction With Dance Pose Sequences</strong></h3><p>Vector embeddings have revolutionized how we interpret the semantics of text documents, enabling applications like Retrieval-Augmented Generation (RAG), which power many AI-assisted systems today. This blog is part of an ongoing exploration: can the same techniques be applied to dance — or more broadly, to movement? Can we understand and retrieve dance sequences based on their semantic structure?</p><p>Potential applications of this research include:</p><ul><li>Dance style comparison</li><li>Technical feedback and training for dancers</li><li>Real-time performance augmentation</li><li>Interactive performances with semantic understanding of movement</li></ul><p>In our first experiment, during <a href="https://kinetecharts.org/">Kinetech Arts</a>’ <a href="https://kinetecharts.org/open-lab">weekly Open Lab</a> on June 25, 2025, we recorded several short videos of dancers moving through space. Using <a href="https://github.com/google-ai-edge/mediapipe">MediaPipe</a>’s pose estimation integrated with the <a href="https://rerun.io/">Rerun</a> framework, we extracted 3D pose data consisting of 33 key points per frame, along with synchronized timecodes.</p><p>We then applied dimensionality reduction techniques — PCA, t-SNE, and UMAP — to project each high-dimensional pose (99 dimensions: 33 points × 3 coordinates) into a 2D or 3D space. This allowed each dance sequence to be visualized as a trajectory in a reduced-dimensional embedding space.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UUQo18hwS0TA47yIcju3kQ.png" /><figcaption>A UMAP pathway constructed form a short sequence with Daiane Lopes da Silva</figcaption></figure><p>In the next lab session on July 2, 2025, we’ll design a systematic set of movement sequences. Instead of embedding individual poses, we will experiment with embedding temporal “chunks” of movement (e.g., one-second segments at 30 fps, resulting in 2970 dimensions) and projecting those into lower dimensions. This approach aims to better capture the dynamics and flow of movement over time.</p><p>Eventually, we plan to train a dedicated embedding model that learns meaningful representations of entire dance sequences.</p><p>This work is part of <a href="https://kinetecharts.org/open-lab">Kinetech Arts’ weekly Open Lab</a>, held every Wednesday from 7:00 to 8:30 PM at <a href="https://odc.dance/">ODC</a> in San Francisco. Everyone is welcome to join and participate in this intersection of dance, AI, and embodied research.</p><p>You can also follow this project on the g<a href="https://github.com/kinetecharts/dance-embedding">ithub repository</a>.</p><p><a href="https://medium.com/@weidong_yang/semantic-interpretation-of-dance-sequences-part-2-ec201b2b8c6d">Next post</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba1f26aac5cd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Investigating Cybersecurity in Public Companies]]></title>
            <link>https://medium.com/kineviz/investigating-cybersecurity-in-public-companies-0dc0fb31a4d8?source=rss-ab264147a186------2</link>
            <guid isPermaLink="false">https://medium.com/p/0dc0fb31a4d8</guid>
            <category><![CDATA[neo4j]]></category>
            <category><![CDATA[visualization]]></category>
            <category><![CDATA[investigation]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[knowledge-graph]]></category>
            <dc:creator><![CDATA[Weidong Yang]]></dc:creator>
            <pubDate>Fri, 15 Mar 2024 18:35:44 GMT</pubDate>
            <atom:updated>2024-03-15T19:23:29.083Z</atom:updated>
            <content:encoded><![CDATA[<h4>Extract and visualize incidents reported in SEC 8K and 6K filings</h4><h4>Introduction</h4><p>The financial statements and background information that publicly traded companies must provide in Securities and Exchange Commission (SEC) filings are an important source of information about the operational stability of a company and its dealings with owners, partners, customers, and regulatory bodies. In 2023, new rules introduced mandatory reporting of cybersecurity-related incidents for all companies listed in the US. Domestic issuers must disclose material cybersecurity incidents in a Form 8-K filing, and private foreign issuers must disclose such incidents in a Form 6-K filing. These SEC filings can be used to construct a history of incidents and to evaluate a company’s cybersecurity vulnerabilities.</p><p>SEC filings are often lengthy. Asking a human investigator to read through many filings can be an overwhelming prospect. Furthermore, cybersecurity incidents are just one of the many kinds of material events that trigger a requirement to file. Manually extracting connected information on that single topic of interest is far too slow and unreliable.</p><p>With SightXR we have a platform that delivers rapid access to the information in unstructured text documents. We can use it to:</p><ul><li>select and ingest relevant filing documents,</li><li>extract and connect specific entities and relationships related to cybersecurity incidents using AI-powered language models,</li><li>store and display the results as a connected knowledge map,</li><li>expand, explore, organize, and simplify the knowledge map, and</li><li>answer further questions about the reported incidents using Q&amp;A chat.</li></ul><p>In this article we’ll show how to use SightXR for a fast, basic investigation into reported cybersecurity incidents.</p><h4>Ingest filing documents</h4><p>SEC filings are provided online through searching its Electronic Data Gathering, Analysis, and Retrieval (EDGAR) system. We created a Streamlit utility to search and ingest selected filings into SightXR. It shows the main properties of the filings and lets us select documents by topic and range of dates. Ingesting a filing retrieves the complete document and submits it to SightXR, together with all the properties available in the table resulting from the search.</p><p>For an initial demonstration, a search for filings with cybersecurity incidents over the last three months returned 15 filings that we then ingested into SightXR.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ysZYnjulWrJJPHCY" /></figure><h4>Extract entities and build the knowledge map</h4><p>In SightXR, we choose the provider of the large language model (LLM) that extracts the entities and relationships used to build the resulting knowledge map. Also, the LLM’s Q&amp;A-style chat adds a new dimension to iterative exploration of the filing documents. For this example, we’re using GPT 3.5 turbo.</p><p>Extraction is based on a data model that specifies the entities of interest. We use the default POLE model (<strong>P</strong>ersons, <strong>O</strong>rganizations, <strong>L</strong>ocations, and <strong>E</strong>vents) popularized by Neo4j, since it’s a standard approach in investigative and security use cases.</p><h4>Store and visualize extracted results as a connected knowledge map</h4><p>Extraction detects entities (nodes and relationships) according to the POLE model. Results are stored in a graph database, in this case Neo4j. Entities are also connected to the source documents through <em>Chunk</em>,<em> Observation</em>, and <em>Source</em> categories. Collectively it forms a Knowledge Map, a loose form of Knowledge Graph.</p><p>That done, we go to the <strong>Visualize</strong> step and click <strong>Everything</strong> to see the POLE entities that have been created and their relationships to <em>Observation</em> nodes. In the workspace, entities are displayed as nodes and connected edges which are assigned category and relationship labels as specified in the model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZlRNL-z2VUDZ0-WPS8vK7Q.png" /></figure><p>With all the nodes and edges present, we can get a visual sense of the data we have to work with, and evaluate the results of our extraction process.</p><h4>Re-organize, and simplify the knowledge map</h4><p>A useful first step is to see that categories have been assigned as we expect.</p><p>For example, when we click <em>Organization</em> and search for nodes with a label that includes SEC, we see a few nodes that should be selected and removed, as they do not actually refer to an organization.</p><p>We also see that due to spelling variations or errors in the filing documents, a few of the categories should be merged into one, and we can do this immediately.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3AFLWvgBa88PcJpkYhmqWQ.png" /></figure><h4>Explore and organize starting with source documents</h4><p>We’ve seen how to view all the extracted entities. Now we’ll start an investigation by working with the source documents. First we clear the workspace by selecting all the entities and deleting them. They still exist in the graph database, as we’ll soon see. To display the source documents within the workspace, we click on the <strong>Source</strong> category and <strong>Pull All</strong> button. A table is displayed, and a separate node for each source document appears in the workspace.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PhLdmzsunwrJAVCfr_NS6g.png" /></figure><p>In the graph workspace we can also work with tables for any node or edge label, perform data cleaning and transformation, run graph analytics such as path finding and centrality, and apply a variety of layout and caption options to help focus an investigation.</p><p>For example, <em>Source</em> nodes have a <em>display_name</em> property that’s a bit complicated. We can use an <strong>f(x)</strong> transform to create a readable <em>company_name</em> property and apply it as the caption</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QVHsbPkQJ2iAd6w-i0VXHg.png" /></figure><p>To connect all the filings from a given company, we use an <strong>Extract</strong> transform to create a new <em>Company </em>category based on a <em>companyName</em> property.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*atRFHkkj4p0lW5MH9p7Nhw.png" /></figure><h4>What organizations are involved in the filings?</h4><p>We can pull <em>Organization</em> nodes from the database to see the organizations that are connected to the companies and their incidents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YG1WJGbwGleH4p5I-YQeZw.png" /></figure><p>The SEC organization node can be removed, since it’s obviously connected to all the source documents (again, it remains in the Neo4j database).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kKRMeDNH6-CxXumsz4MKmA.png" /></figure><h4>Expand with Observations</h4><p>We can Expand along relationships to show an organization’s connected observations, and then arrange the observations over time to see how a cybersecurity incident was handled as it progressed from discovery to investigation to resolution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XKMzUYq3P4qRWVmu_Rs7nQ.png" /></figure><h4>Answer questions using Q&amp;A chat</h4><p>Now that we have the basic story for the company and incident, we can continue to investigate by entering questions in the Q&amp;A chat. Answers are returned with an explanation and a summary. Such conversational response is likely to spark further questions and insights which can in turn be quickly explored.</p><h4>Conclusion</h4><p>The SightXR platform provides seamless and rapid access to unstructured information in ways that have recently become possible with the advent of large language models (LLMs) and GenAI. The key role of the LLM here is to prepare the documents for investigation by humans. You can start from an entity, immediately pull all the observations associated with it, and gain a comprehensive understanding of that entity in the context of provided documents.</p><p>Conceptually, we’re building on retrieval augmented generation (RAG), an architecture that improves the performance of LLM applications by leveraging custom data. By combining observation and knowledge maps, we allow a language model to transform documents into a more open-ended investigative environment that retains essential context and nuance. The resulting knowledge maps provide visual cues to help humans gain quick insights, and at the same time improves the quality of RAG. We find that this approach makes it dramatically faster and easier to obtain insights from masses of unstructured text.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0dc0fb31a4d8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/kineviz/investigating-cybersecurity-in-public-companies-0dc0fb31a4d8">Investigating Cybersecurity in Public Companies</a> was originally published in <a href="https://medium.com/kineviz">Kineviz</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Feature Graph for Proactive Threat Detection]]></title>
            <link>https://medium.com/kineviz/the-feature-graph-for-proactive-threat-detection-fd1e290401c1?source=rss-ab264147a186------2</link>
            <guid isPermaLink="false">https://medium.com/p/fd1e290401c1</guid>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[neo4j]]></category>
            <category><![CDATA[real-time-analytics]]></category>
            <category><![CDATA[threat-detection]]></category>
            <dc:creator><![CDATA[Weidong Yang]]></dc:creator>
            <pubDate>Fri, 08 Dec 2023 18:58:15 GMT</pubDate>
            <atom:updated>2023-12-08T18:58:15.775Z</atom:updated>
            <content:encoded><![CDATA[<p>In a previous article, I introduced a tiered architecture for detecting and combining specific patterns in big data streams.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MwASZUH1WO_nvzE9015UZw.png" /></figure><p>The <strong>feature graph</strong> tier is a central element of that design within the graph environment. Let’s look at it in more detail and see how it works.</p><p><strong>What is a feature graph?</strong></p><p>Simply put, the feature graph consists of a set of pre-defined component graph patterns. A feature implies connection. You are defining individual elements typical of the patterns you’re interested in. In the same way that a smiley face can be built from a set of simpler elements such as a dot, or sub segments of a circle, a complex threat pattern can be built from features.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nP_M0O94fEyRVB1FNcM0Dw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cU_l8-vzOgYhQ9YLZtNCow.png" /></figure><p>We find that even complex threat patterns can be broken down into a manageable set of such features. Features typically fall into a couple of broad categories:</p><ul><li>Features suggesting connections, such as shared IP, shared location, accounts created at the same time, accounts accessing the same resource or accessing the same channels.</li><li>Features suggesting anomalies. These include a variety of coordinated actions (such as flow of funds to a single destination from multiple sources), multiple failed attempts, or unnatural timing and trends.</li></ul><p>Once you capture features they are saved in a graph database. The feature graph is organized according to a graph schema similar to the simplified one shown below, in which actor, event, resource, IP, and feature categories are connected through specified relationships.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VHdDWw_RWanV-Ic104wkXg.png" /></figure><p>The feature graph is designed to be extensible. You can start with building blocks of known threat patterns. Over time, you capture other interesting features, and this serves later to identify further complex and emerging threats.</p><p><strong>From features to threat patterns</strong></p><p>We employ the feature graph to detect features as they arrive in a data stream. Individually a feature is not enough to raise an alert. But when component features connect in specific ways, the resulting graph pattern can clearly highlight an emerging threat before it has a chance to succeed.</p><p>Here’s an example where we look at patterns of usage over time. We are interested in people sharing IP addresses because the accounts might be created by the same person (or group of people acting together). The graph represents users sharing IPs on a specific date. On day one we see many users sharing IP addresses. That in itself may not be a problem. A few days later we looked at the same group of people. Notice again there are many users coming from the same IP. So now we see a pattern implying that there may be collaboration behind the scenes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EdtO29MC67thuNA8-UAvFA.png" /></figure><p>To explore that possibility, we bring the days together using the IP as a connection. Now we can see that a large group of people are actually connected with each other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FrB1zeT1GkaVKSZk4v786g.png" /></figure><p>In this connected group there’s one user who has been flagged as a known bad actor. Previously we had no idea about any of the other connected users over time, but now we can immediately use label propagation to flag all these users over multiple hops and add them to our watch list.</p><p>This is a fairly straightforward example of finding connections among users sharing the same IP over time. Other resources can be shared as well. For example, creating accounts at the same time can be a signal to establish a connection. Or, accessing the same resource at the same time can suggest collusion among bad actors.</p><p>For proactive investigation, threat patterns that are detected in the data stream can be rapidly explored and augmented as needed from the raw data. Once you’re able to visualize the connections as a graph, you can start to see hidden patterns that are very difficult to detect just by looking at the original data. This makes it possible to identify new features of interest and to visualize new patterns of attack that are being set up.</p><p><strong>Conclusion</strong></p><p>Features are used to build patterns that signal a known threat, also stored as graph data. Pre-calculated features…</p><ul><li>Let you immediately visualize simple patterns in big data streams,</li><li>Provide a way to build more complex threat patterns quickly</li><li>Drastically reduce cost and improve speed for threat detection.</li></ul><p>This architecture lets us efficiently detect threat patterns as they emerge, and explore the graph for new variations and strategies as they occur.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fd1e290401c1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/kineviz/the-feature-graph-for-proactive-threat-detection-fd1e290401c1">The Feature Graph for Proactive Threat Detection</a> was originally published in <a href="https://medium.com/kineviz">Kineviz</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Graph Strategy for Proactive Threat Response]]></title>
            <link>https://medium.com/kineviz/graph-strategy-for-proactive-threat-response-dcd058bfc200?source=rss-ab264147a186------2</link>
            <guid isPermaLink="false">https://medium.com/p/dcd058bfc200</guid>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[real-time-analytics]]></category>
            <category><![CDATA[graph-analytics]]></category>
            <category><![CDATA[real-time-alerts]]></category>
            <category><![CDATA[anti-fraud]]></category>
            <dc:creator><![CDATA[Weidong Yang]]></dc:creator>
            <pubDate>Tue, 28 Nov 2023 22:16:48 GMT</pubDate>
            <atom:updated>2023-11-28T22:16:48.524Z</atom:updated>
            <content:encoded><![CDATA[<p>Patterns of events that signal a threat can often be found after the fact in an enterprise’s big data archive. The patterns are typically complex, involving many different entities and connections over time. But they are hard to detect because the events of interest are sparsely embedded in the data stream. In addition, they are highly dynamic because bad actors regularly change strategies to avoid detection.</p><p>With a traditional monitoring approach, a big data flow typically goes to a data lake with a BI dashboard implemented to raise alerts as needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rkf3K6awQtW-iTrukXiTTw.png" /></figure><p>This is effective for detecting known threats, such as activity of known bad actors, or spotting statistical outliers. However, incidents are seen as isolated events and not parts of a connected pattern, and this makes it hard to put events in the proper context or to visualize patterns over time. Worse yet, the response is essentially reactive: patterns that haven’t previously been seen to cause a problem are not flagged.</p><p>We need to be far more proactive at identifying probable threats, ideally in near real time. Expressing threat events as connected graph patterns seems to be a natural solution. Unfortunately, importing entire big data streams into a single graph database is prohibitive in terms of both cost and performance, especially since events relating to threats represent such a miniscule part of the entire data flow.</p><p><strong>Crafting a graph-enabled big data solution</strong></p><p>To deliver near-real time threat detection, we’ve built a tiered solution that focuses on the respective strengths of graph technology and tabular big data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sUeHFXu7hvShjQTbA5XWfQ.png" /></figure><p>Instead of pulling an entire data stream into a graph database, big data is connected to a graph environment in which we define, store, and detect threat patterns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XznD6ZR1ciQtOVEUjA-gQA.png" /></figure><p>To start with, we store pre-defined component patterns, or features, that may be associated with a threat. Examples are users with a shared IP address or shared location, accounts created at the same time, and so on. The feature graph tier continuously monitors the data stream, and extracts and stores component patterns of interest into the graph database. Pre-defined features drastically reduce cost and improve speed of threat detection. In terms of storage requirements, a thousand-fold reduction can be achieved without sacrificing effectiveness. This means, for example, that 10 terabytes of raw data requires only about 10 gigabytes of graph storage.</p><p>Then we combine features into more complex threat patterns. A feature by itself is not enough to raise an alert. But when features connect in specific ways, patterns emerge that do warrant attention. The threat patterns are also stored in the graph and, when detected, are configured to trigger appropriate alerts.</p><p>With the feature graph, pattern search is fast and nimble. We still need tables to store big data so that we can access events related to a feature that may signal a possible emerging threat. But now we’re keeping only the most important data in active memory. The raw data is only accessed for highly focused investigation. This is cheap and fast since only small amounts of relevant historical events are needed at any given time.</p><p>For visualization and exploratory analysis, we embed GraphXR, our browser-based graph visualization platform. This highly visual approach to connected data encourages rapid ad-hoc discovery of hidden patterns, connections, and anomalies. We can connect it to graph as well as SQL databases, and its code-free tools empower subject matter experts to investigate patterns of connection over time and geospatial location. A notebook extension enables fast creation of reproducible workflows for data access, analytics, and both static and animated display.</p><p>Let’s look at an example focused on the flow of transferred funds over time. Here we see a classic pattern in which a group of connected users are sending funds over many hops to a single ultimate destination.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9wvrj7ss9sZMg3RSObWgOQ.png" /></figure><p>We can lay out the display to show an added timeline dimension, which clarifies even further the nature of the possible threat.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BdzyWT18Og-5TkuSdRTUzw.png" /></figure><p>Animating the flow of transactions within the network is a particularly effective way of showing which users are sources of funds, and which are acting as a bridge between the source and ultimate destination.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*VrGH3Z4nHNYKUSPG_YL_sg.gif" /></figure><p><strong>Conclusion</strong></p><p>Graph feature detection, visualization, and exploratory analytics, when coupled with live access to the raw data in a data lake, spotlights specific component patterns hidden in torrents of big data, and enables exploration and extension of patterns in the context of past history and activity. This practical tiered strategy has proven capable of delivering proactive threat detection in ways that BI-big data solutions simply can’t match. It now becomes possible to create more nuanced dashboards that not only deliver statistics, but also connected patterns and the context in which they occur.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dcd058bfc200" width="1" height="1" alt=""><hr><p><a href="https://medium.com/kineviz/graph-strategy-for-proactive-threat-response-dcd058bfc200">Graph Strategy for Proactive Threat Response</a> was originally published in <a href="https://medium.com/kineviz">Kineviz</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI-Powered Knowledge Maps for Navigating Unstructured Data]]></title>
            <link>https://medium.com/kineviz/ai-powered-knowledge-maps-for-navigating-unstructured-data-97a55be43ed3?source=rss-ab264147a186------2</link>
            <guid isPermaLink="false">https://medium.com/p/97a55be43ed3</guid>
            <category><![CDATA[graph-visualization]]></category>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[unstructured-data]]></category>
            <dc:creator><![CDATA[Weidong Yang]]></dc:creator>
            <pubDate>Fri, 15 Sep 2023 23:22:41 GMT</pubDate>
            <atom:updated>2023-09-15T23:22:41.335Z</atom:updated>
            <content:encoded><![CDATA[<p>Investigations often involve analyzing large document sets–from text such as emails to audio and video content–and the amount of pertinent content keeps exploding. Twenty years ago, the investigation into the Enron scandal included a total of a half a million documents. Today, a large organization typically generates that number of documents in just a few days. If a human must read each document and manually extract and cross-reference all the details of interest, finding specific insights or even settling on the right questions to ask simply becomes impossible. So how can information buried in such large masses of unstructured data be brought to light? One solution is to support an iterative discovery process which can spotlight important information. While narrowing down the pool of relevant documents, this approach still supports the discovery of new questions.</p><h3>AI-assisted knowledge mapping</h3><p>By integrating recent advances in large language models (LLMs) with graph data technology, it’s possible to build a <em>knowledge map</em> from unstructured content that enables rapid exploration and modification of data from varied sources.</p><p>Kineviz has recently been implementing these ideas in a new SightXR product designed to work seamlessly with our GraphXR visual analytics platform. We can now easily engage with unstructured data to:</p><ul><li>Automatically tag topics of interest, which drastically narrows the scope of the information to be reviewed in a document set.</li><li>Create a knowledge map to enable fast data navigation.</li><li>Perform keyword search and semantic search to find relevant information and patterns.</li><li>Engage with data via Question-and-Answer (chat) or Summarization.</li></ul><p>Relationships and entities within and across documents can then be accessed through keyword search or semantic search using a natural language question and answer format. Data from any source can be used as input, and the identity of data sources retained to enable evaluation of results for accuracy and reliability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*06cbCGQ_WZwu7CoM" /></figure><p>The architecture includes a data pipeline which allows for processing of different kinds of unstructured data, leveraging a large language model (LLM) to recognize entities and relationships and enable semantic search. We store the extracted entities and relationships in a graph database such as Neo4j, and the embeddings for semantic search in a vector database.</p><h3>Knowledge map-assisted navigation</h3><p>The knowledge map becomes a starting point for navigating the data to spotlight answers to specific questions of immediate interest. We start with a general mapping because in most cases, it’s impractical and unnecessary to build a true knowledge graph that tags and connects all the information in a large unstructured document set. What’s most often needed is a way to reveal which documents hold information of immediate interest, and a way to investigate that information in more detail. From the knowledge map, a human can quickly narrow down an investigation using natural language question and answer format, and then expand upon elements that are deemed important.</p><p>Detailed knowledge graphs based on chat answers are then constructed automatically for rapid visualization using GraphXR layouts, analytics, and transforms. By limiting the LLM-powered semantic search exclusively to a mapped document set, the risk of hallucination is effectively eliminated.</p><p>Document nodes can be pulled directly from the graph database into GraphXR via full text search and the ability to click and expand on relationships. For example, in a public archive of emails related to the Enron bankruptcy litigation, names of people have been encoded as nodes in the knowledge map, and connected to email document nodes through a MENTIONS relationship. Selecting a specific Person node and expanding on its MENTIONS relationships brings just those connected email document nodes into the graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AYixu5Cl_jp8V7-q" /></figure><p>As knowledge becomes available in the map, GraphXR enables a wide variety of graph analytics. For example, pagerank values can be generated with one click to reveal important relationships that might otherwise not be apparent. And GraphXR’s many selection, display, and layout options support the ability to highlight and quickly isolate patterns of interest.</p><h3>Conclusion</h3><p>We find that an application built on LLM capabilities provides an essential missing link for creating focused, explainable discovery of patterns in unstructured data at unprecedented speed. LLMs are the key to automatic parsing of entities and relationships, building a general knowledge map, and enabling natural language query. Iterative exploratory analysis in a visualization environment such as GraphXR then provides the means to rapidly spotlight insights that would otherwise remain hidden.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=97a55be43ed3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/kineviz/ai-powered-knowledge-maps-for-navigating-unstructured-data-97a55be43ed3">AI-Powered Knowledge Maps for Navigating Unstructured Data</a> was originally published in <a href="https://medium.com/kineviz">Kineviz</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>