The Good AI

My intern is 14 years old and, as one would expect for their age, a bit of a rebel. What they are critical of is the use and overuse of generative artificial intelligence (genAI) tools, such as ChatGPT, because of how much energy they use. So when I told them that we would be working with AI, they were resisting the idea.

However, the semantic web and knowledge graphs are also part of AI, namely symbolic AI, and they form the backbone for data. When used correctly, they are actually very energy-efficient: running a query against a graph database takes little time and uses much less computing power than generating the same fact with genAI. Denny Vrandečić from the Wikimedia Foundation gave a fantastic keynote about this topic in 2023 and I made my intern watch it.

Building a Classical Music Knowledge Graph

With my intern appeased that they get to learn about the good AI, I went ahead and taught them about the vocabulary and preparation work that it takes to build a knowledge graph. Their task was to gather a list of orchestras and musicians to start building a Classical Music Knowledge Graph. I showed them how Wikidata can be used to get started and particularly how to use Wikidata’s SPARQL Query Service. What surprised me was that they did not struggle with modifying and altering Wikidata’s cat example and applying it to music. They were happily browsing Wikidata and picking up the URIs for subjects, predicates, and objects. I have seen capable data scientists struggle with a similar task. Not my teenager. Granted, they had some light programming experience, but I would expect data scientists to have that too.

One use case for a classical music knowledge graph is to find events around the world and use different entities, like performer, conductor, orchestra, composer, instrument, or location, as an entry poin tfor browsing through the events. In order to realise an aggregation service like that, one would need to have all concerts expressed semantically. I asked my intern to read through schema.org’s description of Event and use that format to collect a sample of concerts. For that task they had to browse through orchestra websites. What surprised me is how off-putting they found those websites to be. I have myself grown a few gray hairs trying to convince classical music organisations about the necessity of doing better in the digital realm, but seeing my intern’s emotional reaction to some very funky UX problems that is not so atypical for orchestra websites, made it clear to me that orchestras still struggle to improve their digital presence.

Finally I asked my intern to write a report about her week. They wrote it first in Finnish and then translated their own text by hand into English. Below is in their account of the internship, in their own words. It includes a very interesting remark on how Kurt Cobain and Jean Sibelius are presented on Wikidata. They asked that their name not be published, so their text will go below without a proper credit.

What did I do?

During my first day I first searched for information with simple questions from Wikidata, which is Wikimedias information archive website made in 2012. With these questions I had to figure out all classical music orchestras names and possible websites. I moved the information into a spreadsheet and checked that everything looked right, and that all the information was on its respective spot. The next day I did a spreadsheet of all the classical musicians. For that I searched every classical instrument player from Wikidata. At this point I opened Wikipedia’s article about orchestras for all the orchestra instruments as a reference. With the help of the Wikipedia article I found all the instruments and next searched for the orchestra musicians and their possible websites. There were hundreds even though I first wrote the questionnaire [sic query] wrong. On the third day I had a shorter day which I worked fully from home. During that day I searched for how Wikidata works, how you can contribute to it and how often it gets updated. On my fourth day I manually searched for concerts and presented them with the help of schema.org’s prompts [sic schema] about how you can write down tabels of information about certain objects or persons. Schema.org is a website on which you can see the description about things and their meaning that helps you input information. I also searched for information about people and groups that play in classical orchestras. On my last day I wrote this post.

Why is this information searching important?

I’m making a base for a crawler-bot that can later search for information based on the manually made excels. The crawler-bot goes through information and searches for answers to the questions asked from it. With the help of this information the bot can look through the websites and their information and use it. The bot looks through the excels [sic spreadsheets] and searches for information and answers for a website where you can search for any classical artist, group, orchestra, etc. and find information about their upcoming events, home websites, etc. If everything is put into the same column and not into for example into separate columns per every instrument, then this is more usable for the bot.

How did I search for information?

I used the Wikidata Query Service, which allowed me to search for long lists of people and their data with simple coding. The Wikidata Query Service is a way to search for information about the entire Wikidata with simple queries. On the fourth day, when I used Schema.org, I looked there for instructions on what information I should write down about events, people, and groups.

What did I learn?

During the intern week, I learned how to use Wikidata, how to fill in tabels so that bots can more easily go through them, how to manually retrieve information for bots, how and why people use GenAI more and more in their work, and how it can be used responsibly as a tool instead of using it as an answer to everything, and how tiring it is to search for information on orchestras’ websites.

Wikidata and how to find information there.

If you have the energy to learn Wikidata, it is very easy to find information there. However, not all important information is marked on the page of every person or thing. A good example of this is, for example, a problem I found when I was looking for Jean Sibelius in Wikidata.

Differences and similarities between Jean Sibelius’s and Kurt Cobain’s Wikidata pages.

The similarities between them are, for example, the fact that they are both musicians and write their own music, whether it is Kurt Cobain’s “Smells Like Teen Spirit” or Jean Sibelius’s “Finlandia”, Wikidata still counts both in the same category. Musicians. Both of them are also human and biological males, both also have their own songs, which makes it difficult for Wikidata to distinguish the two. Especially since only Kurt Cobain has been provided with the music genre he plays. Grunge and Rock. Jean Sibelius, on the other hand, has no genre recorded, so when looking for information, Wikidata sees similarities and puts both musicians in the same category.

The differences between the two well-known musicians include the country of birth and the languages they speak. Jean Sibelius is a Finnish-born Finland-Swede who also speaks Russian, while Kurt Cobain was born in America and speaks English. Jean Sibelius played the violin and piano in his works and acted as a conductor, while Kurt Cobain played the guitar and sang in his time. They also lived in different decades. Jean Sibelius was famously born in 1865 and died, about ten years before Kurt Cobain was born, in 1957. Kurt Cobain was born ten years after Jean Sibelius’s death in Washington in 1967 and died in 1994.

When you try to search for classical music artists in Wikidata, you can’t find Sibelius because he is not listed as a classical music artist. Then you have to look for all the musicians, including Cobain. Because of this, searching for information can become more difficult. On the other hand, if you try to look for classical musicians who were also conductors, Cobain disappears from the list, but so does Sibelius, even though he is clearly listed as a conductor and musician. This shows us that Wikidata is not perfect either.

What is the difference between making queries on Wikidata and generating it with generative AI?

Even answering a long survey [sic query] is faster for Wikidata than, for example, ChatGPT can generate it. You can also search for information from Wikidata manually, while GenAI only generates the information based on what its database has found online. Unnecessary generation also takes more natural resources compared to datatbase searches. From my point of view, doing simple things (even if they are tiring) manually is always a better option, while, for example, for coding help or going through impossibly long tabels, GenAI can be a pretty good tool.

How would I better the websites of orchestras?

During my intern week, I got to see how badly many orchestras’ websites have been made. This frustrates me a ton. Take, for example, the “Det Norska Kammerorkesteret” website. When you choose an English-language website, you will see different events than the Norwegian-language ones. It is also unnecessarily difficult to find information such as an e-mail or their address. If you even bother to make an English version of the website, it could be made more or less the same as the original and important information, such as the email address and the official address of the orchestra, could be made more easily visible.

This blog post is looking at the latter, which is best realized with the Resource Description Framework (RDF) standard. The part of that standard that enables the conceptual model are the global identifiers. In RDF knowledge graphs, every node and edge has a globally unique identifier within the context of that graph, broadly construed, commonly in the form of an Internationalised Resource Identifier (IRI) or Universal Resource Identifier (URI). These identifiers, which look like URLs, are essential for giving nodes and edges a consistent, recognizable name across systems and applications. This design allows knowledge graphs to have identifiers that aren’t tied to a single application or database but instead represent the concepts globally.

In my past experience designing and realizing knowledge graphs for enterprises, a well-designed knowledge graph should also enable any IRI to resolve into a human-readable webpage. This is key to usability. With resolvable IRIs, enterprise users can directly interact with and understand the information that these IRIs represent. The IRI and URI terms are often used interchangeably, as URIs are a subset of IRIs. See the references for more information.

Structuring IRIs with Domain-Based Namespaces

To ensure identifiers are unique and attributable, IRIs in a knowledge graph usually start with the domain name of the organization that owns them. For example, the domain could be https://knowledge.example-company.org/. Additionally the IRIs are organized into namespaces which is like a file structure within the domain. These folders, also called modules, can be things like business domains, themes, or customer-related topics. Their purpose is to group the IRIs in a way that reflects their purpose and scope within the graph. For example, https://knowledge.example-company.org/customers/ could be the namespace for any part of the graph that looks at customer-specific information. The module https://knowledge.example-company.org/products/ contains all product specifications.

Requirements for Identifiers

When designing identifiers, like IRIs, for a knowledge graph, it’s crucial to meet specific requirements to ensure they function effectively and support the goals of data integration, querying, and management. Here are the three main requirements for IRIs in a knowledge graph:

1. Readability and Recall

IRIs should be easily understandable and memorable at a glance. This is particularly important because IRIs are often manually referenced in SPARQL queries or other data interactions by engineers and analysts. If an IRI clearly reflects the concept or entity it represents, it can significantly improve the efficiency and accuracy of these manual tasks. Readable IRIs allow team members to intuitively understand the context of the data and reduce the cognitive load required to decipher complex queries. When IRIs are human-readable, it also facilitates debugging, maintenance, and the onboarding of new team members who need to quickly grasp the graph’s structure.

2. Reproducibility

For consistent, reliable integration, each IRI should be reproducible from the same input every time. This is especially crucial when data from non-graph sources, such as JSON or relational databases, is ingested and transformed into the graph. A reproducible IRI system ensures that given the same input data, the same IRI will always be generated. This consistency is essential for maintaining data integrity and preventing the creation of duplicate nodes for the same entity. Reproducibility also makes it easier to track and manage changes over time, as data can be reprocessed or updated without altering established links within the graph.

3. Uniqueness

Uniqueness is fundamental to the structure of a knowledge graph. Every IRI must uniquely identify a single node or edge to avoid ambiguity and ensure that each entity or relationship is distinct. Uniqueness is essential for the accuracy of data retrieval, as duplicate or overlapping identifiers can lead to incorrect data associations, confusing results, and potentially costly errors. By enforcing uniqueness, knowledge graph designers create a system where each node represents a one-to-one correspondence with a real-world concept or item, enabling precise querying and reliable insights.

Identifiers in Knowledge Graphs: Pros and Cons of Various Strategies

In creating effective IRIs, various strategies address the needs of knowledge graphs, each with unique strengths and limitations. Let’s examine a few common approaches:

UUIDs (Universally Unique Identifiers)

UUIDs offer strong uniqueness by generating identifiers that are practically guaranteed to be one-of-a-kind. However, they lack reproducibility because UUIDs can’t be recreated from the same input. This means that each UUID must be stored and looked up whenever needed, and UUIDs — made up of 32 characters and dashes — don’t offer inherent meaning, making them less suitable for human readability and recall.

Strengths: Uniqueness

Limitations: Not reproducible, low recall

Hash Values

Hash values, created using algorithms called hash functions like MD5 or SHA-1, generate unique, uniform, and opaque identifiers based on input data. This makes them reproducible from the same input, fulfilling both uniqueness and reproducibility requirements. However, their opaque structure does not convey meaning, reducing their usefulness for human recall.

Strengths: Uniqueness, reproducibility

Limitations: Low recall

Natural Text

Using natural text as identifiers — such as using the label “jaguar” for the animal or car brand — helps with readability and recall, especially in SPARQL queries. However, natural text is not universally unique, and labels can change over time, limiting reproducibility. To ensure uniqueness, natural text identifiers are often paired with namespaces (e.g., https://knowledge.example-company.org/animals/jaguar vs. https://knowledge.example-company.org/brands/jaguar).

Strengths: Uniqueness (within a namespace), recall

Limitations: Not always reproducible

External Identifiers

When integrating external data sources, converting their unique identifiers directly into IRIs provides a straightforward way to ensure reproducibility and uniqueness. This approach simplifies data transformation by linking nodes back to the original system identifier. However, these identifiers often don’t lend themselves to recall, as they may lack an intuitive or human-readable format.

Strengths: Uniqueness, reproducibility

Limitations: Low recall

IRI Decisions for the Enterprise Knowledge Graph

As seen from the various IRI creation strategies, no single approach meets all requirements. This is not a problem, because depending on the kinds of nodes different requirements are more applicable.

To make it concrete, below are some examples and an outline of node types within a typical enterprise knowledge graph that one could use for their IRIs. These nodes represent ontology classes, properties, internal taxonomy concepts, imported taxonomy concepts, and generated data graph nodes:

Ontology classes and properties, internal taxonomy concepts

The first three node types, ontology classes and properties as well as internally defined taxonomy concepts, are created manually, where identifiers are crafted for core concepts without pre-existing identifiers. These concepts are primarily used in SPARQL queries. They are not generated automatically since they only need to be created once and consistently referenced thereafter. The best strategy for these is using natural language text with adequate namespaces.

Examples:

https://knowledge.example-company.org/commerce-ontology/customer

https://knowledge.example-company.org/commerce-ontology/has_shopping_history

https://knowledge.example-company.org/color-taxonomy/salmon_pink

Imported Taxonomy Concepts

These are external taxonomies integrated directly into the enterprise’s knowledge graph. Their structure, identifiers, and labels remain unchanged to preserve reproducibility. Since they’re automatically generated, their readability primarily lies in their association with a specific taxonomy, making it easier to look them up. I have seen them being used as reference points for lateral connections managed in low-code environments. This means that their label and type is displayed and therefore don’t require recall from the IRI directly.

https://knowledge.example-company.org/pantone/14-1323-tpx

Generated Data Graph Nodes

These nodes, similar to imported taxonomies, may not always come with a predefined identifier. When there’s a unique combination of attributes, such as specific text fields, these attributes can be used as input for a hash function to generate a unique, hash-based IRI. Since these identifiers aren’t typically referenced in manual SPARQL queries, they don’t need to be easily memorable.

An example of this could be a product safety warning, such as a notification that highlights a risk. For instance, if a product poses a choking hazard, a consumer must be alerted to ensure safe usage. Many products could require such warnings, and each warning might carry the same title — “Choking Hazard” — but have a different description. This necessitates unique IRIs for each specific choking hazard scenario in the knowledge graph. A practical approach is to combine natural text with hash functions for these IRIs. For example, consider two different choking hazard warnings:

• Choking Hazard: Plastic Film. Remove all packaging before giving to children. Suffocation risk for children under 3 years.
• Choking Hazard: Small Parts. Not for children under 3 years.

By using a hash function like SHA-1, we can get the following identifiers for the above warnings:

• https://knowledge.example-company.org/product-safety/choking_hazard_170eb22e398f2aad774371e7aec8e35e28958161
• https://knowledge.example-company.org/product-safety/choking_hazard_33ebe1b2a642c6f8dee266bb3049a81583be1c97

The benefit of this approach is that a user can at first glance assess the contents of the node and see that there are a few for the risk of a choking hazard.

Final Words

There is no identifier strategy to rule them all, though UUIDs are the worst option. I hope that you found this blog post useful and got inspiration to tackle the important topic of enterprise-wide identifiers. I would love to hear your thoughts in the comments and experiences about defining a good identifier strategy to manage enterprise data.

References

RDF — https://www.w3.org/RDF/

URI — https://datatracker.ietf.org/doc/html/rfc3986

IRI — https://www.ietf.org/rfc/rfc3987.txt

The pyramid defines three layers, which are concepts, categories, and data. For example, in IKEA a concept would be a product, a category would be a bookcase, and one data point would be the BILLY bookcase in white 80x28x202 cm.

Technically, a KG consists of two layers: kind of things and individual things, this is analogous to classes and instances in object-oriented programming or to ideas and the real world from Platon’s Theory of Ideas. In this three-layered pyramid the middle layer, the categories, are technically individual things but are expressing information very close to that of kind of things, that is why it makes sense to distinguish them as a layer of their own.

In IKEA the top layer represents the business concepts central to what the company does. Typically, these concepts lie in the number of hundreds, and as Dave McComb illustrates in his book The Data-Centric Revolution, they are something one can take home with them over the weekend and learn by heart. If one is using the W3C stack for defining the KG, this part would be the ontology, so the definition for classes and properties. If one uses an LPG like Neo4J to build the KG, the concepts would be the node labels and relationship types.

Categories represent the controlled vocabulary or terminology used by IKEA and they lie in the number of thousands. Different experts in IKEA determine what the terminology is. Someone is responsible for defining all the ways in which we categorise our products: bookcase, sofa, coffee table, etc. In general, categorisation is different in every company and it depends on their use cases: some terminology is represented on the concept level, others on the category level, and some even on both levels. For example, in the music industry, an instrument would be a concept, where then the individual instruments, such as violin, piano, cello, etc., would be the categories. Now, a particular musical composition would be categorised by the key instruments that are needed to perform it, e.g. a piano trio or a violin concerto.

The third layer is the data layer. At IKEA these are all the products we are offering to our customers defined by their Swedish iconic name, their colour, and their size information. Our KG is customer-facing and it enhances the digital experience of IKEA. If we would use our KG for logistics, the data layer would distinguish between every single physical product, so each of the many BILLY bookcases in white of the size 80x28x202 cm sold around the world in the stores.

We use the distinction between these three layers to organise our work for building a KG. Since the concept layer is small and creates a company-overarching terminology it makes sense to define it manually with robust governance policies. Categories by definition need domain expertise and agreement so it is an internal collaboration including many subject matter experts. Finally, the data layer is so large it needs to be automated and developed in a test-driven way.

My friend and colleague in the semantic web community, Juan Sequeda, pointed out that one theme that seems to be repeating when companies are forming data strategies is that of centralisation and de-centralisation. Inspired by this I ventured to look at how the three-layered pyramid is authored, owned, and stored in an enterprise based on my applied experience in building enterprise knowledge graphs.

Authoring, Ownership, and Storage

Concepts are authored centrally by the business concept owning team, which in practice is a team of ontologists, those who translate human concepts into a machine-readable format. The concepts need to form a logically coherent structure because they are used to create powerful queries over the data layer and for reasoning. Since their number lies in the hundreds it makes sense to define them manually and centrally.

Categories are best authored in a decentralised manner because the subject matter experts of the enterprise are the ones who decide which terminology should be used and they are scattered around the company. Their number lies in the thousands so dividing the work between expert teams helps to manage the workload.

The data layer is a combination of one or many data sources with references to the category layer, thus its ownership should remain at the data source as also stated by the data mesh idea of data as a product.

At IKEA we store the concepts and categories in a git repository. This way we can tag releases, use versioning, review them line-by-line, and automatically trigger quality assurance tests via git actions.

Depending on performance and usage needs the contents of the data layer can be stored in two following ways: it can be persisted in a database or virtualised by storing instructions on how to access the original data source. Virtualising the data layer means it can be stored in a decentralised manner and accessed at its point of origin. Persisting the data layer might lead to optimised performance, however, if the data ownership is not clear and enforced, this leads to the problem of multiple deferring copies of the same data, which leads to inconsistency in data, the current data problem in most organisations. We have chosen to persist our data layer and keep it updated with respect to its upstream.

Envisioning our KG on three layers is not only useful for organising our work but also for communicating its contents to non-technical stakeholders. While they might not grasp all the technical implications the layers provide, it helps them to focus on what kind of content they need to provide to make the KG successful. It also manages their expectations on what the KG can provide them in return.

IKEA’s Knowledge Graph and Why It Has Three Layers was originally published in Flat Pack Tech on Medium, where people are continuing the conversation by highlighting and responding to this story.