<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Kumo AI on Medium]]></title>
        <description><![CDATA[Stories by Kumo AI on Medium]]></description>
        <link>https://medium.com/@kumoai?source=rss-8342d627d5b2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*8uA2yjK2FDtaAtjlh_2fRw.png</url>
            <title>Stories by Kumo AI on Medium</title>
            <link>https://medium.com/@kumoai?source=rss-8342d627d5b2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 06:57:28 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@kumoai/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Do Large Language Models make accurate personalized recommendations?]]></title>
            <link>https://medium.com/@kumoai/do-large-language-models-make-accurate-personalized-recommendations-0b508ff1ee3d?source=rss-8342d627d5b2------2</link>
            <guid isPermaLink="false">https://medium.com/p/0b508ff1ee3d</guid>
            <category><![CDATA[gloves]]></category>
            <category><![CDATA[graph-transformer]]></category>
            <category><![CDATA[hugging-face]]></category>
            <dc:creator><![CDATA[Kumo AI]]></dc:creator>
            <pubDate>Thu, 03 Oct 2024 16:19:43 GMT</pubDate>
            <atom:updated>2024-10-03T16:19:43.690Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Do Large Language Models make accurate personalized recommendations?" src="https://cdn-images-1.medium.com/max/760/0*WELRLn8PXVAlzfAB.png" /></figure><h4><em>Adding Large Language Models to Kumo’s Graph Transformer for Improved Recommendations</em></h4><p>by <a href="https://www.linkedin.com/in/zechengzhang/">Zecheng Zhang</a>, <a href="https://www.linkedin.com/in/alankrumholz/">Alan Krumholz</a>, <a href="https://www.linkedin.com/in/vid-kocijan-38575321b/overlay/about-this-profile/">Vid Kocijan</a>, <a href="https://rusty1s.github.io/#/">Matthias Fey</a>, <a href="https://www.linkedin.com/in/leskovec/">Jure Leskovec</a></p><p><a href="http://kumo.ai/">Kumo</a> is an AI for quickly building highly accurate predictive models using the entirety of the data warehouse: multiple tables, both structured and unstructured. Kumo uses embedding tuning to uniquely combine pre-trained large language models (LLMs) with graph transformer architectures to improve accuracy by double-digits in days. Kumo’s models can be refined by domain experts for maximum performance improvements.</p><p>Recently the data science and engineering teams at Kumo AI ran several experiments with a product recommendation task to see how different LLMs improve recommendation accuracy. The task is to predict the products a customer is going to buy in the next 7 days.</p><p>For the AI model to produce accurate recommendations it needs a detailed understanding of product properties as well as customer preferences. Products as well as customers are often associated with unstructured textual information like product names, product descriptions, customer reviews, and more. It is critical for the recommender system to include text understanding capability because so much information is stored in unstructured text.<br><br>At the same time, graph information is highly valuable in recommender systems because it captures complex relationships between customers, products, and their interactions. Graphs represent the relationships between customers and products as edges, allowing the system to consider not just direct interactions (like purchases or ratings) but also indirect connections (e.g., customers who like similar products or products that are liked by similar customers).</p><p>Furthermore, in some recommender systems, social relationships between customers (e.g., friendships) are also represented as graphs, which can enhance recommendations by incorporating preferences from a customer’s social circle. In graph-based systems, higher-order connections (i.e., multi-hop relationships) can be leveraged.</p><p>For instance, if Customer A likes a product that Customer B liked, and Customer C is similar to Customer B, the system might recommend that product to Customer C as well. Overall, graph-based models provide a nuanced understanding of customers’ preferences by considering both direct and indirect interactions. This enables highly personalized recommendations, taking into account more contextual information about the customer and their network.</p><p>The Kumo team experimented with four variations of how textual information can be processed on the same recommendation task:</p><p>- <strong>LLM-only:</strong> OpenAI text-embedding-3-large alone<br>- <strong>Kumo-only:</strong> Kumo alone (uses GloVe for embeddings)<br>- <strong>Kumo+HuggingFace:</strong> Kumo + Hugging Face intfloat/e5-base-v2 LLM <br>- <strong>Kumo+OpenAI:</strong> Kumo + OpenAI text-embedding-3-large LLM</p><p>Surprisingly, results show that Large Language Models by themselves work very poorly at personalized recommendations. Even though LLM is used to produce an embedding based on all the product information and characteristics, such models fail to capture nuanced customer behavior in its embeddings. A graph-based approach, like Kumo, captures customer behavior much better and improves the recommendation accuracy over the LLM by 15x (MAP@12). Combining Kumo with LLM-based encoding of textual fields achieves <strong>the highest performance, ranging from 4% to ~11% improvements over the graph-only approach</strong>. Interestingly, we find that Kumo+OpenAI LLM slightly but consistently beats the Kumo+HuggingFace LLM.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/760/0*SKeXBogHG3i68kfK.png" /></figure><h4>What are encoder-based LLMs?</h4><p>Encoder-based LLMs are a type of language model that utilize encoder architecture to extract relevant textual information from text data and generate continuous vector representations, known as embeddings. From <a href="https://arxiv.org/abs/1810.04805">BERT</a> to more recent <a href="https://arxiv.org/abs/1908.10084">sentence transformers</a>, encoder-based large language models have progressively achieved state-of-the-art vector representations for textual data.</p><p>These models efficiently encapsulate not only semantic meaning but also context-sensitive information within the embeddings. The embeddings produced can be directly applied to various downstream tasks, including semantic search, sentiment analysis, and recommendation, without the need to finetune the underlying language model.</p><p>These textual embeddings can then be directly used to identify related products. More importantly, Kumo can automatically combine textual embeddings produced by these encoder-based LLMs with the graph-based information to produce even more accurate recommendations.</p><h4>How does Kumo work with embeddings generated by encoder-based LLMs?</h4><p>In this experiment, Kumo brings together two technologies: graph learning via graph transformers and natural language understanding via encoder-based LLMs.</p><p>Kumo supports a <strong>Sequence</strong> data type that can directly utilize encoder-based LLMs text vector representations (i.e. embeddings), as feature inputs for Kumo’s graph neural network (graph transformer). The Sequence data type allows Kumo’s graph transformer to coalesce text embeddings with various other data types, such as numerical, timestamp, and multi-categorical, across different tables. Encapsulating both semantic meaning and context-sensitive textual information through text embeddings enhances Kumo’s ability to make accurate predictions from structured as well as unstructured relational data.</p><h4>H&amp;M Dataset and Kumo’s graph for personalized recommendations experiments:</h4><p><a href="https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations">H&amp;M Personalized Fashion Recommendations</a> is a dataset for product (articles) recommendation for customers based on data from previous transactions provided publicly by H&amp;M for data science experiments. The dataset contains three tables: <strong>customers</strong>, <strong>transactions</strong> and <strong>articles</strong> which were linked in a Kumo graph using primary (PK) and foreign key (FK) relationships.</p><p><strong>The goal</strong> is to predict/recommend the top 12 articles that each customer is likely to buy in the next 7 days.</p><p>H&amp;M tables linked in a Kumo graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*ZPRiYMeT4iTmMyFP" /></figure><p>Sample of text columns in the <strong>articles</strong> table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*TWvGvGAiYqjNTJ1p" /></figure><p>Samples of categorical columns in the <strong>articles</strong> table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*ea8nruNbnxKkKAk1" /></figure><p>Experiments<br>LLM-only: OpenAI text-embedding-3-large</p><p>Our LLM-only recommender system relies solely on OpenAI LLM embeddings. To generate personalized recommendations for each customer, customer embeddings and embeddings of articles must be computed first.</p><p>We compute product embeddings by concatenating all the product information into a single long sentence (For example: “Product name: . Product description: . Color: . Material: , …”). We then compute the customer embedding as the average of the embeddings of products that the customer has purchased in the past.</p><p>To produce recommendations we take an embedding of a given customer and identify 12 products whose embeddings have the highest cosine similarities with the embedding of the customer.</p><p>Kumo-only (Kumo with GloVe for embeddings)</p><p>Kumo uses the Customer-Transaction-Articles schema to build a heterogeneous temporal graph where each customer is connected to her transactions and each transaction is connected to the corresponding product. Because product nodes have textual information attached to them, Kumo uses <a href="https://aclanthology.org/D14-1162/">GloVe</a> to encode textual information — Kumo averages word-level GloVe embeddings of all the words in a given column.</p><p>Kumo then uses the following predictive query to create the prediction for each customer:</p><p>PREDICT LIST_DISTINCT(TRANSACTIONS.ARTICLE_ID, 0, 7, days) RANK TOP 12<br>FOR EACH CUSTOMERS.CUSTOMER_ID</p><p>Kumo automatically learns useful signals from <strong>Customers</strong>, <strong>Transactions</strong>, and <strong>Articles</strong> tables to make recommendations. In addition to the structured data in the tables, there is useful information included in the unstructured text and categorical columns, such as shape (<em>round</em>), material (<em>cotton</em>), and type (<em>trainer</em>).</p><p>Kumo + HuggingFace (intfloat/e5-base-v2 LLM)</p><p>In the next experiment, the team then combined Kumo’s predictive approach with the <a href="https://huggingface.co/intfloat/e5-base-v2">intfloat/e5-base-v2</a> model, an open source encoder-based large language model, to pre-embed text and string categorical columns in the <strong>articles</strong> table. In general, all semantically meaningful columns, such as the text columns prod_name and detail_desc are concatenated into rows of strings. The text encoder encodes concatenated strings into text embeddings. The same predictive query is then trained with the LLM as the text embedding encoder in the <strong>articles</strong> table.</p><p>Kumo + OpenAI (text-embedding-3-large LLM)</p><p>Finally, the team combined Kumo with <a href="https://openai.com/index/new-embedding-models-and-api-updates">OpenAI text-embedding-3-large</a> with 1024 output dimensions to represent each article by textifying semantically meaningful columns. The same predictive query was trained with OpenAI’s LLM as an encoder of the textual columns in the articles table.</p><p>Kumo’s integration with different LLMs.</p><h4>Results of encoder-based LLM experiments for personalized recommendations</h4><p><strong>MAP@12PRECISION@12RECALL@12F1@12LLM-only:</strong><br>OpenAI text-embedding-3-large 0.00190<br>(-93.33%)0.00329<br>(-67.84%)0.00119<br>(-97.73%)0.0071<br>(-54.60%)<strong>Kumo-only:<br></strong>uses GloVe for text embeddings0.028560.010230.052340.01564<strong>Kumo+HuggingFace</strong>:<br>uses intfloat/e5-base-v2 for text embeddings0.0297<br>(+4.00%)0.01099<br>(+7.43%)0.05531<br>(+5.67%)0.01673<br>(+6.97%)<strong>Kumo+OpenAI:</strong><br>uses text-embedding-3-large for text embeddings0.02976<br>(+4.20%)0.01139<br>(11.34%)0.0567<br>(+8.33%)0.0173<br>(+10.61%)</p><h4>Conclusion</h4><p>From these experiments we can draw several important conclusions:</p><p>- Large Language Models alone perform very poorly in product recommendation and personalization tasks. In fact, depending on the metric LLMs by themselves perform between 2x (F1@12) to 40x worse (RECALL@12) than graph-based approaches.<br>- Graph-based approaches, like Kumo, by themselves give excellent results, heavily improving over the LLM baseline (15x in MAP@12).<br>- Better text embeddings provided by modern LLMs further improve Kumo’s performance by 4% to 11%, depending on the metric.</p><p>Overall, the performance of Kumo’s GNN models can be significantly improved by combining its out of the box predictions with encoder-based large language model text embeddings. Keeping up-to-date with advancements in LLMs will help Kumo customers create the most accurate predictions possible.</p><p>Metrics</p><p>Four different metrics were used to evaluate performance:</p><p>- MAP@12: Mean of the average precision scores, where average precision is the average of precision values calculated at the ranks where relevant items are found, up to the top 12 items.<br>- Precision@12: Proportion of relevant items among the top 12 recommended items.<br>- Recall@12: Proportion of relevant items that have been retrieved in the top 12 recommended items out of all relevant items.<br>- F1@12: The harmonic mean of Precision@12 and Recall@12, which provides a balance between precision and recall.<br>Acknowledgements:</p><p><a href="https://www.linkedin.com/in/wang-dong-69b8771a/">Dong Wang</a>, <a href="https://www.linkedin.com/in/mananshah99/">Manan Shah</a>, <a href="https://www.linkedin.com/in/myunghwankim/">Myunghwan Kim</a>, <a href="https://www.linkedin.com/in/akihironitta/">Akihiro Nitta</a>, <a href="https://www.linkedin.com/in/siyang-xie/">Siyang Xie</a></p><p>Next step: <a href="https://kumo.ai/free-trial/">request a trial</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0b508ff1ee3d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kumo AI + Databricks: Revolutionizing Predictive Modeling with GNNs and LLMs]]></title>
            <link>https://medium.com/@kumoai/kumo-ai-databricks-revolutionizing-predictive-modeling-with-gnns-and-llms-1ac291821d01?source=rss-8342d627d5b2------2</link>
            <guid isPermaLink="false">https://medium.com/p/1ac291821d01</guid>
            <category><![CDATA[gnn]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[databricks]]></category>
            <dc:creator><![CDATA[Kumo AI]]></dc:creator>
            <pubDate>Mon, 23 Sep 2024 20:18:52 GMT</pubDate>
            <atom:updated>2024-09-23T20:18:52.196Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Kumo AI + Databricks: Revolutionizing Predictive Modeling with GNNs and LLMs" src="https://cdn-images-1.medium.com/max/1024/0*RaINujFqFdA4bBLz.png" /></figure><p>By <a href="https://www.linkedin.com/in/hema-raghavan-1581075/">Hema Raghavan</a></p><p>In today’s data-driven world, businesses need cutting-edge tools to handle vast datasets and extract actionable insights quickly and efficiently. Databricks is at the forefront of this transformation, empowering organizations to integrate complex AI models, manage diverse data types, and ensure real-time processing. It helps businesses maintain high model accuracy while managing large datasets, ultimately enhancing productivity and decision-making.</p><p><a href="http://kumo.ai/">Kumo AI</a>, through seamless integration with <a href="https://www.databricks.com/product/unity-catalog">Databricks Unity Catalog</a>, accelerates predictive model generation to produce accurate and actionable insights for various business applications.</p><h4>Kumo AI: Revolutionizing Predictive Modeling</h4><p>Kumo AI combines advanced technologies like graph neural networks (GNNs) and large language models (LLMs) to analyze and predict customer behaviors, lifetime value, and other key business metrics. Kumo works directly with raw data from Databricks without needing a fixed schema or complex feature engineering. By representing each row of data as a node and connecting primary-foreign keys, we create a graph structure that allows for efficient analysis and prediction.</p><figure><img alt="simple diagram of Kumo and Databricks using GNNs and LLMs to create predictive models" src="https://cdn-images-1.medium.com/max/1024/0*CzV8pwbAqBIKvIP0.png" /></figure><p>Using GNNs, Kumo can learn from multiple Databricks tables stored within the Unity Catalog, producing highly accurate predictions. Kumo takes this further by leveraging LLMs such as <a href="https://huggingface.co/docs/transformers/en/model_doc/dbrx">DBRX</a>, <a href="https://huggingface.co/">HuggingFace</a>, and <a href="https://openai.com/">OpenAI</a> to incorporate general-world knowledge into its models. The combination of GNN and LLMs delivers a more complete understanding of the data, enhancing the quality and precision of predictions.</p><p>In practical terms, Kumo offers out-of-the-box models that are typically up to 30% more accurate than baseline models. Domain experts can refine these models further to meet specific business needs, providing flexibility and customization. Within just a few hours, Kumo can generate batch predictions or embeddings that can be used for downstream tasks.</p><h4>Customer Success with Kumo AI</h4><p>A great example of Kumo’s impact comes from Databricks itself. Chris Klaczynski, Staff Data Scientist, and Anoop Muraleedharan, Senior Director for Marketing Technology and Analytics at Databricks, shared their experience using Kumo AI to improve lead scoring accuracy. The result? A <strong>5.4x</strong> increase in prediction accuracy over their previous solution. With Kumo, Databricks’ sellers can focus on leads most likely to convert, driving efficiency and improving sales outcomes.</p><h4>Predictive AI Workflow: Kumo + Databricks</h4><p>Kumo operates directly on raw Databricks tables while utilizing Unity Catalog for data governance and security. This ensures that data never leaves the Databricks Data Intelligence Platform without being transformed, encoded, and deleted after use. This secure and scalable approach allows businesses to generate predictions reliably and at scale without compromising on data security.</p><p>By operating directly on Databricks Unity Catalog, Kumo harnesses features to create predictive models from both structured and unstructured data. This approach ensures accuracy and scalability, enabling organizations to work with terabyte-sized tables without having to sample or reduce their data.</p><p>Kumo learns from the latest data in real-time or on your chosen schedule, ensuring predictions are always based on up-to-date information. The ability to serve both real-time model inferences and batch predictions ensures businesses can get the insights they need, when they need them.</p><h4>The Benefits of Kumo + Databricks Integration</h4><p><strong>Deliver More Predictive AI: </strong>Kumo makes predictive AI easier than ever. By connecting your Databricks Unity Catalog tables, you can generate numerous predictions across various business use cases. Kumo’s automated machine learning pipelines keep models updated with the latest data, ensuring predictions remain accurate over time.</p><p><strong>Improve Model Performance: </strong>With Databricks Model Serving and support for LLMs, Kumo’s GNN models can learn from both structured and unstructured data. This boosts model accuracy, allowing data scientists to deliver even more precise predictions.</p><p><strong>Secure Data Processing</strong>: Kumo prioritizes data security. By keeping all data within the Databricks Unity Catalog, Kumo ensures that no sensitive data leaves your environment. The use of advanced encoding methods protects your data and models throughout the machine learning lifecycle.</p><h4>Unlock Insights with Deep Learning and Explainable AI</h4><p>Traditional methods often fall short when trying to extract value from relational data stored in Databricks Unity Catalog. Kumo AI overcomes these limitations by using graph machine learning on raw data tables. Kumo’s GNN technology capitalizes on the relational structure of Databricks tables, incorporating additional context from LLM models supported by Databricks.</p><p>This combination of GNN and LLM models unlock the full potential of relational and unstructured data, significantly improving the performance and accuracy of predictive AI models. Moreover, Kumo provides explainable AI features that offer deep insights into how predictions are made, enabling businesses to understand the impact of different variables on the final output.</p><figure><img alt="Detailed architecture diagram of how Kumo and Databricks work together to create predictions from the data warehouse." src="https://cdn-images-1.medium.com/max/1024/0*GkizKVSbnlbTCXJF.png" /></figure><h4>Conclusion</h4><p>Kumo AI, in collaboration with Databricks, is transforming how businesses approach predictive modeling. Its unique combination of GNNs and LLMs allows for highly accurate predictions across vast datasets, without the need for complex feature engineering or schema design. By keeping data secure within the Databricks environment, Kumo ensures reliable and compliant data processing, making it a powerful tool for businesses looking to drive better insights and outcomes from their data.</p><p>With its scalability, security, and advanced AI capabilities, Kumo AI is shaping the future of predictive analytics, making it easier for businesses to extract actionable insights and improve decision-making at every level.</p><p>Start a <a href="https://kumo.ai/free-trial/">free trial</a> to join the private preview for Kumo + Databricks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1ac291821d01" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build a Personalized Chatbot Powered by Kumo AI Embeddings and Predictions]]></title>
            <link>https://medium.com/@kumoai/build-a-personalized-chatbot-powered-by-kumo-ai-embeddings-and-predictions-c83ccee599d3?source=rss-8342d627d5b2------2</link>
            <guid isPermaLink="false">https://medium.com/p/c83ccee599d3</guid>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[demo]]></category>
            <category><![CDATA[embedding]]></category>
            <dc:creator><![CDATA[Kumo AI]]></dc:creator>
            <pubDate>Fri, 20 Sep 2024 17:36:53 GMT</pubDate>
            <atom:updated>2024-09-20T17:36:53.272Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Build a Personalized Chatbot Powered by Kumo AI Embeddings and Predictions" src="https://cdn-images-1.medium.com/max/1024/0*S4i-TSgFOsxuM0fH.png" /></figure><p>by <a href="http://linkedin.com/in/naisha-agarwal/">Naisha Agarwal</a></p><p>Follow along in my video and learn more about how Kumo AI can power chatbots to improve predictions.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FrOSuUCcvWrk%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DrOSuUCcvWrk&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FrOSuUCcvWrk%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/8c157b9a29fa92a0d0e7357778709849/href">https://medium.com/media/8c157b9a29fa92a0d0e7357778709849/href</a></iframe><h3>Motivation</h3><p>Imagine you are a user entering a store, looking for items to purchase. Having a personal assistant that can give you customized real-time recommendations for how to find the best item given their past purchase history and constraints can be extremely beneficial to a user’s shopping experience.</p><p>The queries a user can ask an agent of this kind can be divided into two main categories:</p><p>- <strong>ItemToUser</strong><br>- <strong>ItemToItem</strong></p><p>An itemtouser query is the one that we are all most familiar with; what items should I buy given my personal purchase history? These are tailored queries for a specific customer, suited to their preferences. An itemtoitem query is not specific to any one customer; these are more generic queries about what items go well with each other, what items complement each other, etc. Having a chatbot that can answer both of these types of queries can enhance the user experience significantly.</p><h3>Overview</h3><p>There are several steps in order to build a chatbot like this:</p><p>- Obtaining <a href="https://kumo.ai/">Kumo AI</a> predictions and embeddings<br>- Storing data in a database<br>- Understanding what type of context to use for different types of queries<br>- Using a RAG approach to pass in appropriate context into a LLM and obtain results for specified queries<br>- User Experience/Product<br>- Setting up an Evaluation Framework</p><p>This blog will be going into depth about each of these sections.</p><h4>1. Obtaining Kumo Predictions and Embeddings</h4><p>Our chatbot can answer two types of queries: itemtoitem and itemtouser. Let us look at each of these cases separately.</p><p>ItemToUser</p><p>For an itemtouser usecase, we want to obtain the top k item predictions for every customer. Kumo makes this very easy to obtain.</p><p>We use the <a href="https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations">publicly available H&amp;M dataset </a>. The original dataset has 3 datasets:</p><p>- Articles (information about the store inventory)<br>- Customers (information about the store’s users)<br>- Transactions (list of the different purchases made at H &amp; M)</p><p>These three datasets serve as the tables we can use to construct our Kumo graph. We link the article_id and customer_id from the articles and customers table respectively to their foreign_keys in the transactions table as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/423/0*tka3-qwezGB62SfO.jpg" /></figure><p>With this graph, we can run the following <a href="https://docs.kumo.ai/docs/predictive-query">PQuery</a>:</p><p>PREDICT LIST_DISTINCT(transactions.article_id, 0, 30, days)<br>RANK TOP 25<br>FOR EACH customers.customer_id</p><p>This PQuery obtains the top 25 items for every customer ID over the next 30 days. Once training the Kumo model using this PQuery, we can run a batch prediction workflow, and obtain two main pieces of data:</p><p>- <strong>Predictions: </strong>The top 25 predictions for every customer id in our dataset<br>- <strong>Embeddings: </strong>Embeddings for every customer ID, as well as an embedding for every item in the database<br>ItemToItem</p><p>For the itemtoitem use case, our setup looks a bit different. We have two articles tables, duplicates of each other. In this use case, we can think of the nodes in our constructed graph as the items, and an edge between any of the nodes as a co-occurence relationship. Both of these articles table then link to a transactions table, that we can think of a co-occurence table. The graph can be seen below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/423/0*yn87gVnoVxrx42U7.jpg" /></figure><p>We can run the following PQuery:</p><p>PREDICT LIST_DISTINCT(transactions.article_id2, 0, 30)<br>RANK TOP 10<br>FOR EACH articles.article_id</p><p>Once training the Kumo model using this PQuery, we can run a batch prediction workflow, and obtain the following pieces of data:</p><p>- <strong>Predictions: </strong>The top 10 predictions for every item in the dataset<br>- <strong>Embeddings: </strong>The embeddings for both the left hand side articles table, and the right hand side articles table</p><p>After running both of these batch prediction workflows, we now have the predictions and embeddings for both the itemtouser and itemtoitem use cases.</p><h4>2. Storing Data in Database</h4><p>Now that we have the Kumo predictions and embeddings, our next step is to store the data in a database in a format that is easily searchable. I will be using Elasticsearch to store the data.</p><p>Elasticsearch is structured into numerous indices, each with their own specific index mapping. For this application, there are a total of 8 indices in our database: item information, customer information, embedding indices (x4), and prediction indices (x2). Let us investigate each of these indexes further.</p><p>Item Information Index (#1)</p><p>This is an index for storing basic information about every item, taken directly from the H &amp; M dataset. Here, instead of storing each attribute as it is stored in the original dataset, I combined attributes into one field that is easily searchable. This way, users can also query by item name or any other item attribute instead of having to specify the item id in the query.</p><p>For instance, consider the following item from the original dataset:</p><p><em>{“</em><strong><em>article_id</em></strong><em>”: 108775015, “</em><strong><em>prod_name</em></strong><em>”:”Strap top”, “</em><strong><em>product_type_no</em></strong><em>”:253, “</em><strong><em>product_type_name</em></strong><em>”:”Vest top”, “</em><strong><em>product_group_name</em></strong><em>”: “Garment Upper body”, “</em><strong><em>graphical_appearance_no</em></strong><em>”:1010016, “</em><strong><em>graphical_appearance_name</em></strong><em>”:”Solid”, “</em><strong><em>colour_group_code</em></strong><em>”:9, “</em><strong><em>colour_group_name</em></strong><em>”:”Black”, “</em><strong><em>perceived_colour_value_id</em></strong><em>”:4, “</em><strong><em>perceived_colour_value_name</em></strong><em>”:”Dark”, “</em><strong><em>perceived_colour_master_id</em></strong><em>”:5, “</em><strong><em>perceived_colour_master_name</em></strong><em>”:”Black”, “</em><strong><em>department_no</em></strong><em>”:1676, “</em><strong><em>department_name</em></strong><em>”:”Jersey Basic”, “</em><strong><em>index_code</em></strong><em>”:”A”, “</em><strong><em>index_name</em></strong><em>”:”Ladieswear”, “</em><strong><em>index_group_no</em></strong><em>”:1, “</em><strong><em>index_group_name</em></strong><em>”:”Ladieswear”, “</em><strong><em>section_no</em></strong><em>”:16, “</em><strong><em>section_name</em></strong><em>”:”Womens Everyday Basics”, “</em><strong><em>garment_group_no</em></strong><em>”:1002, “</em><strong><em>garment_group_name</em></strong><em>”:”Jersey Basic”}</em></p><p>When storing this data into Elasticsearch, we simplify this item into simply two columns:</p><p><em>{“</em><strong><em>item_id”: </em></strong><em>108775015, </em><strong><em>“item_info”: </em></strong><em>Item </em><strong><em>108775015</em></strong><em> is called ‘</em><strong><em>Strap top</em></strong><em>’. It is a </em><strong><em>Vest top</em></strong><em> (product type number </em><strong><em>253</em></strong><em>) under the </em><strong><em>Garment Upper body </em></strong><em>product group. This item has a graphical appearance of </em><strong><em>Solid</em></strong><em> (appearance number </em><strong><em>1010016</em></strong><em>) and belongs to the </em><strong><em>Black</em></strong><em> colour group (colour group code</em><strong><em> 9</em></strong><em>). It is perceived as </em><strong><em>Dark </em></strong><em>(perceived colour value ID </em><strong><em>4</em></strong><em>) and categorized under the </em><strong><em>Black master colou</em></strong><em>r (master colour ID </em><strong><em>5</em></strong><em>). The item is in the </em><strong><em>Jersey Basic department </em></strong><em>(department number </em><strong><em>1676</em></strong><em>) of </em><strong><em>Ladieswear </em></strong><em>(index code</em><strong><em> A</em></strong><em>). Specifically, it is in the </em><strong><em>Womens Everyday Basics section</em></strong><em> (section number </em><strong><em>16</em></strong><em>). The garment group is </em><strong><em>Jersey Basic</em></strong><em> (garment group number </em><strong><em>1002</em></strong><em>).”}</em></p><p>Customer Information Index (Index #2)</p><p>This is an index for storing basic information about every customer, taken directly from the H &amp;M dataset. Similarly to the item information index, instead of storing every single attribute as a single field, I combined these attributes into a singular <strong>customer_info</strong> field.</p><p>For instance, given the following customer data:</p><p><em>{“</em><strong><em>customer_id</em></strong><em>”:”00000dbacae5abe5e23885899a1fa44253a17956c6d1c3d25f88aa139fdfc657&quot;,” </em><strong><em>FN</em></strong><em>”:null, “</em><strong><em>active</em></strong><em>”:null, “</em><strong><em>club_member_status</em></strong><em>”:”ACTIVE”, “</em><strong><em>fashion_news_frequency</em></strong><em>”:”NONE”, “</em><strong><em>age</em></strong><em>”:49, “</em><strong><em>postal_code</em></strong><em>”:”52043ee2162cf5aa7ee79974281641c6f11a68d276429a91f8ca0d4b6efa8100&quot;}</em></p><p>we store this in Elasticsearch with two columns:</p><p><em>{“</em><strong><em>customer_id”: </em></strong><em>00000dbacae5abe5e23885899a1fa44253a17956c6d1c3d25f88aa139fdfc657</em><strong><em>, “customer_info”: </em></strong><em>Customer with ID </em><strong><em>00000dbacae5abe5e23885899a1fa44253a17956c6d1c3d25f88aa139fdfc657</em></strong><em> is a</em><strong><em> ACTIVE </em></strong><em>member. They are </em><strong><em>49.0</em></strong><em> years old. Their fashion news frequency is set to </em><strong><em>‘NONE’</em></strong><em>. Their postal code is </em><strong><em>‘52043ee2162cf5aa7ee79974281641c6f11a68d276429a91f8ca0d4b6efa8100’</em></strong><em>. Their active status is</em><strong><em> nan</em></strong><em> and FN status is</em><strong><em> nan</em></strong><em>.}</em></p><p>Storing the data in this format into Elasticsearch makes it easily searchable, and allows for users to use other customer attributes when searching.</p><p>Embedding Indices (Indices 3–6)</p><p>To store the embeddings for itemtouser, we will define two separate indices: one for customer embeddings and another for the item embeddings.</p><p>Below are the structure of these two indices:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/0*gFz_ZU2uW5cBjLra.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/912/0*hNVilgptsXLAaNbg.png" /></figure><p>Both indices have the same structure of ID (customer or item), the embedding, and a text field describing either the customer or the item.</p><p>A similar structure is used for the itemtoitem embedding indices, where both embedding indices in this case are of the same structure as the cust_item_embeddings above.</p><p>Prediction Indices (7–8)</p><p>For both itemtouser and itemtoitem, we store two indices: predictions for every customer, and predictions for every item.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/0*qQibcNCYnMkuk0Bb.png" /></figure><p>Item Predictions</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/0*kbsbxvzcq2t-MpkW.png" /></figure><p>Customer Predictions</p><p>Similar to the item and customer information indices, predictions for every item/customer are formatted into one singular text field so that all predictions can be searched easily for any given item.</p><p><em>{“</em><strong><em>item_id”: </em></strong><em>108775015, </em><strong><em>“formatted_predictions”: “</em></strong><em>The 1st prediction for item_id 108775015 is item_id 108775015 with score 317.7856140136719. Item 108775015 is called ‘Strap top’… The 2nd prediction for item_id 108775015 is item_id 815456006 with score 238.77734375. Item 815456006 is called ‘Madison Slim Stretch Chino’… The 10th prediction for item_id 108775015 is…”</em></p><p><em>{“</em><strong><em>customer_id”:</em></strong><em> 00000dbacae5abe5e23885899a1fa44253a17956c6d1c3d25f88aa139fdfc657, </em><strong><em>“formatted_predictions”: “</em></strong><em>The 1st prediction for customer_id 00000dbacae5abe5e23885899a1fa44253a17956c6d1c3d25f88aa139fdfc657 is item_id 108775015 with score 317.7856140136719. Item 108775015 is called ‘Strap top’… The 2nd prediction for customer_id 00000dbacae5abe5e23885899a1fa44253a17956c6d1c3d25f88aa139fdfc657 is item_id 815456006 with score 238.77734375. Item 815456006 is called ‘Madison Slim Stretch Chino’… The 10th prediction for customer_id 00000dbacae5abe5e23885899a1fa44253a17956c6d1c3d25f88aa139fdfc657 is..</em></p><h4>3. Context Retrieval</h4><p>Once data is structured in Elasticsearch, we have to figure out an appropriate way to retrieve context given a user query, and display a final response. The process can be summarized as the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*G5QioNh56BbLLRs1.png" /></figure><p>Classification Response (<a href="https://openai.com/">OpenAI</a>)</p><p>The first step in the workflow is a classification response of the user query, either identifying it as an itemtoitem or an itemtouser prompt. This will help specify what indices to search for in Elasticsearch for relevant context for the user query.</p><p>This classification response is obtained by simply prompting GPT 4-o with the prompt “<em>Please classify the following query as either an itemtoitem task or a itemtouser item task. Please only respond with either itemtoitem or itemtouser, with no additional commentary</em>.”</p><p>Context Retrieval (<a href="https://www.elastic.co/">Elasticsearch</a>)</p><p>Now that we have classified the query as either itemtoitem or itemtouser, we have reached the most important part: retrieving the appropriate context. We have two main pieces of information stored in Elasticsearch: predictions for both customers and items, and embeddings.</p><p>Let’s start with retrieving the appropriate <strong>predictions</strong>! Using the classification response from GPT, we can direct the query to the appropriate indices depending on the classification.</p><p>- <strong>ItemtoItem</strong>: item_information, item_predictions<br>- <strong>ItemtoUser</strong>: customer_information, customer_predictions</p><p>For the specified indices for the given task, combine all relevant context.</p><p>Given an index and a user query, how is the appropriate context retrieved?</p><p>- Searches for the query text in all fields in the given index (which is either the formatted_predictions field or the info field depending on the index)<br>- For every “hit” found in the index, append to our context the relevant information (either the formatted_predictions field or the item_info field).<br>- Return context!</p><p>See below for an example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/0*Jt8nRMlIFojPFNAS.png" /></figure><p>Context Retrieval (Embeddings)</p><p>Now, let us think about a query like “I just bought the Mariette Blazer. Recommend me a white item to buy” or “I am customer x. Recommend me a sweater I can buy based on my purchase history”.</p><p>These queries can be powered with Kumo predictions. We will search for the predictions for the specific item/customer, and then search for the specific attribute within the predictions (ex. white items in the predictions for Mariette Blazer or sweaters in customer x’s predictions)</p><p>However, how about if there isn’t a white item in the predictions for the Mariette Blazer, or there isn’t a sweater in customer x’s predictions? This is where the embeddings come in. Given a specific attribute/filter (ex. white, sweater), instead of searching for the attribute within the pre-computed predictions, we can use embeddings to do a filtered search.</p><p>The workflow for retrieving embeddings can be summarized into four major steps:</p><p>- Input: User Query, Classification Response from GPT, User Filter<br>- Ex. Query: “I am customer 05373b97aa6d6e1e2be3ee243b0c6d060287db1de0118a27f0da4543236b3a02, I am looking for a white sweater”, Classification: “ItemToUser”, Filter: “White Sweater”<br>- Retrieve the appropriate embeddings for the item/customer specified in the query from the left hand embedding index (the user embeddings for a itemtouser query or the left hand side item embeddings for a itemtoitem query)<br>- For the retrieved embeddings, do a KNN search with the embeddings in the right hand side index that match the given filter<br>- The filter is over the item description in the right hand side index.<br>- Ex. retrieves customer_embedding from LHS index, does KNN search with items in RHS index with item description matching “white sweater”<br>- Append information for every top k embedding to context (taken from the item information index). Each of these items will have the attribute specified by the user (ex. white sweater).</p><h4>4. Retrieval Augmented Generation (RAG)</h4><p>So far, we have:</p><p>- Classified the user query as either itemtoitem or itemtouser<br>- Used this classification response to retrieve context either using predictions or embeddings if a specific attribute/filter is specified</p><p>Now that we have the relevant context, how can we use this to obtain a coherent response to display to the user? This is where RAG comes in.</p><p>The RAG approach can be broken down into three simple steps:</p><p>- Input: User Query, Relevant Context<br>- Construct final prompt {”question”: user_query, “context”: context}<br>- Call LLM with the final prompt, and that’s your final response!</p><h4>5. User Experience</h4><p>Now that we have our final response, we want to display it to the user, and have an easy way for the user to interact with our agent. Streamlit provides a simple, elegant UI for users to interact with and get responses to their queries.</p><p>Below is a diagram of the key features of the UI:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Z_I9nVMbYO4ob11F.png" /></figure><h4>6. Evaluation</h4><p>In order to test the accuracy of the responses returned, I created a test dataset of 50 queries: 25 itemtoitem and 25 itemtouser, varying in length and style.</p><p>Overall, the agent had <strong>90% </strong>accuracy (24/25 for itemtouser and 21/25 for itemtoitem)!</p><p>On average:</p><p>- Classification response by GPT is 98% accurate (1/50 query misclassified)<br>- For the itemtoitem queries misclassified, the items listed match the best Kumo recommendations with Jaccard index 0.67 (8 out of 10 match)</p><p>Below is an example of how the evaluation was set up:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/0*yCUPRxSTTn-pNWGk.png" /></figure><p>This chatbot is just one of the many cool things that can be built using Kumo AI!</p><p>Next step: <a href="https://kumo.ai/free-trial/">request a free trial</a> to give it a shot</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c83ccee599d3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[iFood Enhances Recommendation Systems and Ad Performance with Kumo AI]]></title>
            <link>https://medium.com/@kumoai/ifood-enhances-recommendation-systems-and-ad-performance-with-kumo-ai-bd5fbf4754c7?source=rss-8342d627d5b2------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd5fbf4754c7</guid>
            <category><![CDATA[customer-success]]></category>
            <dc:creator><![CDATA[Kumo AI]]></dc:creator>
            <pubDate>Thu, 19 Sep 2024 20:57:19 GMT</pubDate>
            <atom:updated>2024-09-19T20:57:19.754Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="iFood Enhances Recommendation Systems and Ad Performance with Kumo AI" src="https://cdn-images-1.medium.com/max/1024/0*lQzmSOjALyGNgyvb.jpg" /></figure><p>Kumo has the potential to accelerate other projects within iFood. It was easier and faster than creating a model from scratch and gave us high quality recommendations. <br>- <strong>Luiz Mendes, Head of Data Science at iFood</strong></p><p><a href="https://www.ifood.com.br/">iFood</a> is the largest food delivery company in Brazil, known for its extensive range of services, including groceries and payment solutions, in addition to food delivery. Handling over 80 million orders per month, iFood has established itself as a dominant player in the Brazilian food delivery market.</p><p>iFood partnered with <a href="https://medium.com/kumo.ai">Kumo AI</a> to address its challenges in scaling its recommendation systems, managing high computational costs, and delivering personalized, context-aware recommendations. Kumo offered a sophisticated solution that seamlessly integrated with iFood’s existing infrastructure, which included Amazon S3 as its central data storage, various data processing pipelines, and multiple machine learning platforms. This integration allowed iFood to improve recommendation relevance and effectiveness while handling the vast scale of operations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_rpSmhuFIuzLaE1m.png" /></figure><p><strong>CHALLENGES</strong></p><p>- <strong>Scaling Recommendation Systems: </strong>Recommendation was one of the first AI challenges that iFood tackled around 2019. Initially, iFood created an online matrix-based model that worked well for a while. However, as iFood grew, this online-only approach did not scale or perform as expected. iFood then invested in creating a robust recommendation platform to address scalability issues and evolve its recommendation capabilities. Luiz Mendes, Head of Data Science at iFood, highlighted the complexity of maintaining accurate and relevant recommendations at such a large scale. “Managing the vast amount of data and providing real-time, personalized recommendations was a significant challenge,” Mendes explained.</p><p>In the beginning, iFood sought to overcome several challenges with its recommendation system:</p><p>- <strong>Resource Intensity:</strong> The initial online-only matrix-based model required significant computational resources, which increased operational costs and complexity.<br>- <strong>Optimization:</strong> Early optimizations improved efficiency but couldn’t fully capture the intricate patterns and relationships in the data required for highly personalized recommendations.<br>- <strong>Real-Time Processing:</strong> Providing real-time recommendations added an additional layer of complexity that the early models and infrastructure struggled to manage effectively.<br>- <strong>Personalized and Context-Aware Recommendations:</strong> iFood aimed to deliver personalized recommendations for food orders, considering the context such as time of day, user preferences, and location. This level of personalization is complex due to the diverse user base and varying preferences. Mendes highlighted the importance of providing relevant suggestions to enhance user experience and engagement. “The diversity in our user base, with different tastes and preferences, made it crucial to offer recommendations that would be universally relevant and engaging.”.</p><p>To achieve this, iFood employed various methods to personalize recommendations, such as using collaborative filtering techniques and incorporating user feedback. They also experimented with different contextual factors, like time of day, user location, and historical ordering patterns, to refine their suggestions. These efforts included:</p><p>- <strong>Data Diversity:</strong> The vast and varied user data made it challenging to create a one-size-fits-all recommendation system.<br>- <strong>Contextual Complexity:</strong> Incorporating factors such as time of day and user location, but this integration often requires increased processing power and additional storage. For instance, a user’s food preference might change depending on whether they are ordering lunch at work or dinner at home.<br>- <strong>Scalability Issues:</strong> To manage scalability and provide real-time recommendations effectively, iFood employs an innovative offline-online approach. In this strategy, the more complex models operate offline, processing a curated set of candidates, while a faster model reranks these candidates in real time to ensure timely and relevant recommendations. Despite enhancing the system’s responsiveness and efficiency, maintaining some of these sophisticated offline models has proven to be costly and challenging. Integrating Kumo into this existing setup proved seamless, adding substantial value without any complications. This smooth integration underscores Kumo’s compatibility and ease of deployment within our advanced recommendation architecture..</p><p>Luiz Mendes highlighted the complexity of iFood’s recommendation challenges, noting, ‘While our team has made significant strides in incorporating diverse data points and contextual factors, we are always looking for ways to further enhance our models. Kumo provided us with an advanced AI solution that complemented our existing efforts, helping us tackle intricate relationships within our data more effectively.’</p><p>- <strong>High Computational Costs:</strong> Maintaining and updating machine learning models involves high computational costs and complexity. The data science team at iFood faced challenges in managing the computational resources required to run its models, leading to increased operational costs and the need for optimization.</p><p>To manage high computational costs, iFood focused on optimizing its algorithms and models to reduce resource consumption. This involved tuning hyperparameters, simplifying model architectures, and utilizing more efficient data processing techniques. Despite these efforts, the computational demands remained significant, necessitating further innovation and optimization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6ER_NVftoyGUZ9Hs.png" /></figure><p><strong>SOLUTION</strong></p><p>Kumo provided iFood with a sophisticated AI solution to address its recommendation system challenges. Here’s how Kumo’s capabilities transformed iFood’s operations:</p><p><strong>Enhancing Recommendation Systems:</strong> Kumo introduced advanced predictive models to enhance iFood’s recommendation systems. These models effectively handled the complexity and scale of iFood’s data, enabling more accurate and context-aware ad targeting. By analyzing relationships between users and items, Kumo’s AI could predict user behavior and preferences with greater precision.</p><p><strong>Integration with Existing Infrastructure:</strong> Kumo’s team worked closely with iFood’s data scientists to ensure a seamless transition. The process involved configuring Kumo AI with iFood’s data stored on Amazon S3 and their existing data processing pipelines. As Luiz Mendes mentioned, “The integration with S3 makes it much easier than having to build integration between platforms and keeping it updated.” Regular meetings and feedback sessions facilitated a smooth integration, addressing any issues promptly and ensuring optimal performance.</p><p><strong>Specific Features and Capabilities of Kumo’s Technology:</strong></p><p>- <strong>Eliminating Feature Engineering:</strong> Kumo automated much of the feature engineering process by learning embeddings directly from the data, significantly reducing the time and effort required from iFood’s data science team. Luiz explained, “Using Kumo greatly improved our experience with feature engineering over traditional models. By reducing the time and effort required from our team, we were able to develop an initial functional and acceptable MVP much faster.”<br>- <strong>Optimizing Computational Costs:</strong> “iFood had a graph-based recommender system that was our most costly model, requiring numerous machines with GPUs to train and serve. Kumo’s technology helped optimize computational costs by streamlining the data processing and model training processes,” Luiz noted. “Kumo replaced significant parts of our process, allowing us to focus on creating ETL outputs. This optimization has streamlined our offline processes.”<br><strong>Results and Impact</strong></p><p>The integration of Kumo’s AI capabilities yielded significant improvements in iFood’s recommendation systems and ad performance. Here are the key outcomes based on the conversation with Luiz Mendes:</p><p><strong>Improved Recommendation Accuracy:</strong> Kumo’s advanced algorithms brought a substantial boost to iFood’s recommendation accuracy. The components that utilized Kumo’s model experienced conversion rate increases ranging from 1% (for already high-performing components) to 10% (for less popular components), demonstrating Kumo’s ability to drive significant engagement across various scenarios. Notably, users began relying more on recommendations instead of the search and last-orders components, reflecting improved user satisfaction.</p><p><strong>Enhanced User Engagement and Conversion Rates:</strong> The more accurate and context-aware recommendations resulted in higher user engagement. Users were more likely to find relevant food and restaurant suggestions, leading to increased order volumes and higher conversion rates. The personalized recommendations fostered a more engaging and satisfying user experience, driving repeat usage and customer loyalty. iFood saw a remarkable increase in ad performance, with click-through rates (CTR) improving by up to 2% and ad conversions rising by approximately 3%. These enhancements ensured that users received more relevant food and restaurant suggestions, significantly elevating their overall experience on the platform.</p><p><strong>Increased Revenue:</strong> The improvements in recommendation accuracy, user engagement, and conversion rates collectively contributed to an increase in iFood’s revenue. By providing more relevant and timely recommendations, iFood was able to drive higher sales and improve customer satisfaction. The enhanced recommendations not only boosted order volumes but also encouraged users to explore and purchase more frequently.</p><p>Looking forward, iFood aims to continue leveraging Kumo’s AI capabilities to further enhance its recommendation systems, explore new use cases, and maintain its position as a leader in the Brazilian food delivery market. The collaboration between iFood and Kumo underscores the potential of advanced AI technologies in transforming business operations and achieving strategic goals.</p><p><a href="https://kumo.ai/apply/">Request a demo</a> to learn more about how Kumo AI can help you unlock the full potential of your data and create highly accurate predictive models using AI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UOOyERZY_bygbzWf.jpg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd5fbf4754c7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Uncovering Anomalies: How Kumo AI uses Graph Machine Learning for Advanced Fraud Detection]]></title>
            <link>https://medium.com/@kumoai/uncovering-anomalies-how-kumo-ai-uses-graph-machine-learning-for-advanced-fraud-detection-d45e2c02426e?source=rss-8342d627d5b2------2</link>
            <guid isPermaLink="false">https://medium.com/p/d45e2c02426e</guid>
            <category><![CDATA[fraud-detection]]></category>
            <category><![CDATA[anomaly-detection]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Kumo AI]]></dc:creator>
            <pubDate>Thu, 19 Sep 2024 20:54:54 GMT</pubDate>
            <atom:updated>2024-09-19T20:54:54.787Z</atom:updated>
            <content:encoded><![CDATA[<p>By <a href="http://linkedin.com/in/naisha-agarwal/">Naisha Agarwal</a> and <a href="https://www.linkedin.com/in/bstojanovic/">Blaz Stojanovic</a></p><h4><strong>Introduction</strong></h4><p>Fraud is an ever present problem in the financial service industry. Building reliable machine learning (ML) systems that can catch fraud fast is crucial for banks to remove fraudulent users from their platforms, and to retain the money they would have otherwise lost to fraud. As security measures improve, fraudsters adapt by developing new methods and techniques to circumvent detection systems and algorithms. Fraud is closely related to anomaly detection problems, where rare entities (such as fraudsters) must be found within data and systems. <a href="https://medium.com/kumo.ai">Kumo AI</a>, with its graph neural network approach, provides a robust scalable model that delivers outstanding performance on these types of problems in real-world production settings. In this blog post we will explore how Kumo can be used to achieve state of the art performance on a financial anomaly detection dataset — DGraphFin.</p><h4><strong>Anomaly Detection</strong></h4><p>Anomaly detection is concerned with finding entities which are sufficiently different from all other entities. The field has a rich history which contains both supervised approaches, where we manually label some anomalous examples and aim to find similar anomalies in almost a semi-supervised fashion, as well as unsupervised approaches where data analysis is performed to find what anomalous means in the first place. Most of the time both types of approaches are used in tandem to effectively combat fraud. Anomaly detection methods and workflows are also of utmost importance in other high-impact domains, including e-commerce, social networks, cybersecurity, finance and others. Similarly to other fields of applied machine learning, anomaly detection has seen rapid adoption and production of new and more powerful ML approaches in recent years, one of the most promising being graph machine learning.</p><p><strong>Why Graphs?</strong></p><p>In predictive models, data is king, and how we choose to represent and model the available data determines our success. In anomaly detection, much like in other subfields of applied machine learning, we can contrast traditional approaches with a new, more data centric paradigm. In traditional data science we flatten many relational tables and perform careful feature engineering to generate a single training table and train a tabular model. With more modern “data centric” approaches, an emphasis is put on high quality data and its most natural representation. An effort is made to now “throw away” any useful signals as one is inevitably forced to do if performing feature engineering. Additionally, powerful ML models that can naturally learn over data in this form are used for more predictive power. Over the past few years we have already seen this paradigm shift happen in computer vision, moving from hand designed filters to CNNs and now vision transformers, as well as in natural language processing with the dawn of LLMs. Kumo’s graph centric approach embodies this paradigm shift for tabular/relational datasets. Our graph transformers operate on the relational database directly, without any feature engineering and provide the same performance edge over traditional approaches.</p><p>The subfield of anomaly detection which explores the use of graph machine learning approaches is commonly referred to as Graph Anomaly Detection (GAD). Graphs have two basic components: nodes and edges. Nodes represent entities while edges represent relationships between those entities (e.g., nodes can be bank accounts and edges are transactions between accounts). Furthermore, graphs can be either static or dynamic. A static graph is a fixed graph that does not change over time, and all entities remain immutable throughout time. Dynamic graphs (or temporal graphs) are ones that change dynamically (users joining/leaving and interacting in a social network can be represented as a temporal graph example of a case that would require a dynamic graph).Why use graphs and graph-based approaches for<a href="https://www.notion.so/IBM-Transactions-for-Anti-Money-Laundering-Case-Study-29219a8946bc49e09f9a72194b30432a?pvs=21"> anomaly detection</a>? By structuring our data as graphs, we can reframe the problem as detecting anomalies within these graphs, more specifically abnormal nodes (e.g. users), edges (e.g. user transactions), or subgraphs (e.g. a small group of users and their transactions). Graphs are in particular well-suited for dynamic data due to their inherent ability to model relationships and patterns between different entities without any simplification to the underlying data. Additionally, graph machine learning approaches are adaptable to topological structure, can be scalable to very large networks, incorporate multimodal attribute data, are highly interpretable, and can be extended in many ways e.g. with self-attention or transformer components. Graph anomaly detection has massive impact and benefits in the real world. Let us now look at this problem in the context of the DGraphFin dataset.</p><h4><strong>Dataset</strong></h4><p>The DGraphFin dataset is a<a href="https://arxiv.org/pdf/2207.03579"> publicly accessible dataset</a> provided by <em>Finvolution Group</em>, a Chinese consumer finance industry with over 140 million consumers; the Finvolution platform connects underserved borrowers with financial institutions. In order for a borrower to use the Finvolution platform, they must register with an account, where they complete a basic personal profile with age, gender, and other factors to determine their loan limit. One of the required elements of this personal profile is an emergency contact — the person in the user’s life that should be contacted in the event of an emergency. Before each new loan application, users must give one contact’s name; the platform then evaluates loan requests and determines whether to give loans to users.</p><p>There are some users on this platform that borrow money but do not pay it back, which the platform labels as <em>fraudulent</em>. These fraudsters are anomalies on the platform, and hence the machine learning task Finvolution is trying to solve is detecting these anomalies on their platforms, an industrial challenge. These fraudulent users often fill in false information on their profiles, an important part that can be used to detect them. In particular, the emergency contact that all users are required to fill out is a highly predictive signal of fraud for this task. In the graph centric approach the emergency contact is treated as part of the overall network, providing crucial topological/structural signals that turn out to be very useful for classification/prediction on this task.</p><p>In essence the DGraphFin dataset is structured as a directed multigraph, with nodes representing users and eleven different types of edges connecting these users (the edges are different interactions between users, one of them being an emergency contact). The edges are also attributed with a TIMESTAMP signifying when the interaction happened. In total, the graph contains 3,700,550 nodes and 4,300,999 directed edges. Each user node is a vector of 17 dimensions, dimensions which correspond to a certain portion of the personal profile that users fill out when entering the platform — i.e. user attributes.</p><figure><img alt="Visualization of what the DGraphFin dataset looks like." src="https://cdn-images-1.medium.com/max/518/0*CkgsI7ZiyO3IV9T6.png" /></figure><p><em>Visualization of what the DGraphFin dataset looks like. Can model the dataset as a directed graph between users, where one user is the emergency contact of another. The different colors indicate the different types of interactions in the graph, and each interaction is labeled with a timestamp.</em></p><p>32.2% of the nodes in DGraph (1,225,601) have related borrowing records, and are labeled as such, and of those, 15,509 are classified as anomalous and the rest as normal. DGraph is a challenging anomaly detection dataset. Not only is the classification problem heavily imbalanced, but the many relationships between users need to be considered and there is a large variety between user neighborhoods — traditionally this would require careful feature construction and evaluation. Additionally, this graph is very sparse, as some users do not fill emergency contact information, and emergency contacts who are not in the Finvolution network are also filtered out. Hence, DGraph contains 49.9% missing values, making the problem of identifying fraudsters in the graph even more difficult.</p><h4><strong>Solution with Kumo</strong></h4><p>The original dataset contains two tables:</p><p>- users table: includes user_id a unique identifier of each user, anomaly <strong>label</strong>, plus several features from their personal profile<br>- interactions table: which contains four columns, two with user_id’s, one for each user part of this interaction in the interaction, a timestamp column indicating the point in time when the interaction happened, and the type column which records the interaction type.</p><p>We can structure the dataset/graph in two different ways:</p><p>- Maintain the original structure of users and interactions tables, connect the tables using the primaryforeign user_ids in the respective tables (see Graph 1).<br><br>- Alternatively, we can split the interactions tables into a separate table for each interaction type, and drop the interaction type column. We do this for illustrative purposes here, but this step might be beneficial if each interaction type had many different features associated with it, or if the interaction types are very different. We connect these interaction tables to the users table in the same way as in approach 1 (see Graph 2).<br><strong>Graphs</strong></p><p>The first graph is composed of two tables, with the primary key being the user_id in the users table, and the foreign keys in the interactions table being the user_source_id and user_target_id in the respective interaction tables. Graph 1 looks as follows:</p><figure><img alt="Graph with primary key being the user_id in the users table" src="https://cdn-images-1.medium.com/max/422/0*Enid3osbp0TvVsrl.png" /></figure><p><em>Graph 1: no restructuring was done.</em></p><p>In the second graph, there are twelve tables tables in total. It is composed of the users table and the eleven different interactions tables, each interaction table corresponding to a different type of interaction captured in the dataset. Graph 2 looks as follows:</p><figure><img alt="Graph composed where interactions table is split into separate tables for each interaction" src="https://cdn-images-1.medium.com/max/422/0*pSi21wPWEhCCpwxx.jpeg" /></figure><p><em>Graph 2: Interactions table is split into separate tables for each interaction</em></p><p><strong>The Predictive Query</strong></p><p>Machine learning models in Kumo can be defined with a declarative <strong>P</strong>redictive <strong>Q</strong>uery <strong>L</strong>anguage (PQL). The predictive query for a binary classification model, where we’re predicting labels at the user level (we have the label column available in that table) is very simple:</p><figure><img alt="simple predictive query using PQL" src="https://cdn-images-1.medium.com/max/972/0*5a4TLcVbPS-jw-z9.png" /></figure><p>The PQL is very expressive and can be used to define a variety of ML tasks; for reference, refer to the<a href="https://docs.kumo.ai/docs/tutorial"> Predictive Query Tutorial</a>. After we define the ML problem with PQL, we set the model plan. Here are some of our settings for this particular task:</p><figure><img alt="Predictive Query for fraud detection task" src="https://cdn-images-1.medium.com/max/948/0*Zx_HZVhCtmqLMR_j.png" /></figure><p>The search for these parameters was performed by using Kumo in BEST run-mode. Once the model is defined and we run training, Kumo takes care of many crucial data science steps:</p><p>- Ingests the data from individual tables in the graph, takes care of missing values, encodes/vectorizes the data in individual columns according to our<a href="https://docs.kumo.ai/docs/encoder_overrides#supported-encoders"> encoder settings</a>.<br>- Constructs the temporal graph based on primary and foreign key connections<br>- Generates the training table according to our specifications, automatically takes care of point-in-time correctness for all entities in the graph with timestamps<br>- Performs model architecture search over the model space we defined in the model plan<br>- Provides relevant evaluation and explanations of the best model in our search<br><strong>Kumo’s graph transformers</strong></p><p>Kumo employs an advanced graph transformer model that captures the nuanced patterns and relationships within the dataset, outperforming all other existing graph machine learning models. Our model makes predictions based on the aggregation of information in a node’s local neighborhood. An example computational graph would look like this:</p><figure><img alt="Computational graph for DGraphFin data set given target node and input graph" src="https://cdn-images-1.medium.com/max/1024/0*P1GOnIsbgzMUMxdc.png" /></figure><p>Computational graph for DGraphFin data set given target node and input graph</p><p>Kumo’s graph transformer model architecture is based on many recent graph neural network and LLM advancements and cutting edge academic research. This includes models such as GraphSAGE that enables inductive representation learning, making it effective even with limited interaction data. ID-GNN is used to discern patterns such as repeated transactions, while PNA introduces diverse aggregation operators. GCN uses mean-pooling aggregation to capture user similarities, GIN focuses on frequency signals to understand complex behaviors, NBF efficiently captures paths between notes, and GraphMixer employs temporal representation learning to analyze sequences of user actions. LLM techniques are carefully used to ensure that proper learning of temporal sequences is possible.</p><h4><strong>Results</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*jrpubaY4ir5ec6ZK" /></figure><p>Kumo significantly outperforms other<a href="https://proceedings.neurips.cc/paper_files/paper/2023/file/5eaafd67434a4cfb1cf829722c65f184-Paper-Datasets_and_Benchmarks.pdf"> benchmarks</a>, particularly for the graph with the two table setup, achieving an AUROC of 0.841 and an AUPRC of 0.063. The entire training and prediction time for the Kumo model took approximately 4 hours, no feature engineering required. Achieving this type of performance using traditional ML methods may not be possible for this particular dataset and even getting close would take an experienced data scientist many days of iteration and careful feature engineering. These metrics indicate a robust capability to detect fraudulent activities, making Kumo an excellent choice for real-world applications in fraud detection.</p><h4><strong>Conclusion</strong></h4><p>In this blog, we introduced the field of anomaly detection, focusing on its importance in identifying fraud. We explore the massive benefits of restructuring these types of problems into a graph, and using graph machine learning approaches to detect anomalies. Not only is graph ML adaptable to the topological structure of the data, it is scalable to very large networks, can incorporate multimodal attribute data, is highly interpretable, and can be extended in many ways. Kumo, with its advanced nuanced graph transformer model, shows the power of machine learning in handling anomaly detection tasks, significantly outperforming traditional machine learning approaches, specifically on the DGraphFin dataset.</p><p>Kumo’s approach offers several key advantages: it can seamlessly integrate various types of data, scale to large datasets, and constantly adapt to patterns in the data. This makes the product highly effective for real-world fraud detection and other anomaly detection tasks. The use of GNNs allows for the automatic learning of complex relationships and patterns in the data and incorporation of LLM techniques ensures the resolution of complex temporal behaviors, without having to do manual feature engineering and enabling more accurate predictions.</p><p>To get started with Kumo for fraud or other anomaly detection tasks, check out this <a href="https://docs.kumo.ai/docs/fraud-detection">guided walkthrough of Kumo for a fraud detection</a> task. Kumo offers a scalable efficient solution for any organization looking to utilize graph AI to enhance their data analysis and prediction capabilities.<br><br>Next step: <a href="https://kumo.ai/apply">request a demo</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d45e2c02426e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Recommendation Systems are Better Off Using Hybrid Graph Neural Networks]]></title>
            <link>https://medium.com/@kumoai/why-recommendation-systems-are-better-off-using-hybrid-graph-neural-networks-8adc5e62e554?source=rss-8342d627d5b2------2</link>
            <guid isPermaLink="false">https://medium.com/p/8adc5e62e554</guid>
            <category><![CDATA[blog-posts]]></category>
            <dc:creator><![CDATA[Kumo AI]]></dc:creator>
            <pubDate>Thu, 19 Sep 2024 20:49:33 GMT</pubDate>
            <atom:updated>2024-09-19T20:49:33.009Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Why Recommendation Systems are Better Off Using Hybrid Graph Neural Networks" src="https://cdn-images-1.medium.com/max/998/0*Hrqy6q8pSdFsba0N.png" /></figure><p><strong>Discover how Kumo AI utilizes hybrid graph neural networks (GNNs) to revolutionize recommendation systems. Explore the effectiveness of this approach in both Kaggle data science challenges and real-world customer scenarios.</strong></p><h4>Hybrid GNNs: Transforming recommendation systems with <a href="https://medium.com/kumo.ai">Kumo AI</a></h4><p>Recommendation systems have been a subject of great innovation over the past few decades, starting with matrix factorization in the early 2000s and evolving into two-tower and other deep learning approaches in the 2010s. In recent times, graph neural networks emerged as the leading approach to power recommender systems, enabling many well known tech companies (such as Pinterest, Uber, Amazon, and others) to deliver magical customer experiences and double-digit lifts in business metrics. Kumo offers a robust architecture known as hybrid graph neural networks (hybrid GNNs), which has been empirically shown to deliver outstanding performance on both public Kaggle data science challenges, and real-world production deployments.</p><h4>Understanding the complexity of recommendation systems</h4><p>At their core, recommendation systems are responsible for recommending content or products to users, with the goal of providing inspiration to users. However, developing these systems is inherently challenging, due to fickle preferences and complex patterns of human behavior. Users vary significantly in preferences; some are explorers who constantly seek new experiences, while others are repeaters who prefer familiarity. This is made even more challenging in the face of big data, cold-start items, new users with very little interaction history, and lack of data diversity.</p><p>Because this problem is so challenging, many engineering teams have developed multi-stage recommendation pipelines involving numerous candidate generation steps followed by a complex ensemble of ranking models. These systems take tens of millions of dollars to build, in terms of human and infrastructure cost, and are challenging to maintain over time.</p><h4>Introducing the hybrid GNN architecture</h4><p>To address these challenges, Kumo developed a hybrid GNN approach that can create great recommendations with a single model, while capturing the nuanced behaviors of different users with remarkable accuracy. The hybrid GNN models two distinct user behaviors differently within a single backbone GNN model: repeated interactions and explorative interactions; hence, referring to this model as a <strong><em>hybrid GNN</em></strong>. The hybrid GNN is the default model architecture for recommendation and personalization tasks at Kumo, and it can be fine tuned to your specific dataset using the Model Planner.</p><h4>Why GNNs are ideal for recommendations</h4><p>GNNs are highly-suitable for recommendation tasks because, unlike traditional models, they can leverage rich graph connectivity patterns to gain a deeper understanding of user preferences and insights that are often missed by other algorithms.</p><p>The recommendation problem forms a <strong><em>bipartite graph</em></strong> between users and items, where nodes represent the users and items, and edges represent the user-item interactions. Edges often come with timestamps. Moreover, multiple edges may exist between pairs of nodes, since a user may repeatedly interact with the same item (e.g., repeat ordering of the same product in e-commerce). Given the bipartite graph of the past interactions, a recommendation task can be cast as a link prediction task — one that calls for predicting future interactions between user nodes and item nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/998/0*p7FTtMv6CIZNK2Pu.png" /></figure><h4>Model input: Processing data with the hybrid GNN</h4><p>The hybrid GNN model is designed to capture fine-grained user behaviors by leveraging graph connectivity. Similar to a standard GNN model (e.g., GraphSAGE), the Hybrid GNN model processes input through a subgraph centered around each user node. For simplicity and efficiency, consider a 1-hop neighbor sampler:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nL14IAW8GUcuTo6Y.png" /></figure><p>A 1-hop neighbor subgraph contains items that a user previously interacted with, as well as features associated with the sampled users, items, and edges (e.g., timestamp, price etc). Given the subgraph, a hybrid GNN employs a heterogeneous GNN to compute embeddings of the user and items.</p><h4><strong>Exploring the hybrid GNN model architecture</strong></h4><p>The key innovation of the hybrid GNN is its <strong><em>hybrid</em></strong> approach to computing item scores per user, which are then sorted to produce the top K item recommendation for the user. Specifically, the <strong>hybrid GNN computes item scores differently based on whether or not items are sampled within the subgraph.</strong></p><p>These differing scoring approaches are as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Xjn9ny2Vzf471oc4.png" /></figure><p><strong>(1) For items sampled in the subgraph, </strong>the Hybrid GNN computes the item scores by applying a multi-layer perceptron (MLP) over the GNN’s item embeddings. Since the GNN’s item embedding contains information about historical interactions between the user and the item, the MLP can be applied to the item embedding <strong>to predict whether or not the user would repeat the interaction with the item.</strong></p><p><strong>(2) For items not sampled in the subgraph,</strong> the Hybrid GNN computes the item scores by taking an inner product between GNN’s user embeddings with shallow item embeddings. As the low-dimensional item embeddings can capture similarity between items (à la matrix factorization), GNNs can use this method <strong>to recommend similar items that a user has never interacted with before.</strong></p><p>The observation that <strong><em>user behaviors are diverse </em></strong>is the final element that makes the hybrid GNN work. Some users prefer to repeatedly interact with the same set of items (better captured by the first hybrid GNN scoring approach), while others like to constantly explore new items (better captured by the second hybrid GNN scoring approach). To accommodate such diversity across different users, the hybrid GNN learns a <strong><em>user-specific repetition scalar</em></strong> predicted from GNN’s user embeddings with another MLP. The scalar is added to the score of the first approach to capture the repetitiveness of each user’s behavior. The more repetitive a user is, the hybrid GNN will predict a higher <strong><em>user-specific repetition scalar</em></strong>. On the other hand, if users are highly explorative (i.e., they interact with new items a lot), the hybrid GNN will predict lower <strong><em>user-specific repetition scalar</em></strong>.</p><h4>Training and optimization: Maximizing performance with the hybrid GNN</h4><p>The hybrid GNN is trained end-to-end, optimizing both types of item scores as well as the repetition scalar altogether to maximize the predictive performance of future user-item interactions.This way, the hybrid GNN will figure out the user behaviors from data on its own, producing highly accurate predictions that capture the complex nature of repetition versus exploration behaviors.</p><h4>Empirical studies: Assessing hybrid GNN performance</h4><p>The hybrid GNN performance was tested using a <a href="https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations">Kaggle H&amp;M recommendation challenge</a>. The challenge called for predicting the top 12 items each user would purchase in the next 7 days, with model performance measured by mean average precision (MAP) @ 12. The dataset contains two years of historical data consisting of 1.4M users, 106K items, and 31.7M interactions between them. The challenge attracted a total of 3,000+ teams that submitted results to the public Kaggle leaderboard, over the course of the 3-month competition held in 2022.</p><h4>Comparing hybrid GNN results to top Kaggle competitors</h4><p>The following are the results of the hybrid GNN, evaluated on the hidden test set after the competition (Kaggle allows post-competition submissions). A comparison of the hybrid GNN results to the top Kaggle competitor submissions is also provided below:</p><p>ModelMAP@12 score on Kaggle public leaderboard<strong>Hybrid GNN0.031</strong>Kaggle top 10%0.024Kaggle Median0.021</p><p>The hybrid GNN placed in the top 1% of all submissions in the Kaggle H&amp;M recommendation challenge, which is 47% better than the median score (which is what an average data scientist will be able to achieve with traditional techniques). Using Kumo, the entire hybrid GNN training and prediction time took approximately two hours on a single GPU, without any feature engineering required. In contrast, the leading Kaggle competition challengers utilized complex model ensembling techniques and feature engineering code that would require months to develop and maintain in production.</p><h4>Ablation studies</h4><p>In order to confirm that the hybrid GNN produces better results than a traditional GNN approaches, Kumo ran an ablation study, where only one ranking technique was used at prediction time. The hybrid GNN was more than 100% better than the “inner product” approach, which is the standard approach used by two-tower recommendation models.</p><p>ModelMAP@12Hybrid GNN isApproach (1) — Use MLP to score items in the sampled GNN subgraph.0.02335% betterApproach (2) — Inner product between user embedding and item embeddings.0.015107% better</p><h4>Real-world applications: Deploying the hybrid GNN in production</h4><p>Kumo has deployed the hybrid GNN model architecture to production in many enterprises, resulting in significant boosts in model performance when compared to internal baselines and improvements in revenues and customer experiences. The following illustrates Kumo’s recommendation performance on a large-scale local food delivery service — with the task of recommending the restaurants that each customer will most likely order from in the next 7 days (out of 600K+ restaurants). The impact of the Kumo recommendations powered by the hybrid GNN architecture generated over $100 million in additional sales for the food delivery company.</p><p>ModelMAP@12 score<strong>Kumo hybrid GNN0.32</strong>Approach (1)0.31Approach (2)0.27</p><h4>Conclusion: The power of the hybrid GNN in recommendation systems</h4><p>The hybrid GNN model is a testament to the power of innovative machine learning techniques in handling the intricacies of recommendation systems. By simplifying the recommendation process into a single model that adapts to various user behaviors, this not only enhances user satisfaction but also provides a scalable, efficient solution for businesses aiming to personalize their services. As enterprises continue to seek out technologies that can deliver precise recommendations in real-time, hybrid GNN stands out as a beacon of innovation and performance in the data science community and beyond.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8adc5e62e554" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kick-off Kumo in Snowflake with Smarter Product Recommendations via Snowflake Solution Center]]></title>
            <link>https://medium.com/@kumoai/kick-off-kumo-in-snowflake-with-smarter-product-recommendations-via-snowflake-solution-center-779f24700f04?source=rss-8342d627d5b2------2</link>
            <guid isPermaLink="false">https://medium.com/p/779f24700f04</guid>
            <dc:creator><![CDATA[Kumo AI]]></dc:creator>
            <pubDate>Thu, 19 Sep 2024 20:47:54 GMT</pubDate>
            <atom:updated>2024-09-19T20:47:54.074Z</atom:updated>
            <content:encoded><![CDATA[<h4>Improve average order size with targeted product recommendations</h4><p>by <a href="https://www.linkedin.com/in/zack-drach-12422548/">Zach Drach</a></p><p><a href="https://medium.com/kumo.ai">kumo.ai</a></p><p>In today’s digital landscape, where information overload and choice paralysis are the norm, recommendation systems are pivotal in guiding users toward the most relevant content. These systems provide a more personalized customer experience, improving consumer happiness and your bottom line.</p><p>Used by retailers, media, entertainment, advertisers, and more — Kumo powered recommendation systems are relied upon by leading organizations to drive key KPIs like average order size, sales, and even monthly active users. Kumo now offers its recommendation solution to interested customers directly via the <a href="https://developers.snowflake.com/solution/making-product-recommendations-to-high-value-customers/">Snowflake Solution Center</a> and our <a href="https://app.snowflake.com/marketplace/listing/GZTYZBN5P0/kumo-ai-kumo?originTab=provider&amp;providerName=KUMO.AI&amp;profileGlobalName=GZTYZBN5MD">Snowflake Native App</a>, bringing the powerful predictive platform directly into the Snowflake environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/857/0*rp7lwsaVyJI3Z9t0.png" /></figure><h4>Why Kumo in Snowflake</h4><p>Kumo saves time during predictive model development because it eliminates the complex process of rebuilding the end-to-end pipeline to suit each new query. Kumo helps data scientists make highly accurate predictions about user and customer behavior by combining graph learning over structured data and generative AI models trained for unstructured data — all entirely within Snowflake.</p><p>Named Forbes AI 50 in 2024, Kumo’s predictive AI learns across multiple Snowflake tables, to build accurate predictive models in hours — not months. The out of the box machine learning models can be refined for maximum performance and predictions can be delivered in batches or as embeddings for use downstream. Use a low-code interface to simplify MLOps, combine graph learning and LLMs, and use Kumo to accurately predict the right products, actions, and offers for each person to drive improvements to KPIs like conversion, engagement, and retention.</p><p><strong><em>⭐ If you want to try this yourself, </em></strong><a href="https://app.snowflake.com/marketplace/listing/GZTYZBN5P0/kumo-ai-kumo"><strong><em>download the Kumo app here</em></strong></a><strong><em> and dig into </em></strong><a href="https://developers.snowflake.com/solution/making-product-recommendations-to-high-value-customers/"><strong><em>the solution here</em></strong></a><strong><em>. ⭐</em></strong></p><h4>What you’ll build:</h4><p>Using Kumo as a Snowflake Native App, you will build predictive models for determining customer lifetime value (LTV) and delivering top ten product recommendations to those customers, all within your Snowflake environment. You can easily access those predictions from within a Snowflake worksheet, and deploy these predictions to your production website or email notification system.</p><h4>Kumo’s solution includes a step-by-step guide that users can use to follow along:</h4><p>- Setting Up Data in Snowflake<br>- Registering Your Schema in Kumo and Create Your Graph<br>- Building Your Predictive Models<br>- Evaluating Model Performance<br>- Productionizing Your Model</p><h4>Related Resources:<em></em></h4><p><a href="https://quickstarts.snowflake.com/guide/leveraging_kumo_for_smarter_recommendations/index.html?index=....index#0">Recommendations quickstart guide<br></a><a href="https://docs.kumo.ai/">Kumo Documentation<br></a><a href="https://app.snowflake.com/marketplace/listing/GZTYZBN5P0/kumo-ai-kumo">Snowflake Native Apps<br></a><a href="https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview?_fsi=7UEfbZ9e">Snowflake Snowpark Container Services<br></a><a href="https://docs.kumo.ai/docs/tutorial">Predictive Query Tutorial<br></a><a href="https://docs.kumo.ai/docs/installing-kumo-on-spcs">Installing Kumo on SPCS</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=779f24700f04" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Are Shallow Embeddings Important in Recommendations?]]></title>
            <link>https://medium.com/@kumoai/why-are-shallow-embeddings-important-in-recommendations-855d567f432c?source=rss-8342d627d5b2------2</link>
            <guid isPermaLink="false">https://medium.com/p/855d567f432c</guid>
            <category><![CDATA[predictions]]></category>
            <category><![CDATA[gnn]]></category>
            <category><![CDATA[embedding]]></category>
            <dc:creator><![CDATA[Kumo AI]]></dc:creator>
            <pubDate>Thu, 19 Sep 2024 20:44:18 GMT</pubDate>
            <atom:updated>2024-09-19T20:46:01.955Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="diagram of user and items making the shape of a graph for use in predictions" src="https://cdn-images-1.medium.com/max/1023/0*Oplq7hdP_ACKYFjo.jpg" /></figure><p>By <a href="https://www.linkedin.com/in/daveywang/">Davey Wang</a> and <a href="https://www.linkedin.com/in/matthias-fey-b4a947221/">Matthias Fey</a><br><a href="http://kumo.ai/">Kumo.ai</a></p><p>In the construction of recommendation systems, embeddings play a crucial role in enhancing performance by capturing the nuanced and complex relationships between users and items. These representations help models understand not just direct interactions but also potential, implicit interactions that have not yet occurred, making recommendations more accurate and personalized.</p><h4>The importance of sampling negatives</h4><p>Sampling negatives is the process of selecting non-interacted or non-relevant items for training a recommendation model. In recommendation systems, the data of positive interactions — examples where a user has shown interest in an item by clicking, purchasing, or liking it are greatly outnumbered by negatives: the absence of an interaction.</p><p>Given the quantity of the negatives, it is impractical to use all examples, so “negative sampling” is employed to select a manageable subset for training the model. There are two key reasons for sampling negatives:</p><p>- <strong>Handling imbalance</strong>: in most datasets, negative interactions vastly outnumber positive ones. Negative sampling balances the dataset and ensures that the model learns not only from what users like but also from what they ignore.<br>- <strong>Improving efficiency</strong>: it is computationally expensive to consider all possible negative examples (which could be millions) for each user. Sampling a representative set of negative examples reduces the computational load during training.</p><h4>The role of shallow embeddings in sampling negatives</h4><p>Shallow embeddings offer an efficient and scalable solution for handling the negative examples that are abundant in recommendation systems. Negative examples, or the absence of interactions between a user and an item, make up the vast majority of the data in recommendation systems. In this context, shallow embeddings become invaluable, allowing the system to effectively sample a large set of these negatives and process them efficiently during training.</p><h4>Why performance matters</h4><p>Shallow embeddings are particularly effective in handling imbalanced and sparse datasets. In most recommendation systems, a significant portion of user-item interactions are unobserved. Shallow embeddings, leveraging latent factors, can capture general patterns such as:</p><p>- Popularity trends<br>- Seasonal variations<br>- User preferences for certain categories<br>- Demographic appeals</p><p>By understanding these general patterns, shallow embeddings enable the model to infer potential interactions — even for negative examples — thereby enhancing the system’s performance. They allow the model to extrapolate to unseen interactions, making the recommendation process more reliable and robust.</p><h4>Scalability in sampling negatives</h4><p>One of the key benefits of shallow embeddings is their computational efficiency when sampling negative examples. Given the scale of most recommendation systems, this efficiency is critical. Shallow embeddings can process large datasets quickly without the computational burden that comes with more complex deep learning techniques, making them <strong>ideal for real-time or large-scale applications</strong>.</p><h4>Shallow vs. deep embeddings</h4><p>Where shallow embeddings are powerful in capturing broad patterns and handling negative examples, pure graph neural network (GNN) embeddings have historically struggled with these signals. This is because deep GNN embeddings typically focus on local similarity — the relationships between items within a user’s direct network or sub-graph.</p><p>Shallow embeddings, on the other hand, capture both local <em>and </em>positional similarity:</p><p>- Local similarity refers to how items in the immediate neighborhood (or sub-graph) of a user are similar to one another.<br>- Positional similarity considers how close items are to one another in the broader graph, even if they are not directly connected.</p><p><a href="https://arxiv.org/abs/1905.08108">Learn more about the research in the area of shallow vs deep embeddings.</a></p><p>Because of this, shallow embeddings provide a more comprehensive representation, which is why they tend to outperform deep GNNs in certain recommendation tasks.</p><h4>When are shallow embeddings applied?</h4><p>Kumo applies shallow embeddings depending on the item’s relationship to the user’s sub-graph. For items that belong to the sub-graph of a user, deep GNNs are used to capture the detailed local similarities. However, for items that do not belong to the sub-graph, shallow embeddings are leveraged to capture broader patterns and provide recommendations.</p><p><a href="https://kumo.ai/ns-newsarticle-why-recommendation-systems-are-better-off-using-hybrid-graph-neural-networks/">Learn more about hybrid GNNs.</a></p><h4>Conclusion</h4><p>In conclusion, by offering both deep and shallow embeddings, Kumo provides the key enablers of performant and scalable recommendation systems. By balancing local and positional similarities and efficiently handling negative sampling, it is easier to deliver accurate, timely, and relevant recommendations in large-scale environments.</p><p>Interested in learning more? <a href="https://kumo.ai/apply/">Request a demo</a> to get started.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=855d567f432c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>