<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Memgraph on Medium]]></title>
        <description><![CDATA[Stories by Memgraph on Medium]]></description>
        <link>https://medium.com/@memgraphdb?source=rss-ad8382481555------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Nb4tlhbQhdupjOLMgCjQBw.png</url>
            <title>Stories by Memgraph on Medium</title>
            <link>https://medium.com/@memgraphdb?source=rss-ad8382481555------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 10:13:17 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@memgraphdb/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Memgraph Helm Chart Kubernetes]]></title>
            <link>https://medium.com/memgraph/memgraph-helm-chart-kubernetes-a2a37100f586?source=rss-ad8382481555------2</link>
            <guid isPermaLink="false">https://medium.com/p/a2a37100f586</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[memgraph]]></category>
            <category><![CDATA[helm-chart]]></category>
            <category><![CDATA[helm]]></category>
            <dc:creator><![CDATA[Memgraph]]></dc:creator>
            <pubDate>Fri, 08 Sep 2023 14:03:17 GMT</pubDate>
            <atom:updated>2023-09-08T14:03:17.017Z</atom:updated>
            <content:encoded><![CDATA[<p>Managing software in different environments is a challenge. Your application typically has various dependencies, which may include libraries, interpreters, packages, configurations, etc. Each of these elements also has a tendency to perform differently in different environments. In order to make all dependencies work, the environment should not change, and this is where containers come into the picture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ffEMSGBT83fVJyPqiX4YoA.png" /></figure><p>Nowadays, running an application in a low-cost abstraction container saves the days of managing software dependencies. There are multiple solutions on the market for packing your applications to containers, Docker being one of the most popular. Due to the switch to containerized application, now you have loads of different containers running on servers. This means containers must somehow be tracked, scaled, managed, and configured consistently and programmatically — meet <a href="https://kubernetes.io/">Kubernetes</a>, the open-source container orchestration system founded by Google in 2014.</p><p>Since then, it has become the standard for managing containers. Kubernetes has its specifics of handling resources, such as clusters, nodes, pods, services, objects, etc. This blog post won’t dive into the details of Kubernetes here, so if you want to learn more, feel free to explore their <a href="https://kubernetes.io/docs/home/">documentation</a>.</p><h3>Helm to the rescue</h3><p>Configuring Kubernetes involves a series of YAML files that define all Kubernetes resources. The YAML files usually involve a bunch of boilerplate configurations that are useful to different users. Suppose multiple users want to deploy Memgraph in the Kubernetes cluster. In that case, it is likely they will do it very similarly, and YAML files will be very analogous, except for a few configuration changes. On top of that, you do not need to investigate how to do it, but you should spend time configuring a service or a product to your needs. Hence, you probably want preconfigured YAML files in a packed manner, and this is where Helm comes to the rescue.</p><p><a href="https://helm.sh/">Helm</a> is a package manager for Kubernetes that allows developers to package, configure, and deploy applications and services onto Kubernetes clusters with minimal effort. Helm will help you install, upgrade, configure, and fetch all resources for deploying products and services in Kubernetes. Essentially, Helm provides charts defining how different containers should be deployed. Helm and Kubernetes are separate rabbit holes to dive into. For now, let’s see what Memgraph has to offer.</p><h3>Meet Memgraph Helm chart repository 🚢</h3><p>Helm chart for deploying Memgraph to Kubernetes is one of the most requested things by our community. You asked — we listened! Introducing our brand new Helm chart repository. This should ease the way of deploying Memgraph to a set of simple Helm commands. Currently, it holds the <a href="https://github.com/memgraph/helm-charts/tree/main/charts/memgraph">standalone MemgraphDB chart</a> that you can use to deploy <a href="https://memgraph.com/memgraphdb">MemgraphDB</a> inside Kubernetes.</p><p>The prerequisite is that you have a Helm installed and a place to deploy your Kubernetes cluster. That could be any cloud environment such as AWS, GCP, or MS Azure, just to name a few. For the sake of local testing, you can also use <a href="https://minikube.sigs.k8s.io/docs/start/">Minikube</a>, which is great for prototyping and learning Kubernetes.</p><p>To deploy Memgraph, first you need to add the Memgraph Helm chart repository to your local Helm setup by running the following command:</p><p>helm repo add memgraph <a href="https://memgraph.github.io/helm-charts">https://memgraph.github.io/helm-charts</a></p><p>After you have a local Helm chart repository from Memgraph, you can install Memgraph to Kubernetes via the following command:</p><p>helm install my-release memgraph/memgraph</p><p>You can replace my-release with a name of your choice for the release. This will install Memgraph in the Kubernetes cluster with default values specified inside the <a href="https://github.com/memgraph/helm-charts/blob/main/charts/memgraph/values.yaml">values.yaml</a>. That includes MemgraphDB with appropriate volumes for storage and logs. At this point, you should be able to <a href="https://memgraph.com/docs/memgraph/connect-to-memgraph">connect to your Memgraph instance</a> via Memgraph Lab and mgconsole if the appropriate IP address and port are specified (this will depend on your deployment environment).</p><p>The default configuration can be overridden during the install process:</p><p>helm install &lt;resource-name&gt; memgraph/memgraph --set &lt;flag1&gt;=&lt;value1&gt;,&lt;flag2&gt;=&lt;value2&gt;,...</p><p>For more details on what can be configured further, take a look at the <a href="https://memgraph.com/docs/memgraph/reference-guide/deployment/kubernetes">Memgraph Helm chart documentation</a>.</p><h3>Future work</h3><p>The current release of Memgraph Helm repo is in the early stages of development. There is a lot of work to be done to make Memgraph more approachable to the Kubernetes environment, but it is a continuous development process.</p><p>Future work will involve making Memgraph MAGE and Memgraph platform charts for easier deployment to Kubernetes, making the process more flexible. To keep up with our next release follow the repository <a href="https://github.com/memgraph/helm-charts/milestones">milestones</a>.</p><p>On top of that, it is always nice to have a helping hand if you notice that something is not working for you or you wish to contribute a chart you have created for Memgraph. So feel free to open a pull request or an issue on <a href="https://github.com/memgraph/helm-charts">our new repository</a> and definitely throw us a star there! ⭐️</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a2a37100f586" width="1" height="1" alt=""><hr><p><a href="https://medium.com/memgraph/memgraph-helm-chart-kubernetes-a2a37100f586">Memgraph Helm Chart Kubernetes</a> was originally published in <a href="https://medium.com/memgraph">Memgraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[In-memory vs. disk-based databases: Why do you need a larger than memory architecture?]]></title>
            <link>https://medium.com/memgraph/in-memory-vs-disk-based-databases-why-do-you-need-a-larger-than-memory-architecture-ea6d54900af2?source=rss-ad8382481555------2</link>
            <guid isPermaLink="false">https://medium.com/p/ea6d54900af2</guid>
            <category><![CDATA[memgraph]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[database-development]]></category>
            <dc:creator><![CDATA[Memgraph]]></dc:creator>
            <pubDate>Tue, 05 Sep 2023 14:43:44 GMT</pubDate>
            <atom:updated>2023-09-05T14:43:44.497Z</atom:updated>
            <content:encoded><![CDATA[<p>Memgraph is an in-memory graph database that recently added support for working with data that cannot fit into memory. This allows users with smaller budgets to still load large graphs to Memgraph without paying for (more) expensive RAM. However, expanding the main-memory graph database to support disk storage is, by all means, a complex engineering endeavor. Let’s break this process down into pieces.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q-c6xKrZd1zNS4PZDTpIyA.png" /></figure><h3>On-disk databases</h3><p>Disk-based databases have been, for a long time, a de facto standard in the database development world. Their huge advantage lies in their ability to store a vast amount of data relatively cheaply on disk. However, the development can be very complex due to the interaction with low-level OS primitives. Fetching data from disk is something that everyone strives to avoid since it takes approximately 10x more time than using it from main memory. Neo4j is an example of a graph, an on-disk database that uses disk as its main storage media while trying to cache as much data as possible to main memory so it could be reused afterward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6b5EbcISwLd8C55V.png" /></figure><h3>In-memory databases</h3><p>In-memory databases avoid the fundamental cost of accessing data from disk by simply storing all its data in the main memory. Such architecture also significantly simplifies the development of the storage part of the database since there is no need for a buffer pool. However, the biggest issue with in-memory databases is when the data cannot fit into the random access memory since the only possible way out is to transfer the data to a larger and, consequently, more expensive machine.</p><p>In-memory database users rely on the fact that durability is still secured through durability mechanisms like transaction logging and snapshots so that data loss does not occur.</p><h3>Larger-than-memory architecture</h3><h3>Main memory computation</h3><p>Larger-than-memory architecture describes a database architecture when the majority of computations are still within the main memory, but the database offers the ability to store a vast amount of data on disk, too, without having the computational complexity of interacting with buffer pools.</p><h3>Identify hot &amp; cold data</h3><p>The larger-than-memory architecture utilizes the fact that there are always hot and cold parts of the database in terms of accessing it. The goal is then to find cold data stored and move it to the disk so that transactions still have fast access to hot data. Cold data identification can be done either by directly tracking transactions’ access patterns (online) or by offline computation in which a background thread analyzes data.</p><p>The second very important feature of the larger-than-memory architecture is the process of evicting cold data. This can be done in two ways:</p><ol><li>DB tracks the memory usage and starts evicting data as soon as it reaches a predefined threshold.</li><li>Eviction can be done only when new data is needed.</li></ol><h3>Transaction management</h3><p>Different systems also behave differently regarding transaction management. If the transaction needs data that is currently stored on the disk, it can:</p><ol><li>Abort the transaction, fetch data stored on the disk, and restart the transaction.</li><li>Stall the transaction by synchronously fetching data from the disk.</li></ol><h3>Transaction must fit into memory</h3><p>The question is, what happens when the transaction data cannot fit into random access memory? In Memgraph, we decided to start with an approach that all transaction data must fit into memory. This means that some analytical queries cannot be executed on a large dataset, but this is the tradeoff we were willing to accept in the first iteration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hbaPECri1XHDOgAf.png" /></figure><h3>Benefits of larger-than-memory databases</h3><p>Memgraph uses <a href="https://rocksdb.org/">RocksDB</a> as a <a href="https://memgraph.com/blog/what-is-a-key-value-database">key-value store</a> for extending the capabilities of the in-memory database. Not to go into too many details about RocksDB, but let’s just briefly mention that it is based on a data structure called <a href="https://www.cs.umb.edu/~poneil/lsmtree.pdf">Log-Structured Merge-Tree</a> (LSMT) (instead of B-Trees, typically the default option in databases), which are saved on disk and because of the design come with a much smaller <a href="https://smalldatum.blogspot.com/2019/05/crum-conjecture-read-write-space-and.html">write amplification</a> than B-Trees.</p><p>The in-memory version of Memgraph uses Delta storage to support multi-version concurrency control (MVCC). However, for larger-than-memory storage, we decided to use the Optimistic Concurrency Control Protocol (OCC) since we assumed conflicts would rarely happen, and we could make use of <a href="https://github.com/facebook/rocksdb/wiki/Transactions">RocksDB’s transactions</a> without dealing with the custom layer of complexity like in the case of Delta storage.</p><p>We’ve implemented OCC in a way that every transaction has its own private workspace, so potential conflicts are detected at the commit time. One of our primary requirements before starting to add disk-based data storage was not to ruin the performance of the main memory-based storage. Although we all knew there was no such thing as <a href="https://www.youtube.com/watch?v=rHIkrotSwcc">zero-cost abstraction</a>, we managed to stay within 10% of the original version. We decided to use <a href="https://memgraph.com/blog/acid-transactions-meaning-of-isolation-levels">snapshot isolation</a> as an appropriate concurrency isolation level since we believed it could be the default option for the large majority of Memgraph users.</p><h3>Disadvantages of larger-than-memory databases</h3><p>As always, not everything is sunshine and flowers, especially when introducing such a significant feature to an existing database, so there are still improvements to be made. First, the requirement that a single transaction must fit into memory makes it impossible to use large analytical queries.</p><p>It also makes our <a href="https://memgraph.com/docs/memgraph/import-data/load-csv-clause/">LOAD CSV</a> command for importing CSV files practically unusable since the command is executed as a single transaction. Although RocksDB is really good, fits really well into our codebase, and has proved to be very efficient in its caching mechanisms, maintaining an external library is always hard.</p><h3>In retrospect</h3><p>Albeit the significant engineering endeavor, the larger-than-memory architecture is a super valuable asset to Memgraph users since it allows them to store large amounts of data cheaply on disk without sacrificing the performance of in-memory computation. We are actively working on resolving issues introduced with the new storage mode, so feel free to <a href="https://discord.com/invite/memgraph">ask</a>, <a href="https://github.com/memgraph/memgraph/issues">open an issue</a>, or <a href="https://github.com/memgraph/memgraph/pulls">pull a request</a>. We will be more than happy to help. Until next time 🫡</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea6d54900af2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/memgraph/in-memory-vs-disk-based-databases-why-do-you-need-a-larger-than-memory-architecture-ea6d54900af2">In-memory vs. disk-based databases: Why do you need a larger than memory architecture?</a> was originally published in <a href="https://medium.com/memgraph">Memgraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exciting News: LangChain Now Supports Memgraph!]]></title>
            <link>https://medium.com/memgraph/exciting-news-langchain-now-supports-memgraph-a0000885e639?source=rss-ad8382481555------2</link>
            <guid isPermaLink="false">https://medium.com/p/a0000885e639</guid>
            <category><![CDATA[memgraph]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[langchain]]></category>
            <dc:creator><![CDATA[Memgraph]]></dc:creator>
            <pubDate>Fri, 25 Aug 2023 07:08:42 GMT</pubDate>
            <atom:updated>2023-08-25T08:10:59.111Z</atom:updated>
            <content:encoded><![CDATA[<p>We’re thrilled to announce a powerful integration between LangChain and Memgraph, bringing you an unparalleled natural language interface to your Memgraph database. Say goodbye to complex queries and welcome a seamless and intuitive way to interact with your data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3OMOKtrGEnXYEV7IDKDsbg.png" /></figure><h3>Memgraph QA chain tutorial</h3><p>If you’ve ever wanted to effortlessly query your Memgraph database using natural language, this tutorial is for you. This step-by-step guide will walk you through the process, ensuring you have all the tools you need to get started.</p><h3>Prerequisites</h3><p>Before you dive in, make sure you have Docker and Python 3.x installed on your system.</p><h3>Get started</h3><p><strong>Launch a Memgraph Instance</strong>: With a few simple commands, you can have your Memgraph instance up and running using Docker. Just <a href="https://python.langchain.com/docs/use_cases/more/graph/graph_memgraph_qa">follow our script</a> to set it up.</p><p><strong>Install dependencies</strong>: We’ve got you covered with the required packages. Use pip to install langchain, openai, neo4j, and gqlalchemy. Don’t forget the --user flag to ensure smooth permissions.</p><p><strong>Code playtime</strong>: Whether you prefer working within this notebook or want to use a separate Python file, the tutorial offers code snippets to guide you through the process.</p><h3>What’s inside</h3><p>Explore the rich features and functionalities that LangChain and Memgraph offer together:</p><p><strong>API reference</strong>: We provide an overview of the key components you’ll be working with, such as ChatOpenAI, GraphCypherQAChain, and MemgraphGraph.</p><p><strong>Populating the database</strong>: Learn how to populate your Memgraph database effortlessly using the Cypher query language. We guide you through the process of seeding data that serves as the foundation for your work.</p><p><strong>Refresh graph schema</strong>: Familiarize yourself with refreshing the graph schema, a crucial step in setting up the Memgraph-LangChain graph for Cypher queries.</p><p><strong>Querying the database</strong>: Discover how to interact with the OpenAI API and configure your API key. We’ll show you how to utilize the GraphCypherQAChain to ask questions and receive informative responses.</p><p><strong>Chain modifiers</strong>: Customize your chain’s behavior with modifiers like return_direct, return_intermediate_steps, and top_k. Tailor the experience to your preferences.</p><p><strong>Advanced querying</strong>: Delve into advanced querying techniques and uncover tips for refining your prompts to improve query accuracy.</p><p>Ready to take your data interaction to the next level? Join us in exploring the seamless synergy between LangChain and Memgraph. No more wrangling with queries — just natural language and meaningful insights. Simplify complexity, elevate your insights, and share your projects in our <a href="https://discord.com/invite/memgraph">community</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a0000885e639" width="1" height="1" alt=""><hr><p><a href="https://medium.com/memgraph/exciting-news-langchain-now-supports-memgraph-a0000885e639">Exciting News: LangChain Now Supports Memgraph!</a> was originally published in <a href="https://medium.com/memgraph">Memgraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is a Graph Database?]]></title>
            <link>https://medium.com/memgraph/what-is-a-graph-database-160858c54371?source=rss-ad8382481555------2</link>
            <guid isPermaLink="false">https://medium.com/p/160858c54371</guid>
            <category><![CDATA[nosql-database]]></category>
            <category><![CDATA[graph-database]]></category>
            <category><![CDATA[memgraph]]></category>
            <category><![CDATA[nosql]]></category>
            <dc:creator><![CDATA[Memgraph]]></dc:creator>
            <pubDate>Wed, 23 Aug 2023 15:22:38 GMT</pubDate>
            <atom:updated>2023-08-23T15:22:38.503Z</atom:updated>
            <content:encoded><![CDATA[<p>While relational databases have been the go-to choice for data storage, they fall short when it comes to handling complex relationships and traversing interconnected data, which puts graph databases in a special spotlight. A graph database is a specialized database system designed to store, manage, and query highly connected data using graph theory principles. As data volumes continue to explode, companies need efficient and scalable solutions to handle the complexities of their data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MRBvFTrMFfQ6m76JIOaLFQ.png" /></figure><p>Specialized database systems like graph databases offer a more natural and efficient way to model, query, and store data, leading to improved performance and better data insights.</p><h3>Understanding graphs</h3><p>In simple terms, at the core of graph databases lies the concept of a graph. In mathematics and computer science, a graph is a collection of nodes (also known as vertices) connected by edges. Nodes represent entities or objects, while edges depict the relationships or connections between them. This straightforward yet effective structure forms the foundation of graph databases.</p><h3>Components of graphs</h3><p>It’s your time to shine! Let’s reiterate on what we’ve learned so far. Graphs consist of two fundamental components: nodes and edges.</p><ol><li>Nodes represent entities or objects and can have various attributes associated with them.</li><li>Edges, on the other hand, depict the relationships or connections between nodes and can also carry properties.</li></ol><p>Together, nodes and edges create a rich network of connected data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_bMg0cRcUnvNOOJg.png" /></figure><h3>Graph theory basics</h3><p>Another common term you may hear here and there, alluding to graphs or graph databases, is graph theory, which is a branch of mathematics, that provides the theoretical underpinning for understanding and analyzing graphs. It defines vertices as the fundamental building blocks of a graph and edges as the connections between vertices. Relationships in a graph can be represented by directed or undirected edges, capturing the nature and direction of the connections.</p><h3>Relational databases vs. graph databases</h3><p>Opinions split when it comes to <a href="https://memgraph.com/blog/how-to-choose-a-database-for-your-needs">choosing a database</a>, however, the debate around <a href="https://memgraph.com/blog/graph-database-vs-relational-database">relational vs. graph databases</a> is still hot. Relational databases have long been the dominant database model, organizing data into structured tables with predefined schemas. They excel in handling structured data and transactions but face challenges when dealing with complex relationships and traversing connected data. This is largely due to their rigid tabular structure.</p><p>Joining multiple tables and navigating through numerous relationships can lead to performance bottlenecks and complex query formulations. This limits their effectiveness in scenarios where relationships play a crucial role.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mekq9TtNBciQX5EJ.png" /></figure><p>Graph databases excel in modeling and querying relationships. They store connections explicitly, allowing for efficient traversals between nodes and enabling complex relationship queries with ease. And of course, graph databases provide flexibility, scalability, and performance advantages over a relational database when it comes to handling interconnected data.</p><h3>Characteristics of graph databases</h3><p>So far, you’ve been introduced to a few qualities that are typical to graphs, so let’s put the learnings into structure and build off of what you’ve grasped.</p><h3>Schema-less nature</h3><p>Unlike relational databases, graph databases are schema-less, meaning they do not require a predefined structure or schema for data. This flexibility allows for the dynamic addition of new node types, properties, and relationships, making graph databases highly adaptable to evolving data models.</p><h3>Native graph processing</h3><p>Graph databases are purpose-built for processing graph data. They employ optimized algorithms and data structures to efficiently traverse and manipulate the graph structure, resulting in faster query response times and improved performance compared to non-native graph databases.</p><h3>Graph traversal and pattern matching</h3><p>One of the key strengths of graph databases is their ability to traverse and explore relationships between nodes. Graph traversal algorithms can efficiently navigate the graph to discover patterns, uncover hidden connections, and retrieve data based on specific criteria. This capability is particularly valuable in applications such as recommendation engines, fraud detection, and knowledge graphs, which we will explore in the sections to come.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*w0jg7fBkLCOPtOwx.png" /></figure><h3>Use cases of graph databases</h3><p>Unlike a traditional relational database that relies on tabular data, a graph database utilizes a flexible and intuitive data model, allowing for the representation of intricate relationships between entities. With its ability to efficiently capture and traverse vast networks of data, a graph database has emerged as an advanced tool for diverse domains, including:</p><h3>Social networks and recommendation engines</h3><p>Graph databases have revolutionized social networking platforms and <a href="https://memgraph.com/blog/building-a-recommendation-system-using-memgraph">recommendation engines</a>. They enable personalized recommendations, friend suggestions, and social network analysis by leveraging the rich network of connections between users, interests, and entities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gvu9o8He9ioVb3Ox.png" /></figure><h3>Fraud detection and network analysis</h3><p>Graphs also excel in <a href="https://memgraph.com/blog/how-memgraph-helped-companyx-save-7-figures-fraud-detection">fraud detection</a> and network analysis. By representing complex networks of relationships, they can identify suspicious patterns, detect fraudulent activities, and uncover hidden connections that might indicate illicit behavior, making them an invaluable tool for cybersecurity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gEEeUQQBei8ACSSa.png" /></figure><h3>Knowledge graphs and semantic networks</h3><p>Last but not least, graph databases serve as a foundation for building knowledge graphs and semantic networks. By representing data as nodes and relationships, they capture the semantics and context of information, enabling sophisticated knowledge discovery, semantic search, and data integration across disparate sources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*G2FSbEW9-l3Kt6sr.png" /></figure><h3>Sneak peek into graph algorithms</h3><p>Surely enough, <a href="https://memgraph.com/white-paper/beginners-guide-to-graph-algorithms">graph algorithms</a> play a crucial role in leveraging the power of graph databases and unlocking valuable insights from connected data. In this section, we provide a sneak peek into some fundamental graph algorithms that form the backbone of graph database operations.</p><p><strong>Breadth-First Search (BFS)</strong>: Breadth-First Search is a fundamental algorithm used to explore and traverse a graph in a breadth-first manner. Starting from a given source node, BFS systematically explores all the neighboring nodes before moving deeper into the graph. This algorithm is commonly used to find the shortest path between two nodes, identify connected components, and perform level-based analysis.</p><p><strong>Depth-First Search (DFS)</strong>: Depth-First Search is another crucial graph algorithm that explores a graph with a depth-first principle. DFS starts from a given source node and traverses as far as possible along each branch before backtracking. The algorithm is useful for identifying cycles in a graph, performing topological sorting, and searching for specific nodes or patterns.</p><p><strong>PageRank algorithm</strong>: Developed by Google’s founders, <a href="https://memgraph.com/blog/pagerank-algorithm-for-graph-databases">PageRank</a> is a graph algorithm used to measure the importance or relevance of nodes in a graph, particularly in web graphs. PageRank assigns each node a numerical value based on the number and quality of incoming links, and plays a vital role in search engine ranking, recommendation systems, and social network analysis.</p><p>These are just a few examples of the numerous graph algorithms available, however, graph databases employ a wide range of algorithms to perform tasks such as <a href="https://memgraph.com/docs/mage/algorithms/traditional-graph-analytics/community-detection-algorithm">community detection</a>, centrality analysis, graph clustering, and more.</p><h3>Sum up</h3><p>In this article, we explored the world of graph databases and their significance in modern data management. We defined graph databases and highlighted their importance in handling complex relationships and interconnected data. If you’re curious and want to learn more about the fascinating world of graphs, make sure to check out our <a href="https://memgraph.com/blog">blog</a> and give us a shout in our <a href="https://memgraph.com/community">community</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=160858c54371" width="1" height="1" alt=""><hr><p><a href="https://medium.com/memgraph/what-is-a-graph-database-160858c54371">What is a Graph Database?</a> was originally published in <a href="https://medium.com/memgraph">Memgraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Security Analysis with JupiterOne’s Starbase and Memgraph]]></title>
            <link>https://medium.com/memgraph/security-analysis-with-jupiterones-starbase-and-memgraph-298c65764cbf?source=rss-ad8382481555------2</link>
            <guid isPermaLink="false">https://medium.com/p/298c65764cbf</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[memgraph]]></category>
            <category><![CDATA[starbase]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[Memgraph]]></dc:creator>
            <pubDate>Tue, 22 Aug 2023 16:39:25 GMT</pubDate>
            <atom:updated>2023-08-22T16:39:25.317Z</atom:updated>
            <content:encoded><![CDATA[<p>Starbase is an open-source graph-based security analysis tool that unifies all of JupiterOne’s integrations into one. It collects assets and relationships from services and systems, including cloud infrastructure, SaaS applications, security controls, and more, into an intuitive graph visualization. With over 115 open-source graph integrations, Starbase collaborates with your existing toolkit enabling easy and insightful cyber security analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jP4kK1eemtNGcE2b58Uwbw.png" /></figure><p>In this article, we’ll dig into Starbase, guiding you through the setup of two example integrations and enabling Starbase to work with Memgraph for easy ingestion and visualization of your graph data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/726/0*4EYYF1JkibdvCSjN.png" /></figure><h3>Prerequisites</h3><ol><li>Installed <a href="https://yarnpkg.com/">Yarn</a> package manager.</li><li>Installed <a href="https://nodejs.org/en">Node.js</a>.</li><li>A running Memgraph instance — visit Memgraph’s docs for instructions on how to <a href="https://memgraph.com/docs/memgraph/installation">install</a> and <a href="https://memgraph.com/docs/memgraph/connect-to-memgraph">connect</a> to Memgraph.</li></ol><h3>Setting up Starbase</h3><ol><li>To kick-start your Starbase setup, first, you need to clone the <a href="https://github.com/JupiterOne/starbase">JupiterOne/Starbase</a> repo into your local directory and ensure you have <strong>Yarn</strong> and <strong>Node.js</strong> installed.</li><li>Once you’ve successfully cloned the repository and installed the prerequisites, place yourself in the terminal in the directory where you cloned the repo and run the yarn command. The command installs all of the necessary project dependencies.</li><li>The next step is setting up configurations for your integration of choice. You can find a list of all integrations on JupiterOne’s GitHub repo. Moving forward, we are going to explore two options for possible integration, <a href="https://github.com/jupiterone/graph-zoom">Zoom</a>, and <a href="https://github.com/jupiterone/graph-github">GitHub</a>.</li></ol><h3>Setting up integrations</h3><p>In order to set up an integration, you need to register an account in the system the integration targets for ingestion and obtain the necessary API credentials. Starbase leverages credentials from external services to authenticate and collect data. When Starbase is started, it reads configuration data from a single configuration file named config.yaml at the root of the project.</p><h3>Zoom integration</h3><p>In order to configure the Zoom integration, we need to create a Zoom app to retrieve the needed credentials:</p><ol><li>Go to the <a href="https://marketplace.zoom.us/">Zoom App Marketplace</a> and sign into your Zoom account.</li><li>In the top right corner, go to the Develop dropdown menu and select Build App.</li><li>Choose to create an <strong>OAuth</strong> type of app.</li><li>Take note of your <strong>Account ID</strong>, <strong>Client ID</strong>, and <strong>Client secret</strong> which we’ll need for the configuration file later on.</li><li>In the Scopes section, add group:read:admin, role:read:admin, user:read:admin, and account:read:admin.</li></ol><p>After you’ve successfully created your Zoom App, open up the starbase repo in your editor of choice and create your config.yaml file. This is an example of a config.yaml file for Zoom integration:</p><pre>integrations:<br>  -<br>    name: graph-zoom<br>    instanceId: testInstanceId<br>    directory: ./.integrations/graph-zoom<br>    gitRemoteUrl: &lt;https://github.com/JupiterOne/graph-zoom.git&gt;<br>    config:<br>      ACCOUNT_ID: &lt;ACCOUNT_ID&gt;<br>      CLIENT_ID: &lt;CLIENT_ID&gt;<br>      CLIENT_SECRET: &lt;CLIENT_SECRET&gt;<br>      SCOPES: &#39;read:admin role:read:admin user:read:admin account:read:admin&#39;</pre><h3>GitHub integration</h3><p>In order to configure GitHub integration, we need to create a GitHub app to retrieve the needed credentials:</p><ol><li>Go to the <a href="https://github.com/settings/apps">GitHub Apps</a> and select to create a new GitHub App</li><li>Name your app, and enter a homepage URL (in this case, you can use the JupiterOne’s <a href="https://github.com/JupiterOne/starbase">Starbase repo URL</a>), uncheck the webhook and adjust the repository permissions. The following permissions need to be set to <strong>read-only</strong>: -<em>Repository Permissions</em>: Actions, Environments, Issues, Pull Requests and Secrets -<em>Organization Permissions</em>: Administration, Members, Secrets. The rest of the permissions are <strong>No access</strong> by default.</li></ol><p>Read-only access for secrets repo doesn’t give read-only access to actual secret content, it only gives read-only info to the existence of the metadata about the secrets.</p><ol><li>Select <strong>Any account</strong> and create your GitHub App.</li></ol><p>After you’ve successfully created your GitHub App, open up the cloned Starbase repository in your editor of choice and create your config.yaml file. Generate your private key and retrieve other needed credentials from the GitHub App you previously created. Below is an example of a config.yaml file for a GitHub integration:</p><pre>integrations:<br>    -<br>  name: graph-github<br>  instanceId: testInstanceId<br>  directory: ./.integrations/graph-github<br>  gitRemoteUrl: &lt;https://github.com/JupiterOne/graph-github.git&gt;<br>  config:<br>  GITHUB_APP_ID = &lt;GITHUB_APP_ID&gt;<br>GITHUB_APP_LOCAL_PRIVATE_KEY_PATH={YOURPATH}/{YOURFILENAME}.private-key.pem<br>  INSTALLATION_ID=&lt;INSTALLATION_ID&gt;<br>  GITHUB_API_BASE_URL=https://api.github.com   </pre><h3>Use Starbase with Memgraph</h3><p>After you’ve successfully created your config.yaml file, the last step is to adjust your queries to work with Memgraph. In order to do that, run the following steps:</p><ol><li>First, you need to place yourself in the terminal in the folder you cloned your Starbase repo and run yarn starbase setup command to clone or update all integrations listed in the config.yaml file, as well as install all dependencies for each integration.</li><li>Run your Memgraph instance. Follow the instructions from Memgraph’s docs on how to connect to Memgraph, or if you are using Docker, simply run the following command: docker run -it -p 3000:3000 -p 7444:7444 -p 7687:7687 memgraph/memgraph-platform</li><li>By modifying just a single line of code, you are ready to use Starbase with Memgraph. Inside the neo4jGraphStore.js file, locate the addEntities() function. To enable compatibility with Memgraph, simply update the following line:</li></ol><pre>await this.runCypherCommand(`CREATE INDEX index_${entity._type} IF NOT EXISTS FOR (n:${entity._type}) ON (n._key, n._integrationInstanceID);`);</pre><p>With:</p><pre>await this.runCypherCommand(`CREATE INDEX index_${entity._type} IF NOT EXISTS FOR (n:${entity._type}) ON (n._key, n._integrationInstanceID);`);<br>await this.runCypherCommand</pre><p>You are all set to utilize Starbase with Memgraph. The instance is actively listening to port 7687, as defined in the code.</p><p>The final step is to run the yarn starbase run command. Afterward, launch your browser and go to localhost:3000 to access <strong>Memgraph Lab</strong> or open your desktop version to explore and visualize your graph data.</p><h3>Explore your dataset</h3><p>Below, we’ve provided a few query examples that demonstrate how you can dig into your dataset and extract valuable insights. The following examples assume the use of GitHub integration.</p><p>With the following query, you are retrieving the information of all of the extracted GitHub users from a certain organization:</p><p>MATCH (n:github_user) RETURN n LIMIT 3;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HLUrsBhylw5e8AZb.png" /></figure><p>If you also want to determine which code owners of organization repositories grant access to outside contributors, execute the following query:</p><p>MATCH (account:github_account) - [e:OWNS] -&gt; (repo:github_repo) -&gt; [f:ALLOWS] -&gt; (user:github_user {role: ‘OUTSIDE’}) RETURN account, repo, user, e, f;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iJnpeWfn7aBdl72v.png" /></figure><h3>Takeaways</h3><p>Starbase is a powerful tool that simplifies security analysis by unifying integrations into a user-friendly graph view, enhancing cybersecurity insights. Incorporating Memgraph for data ingestion adds another dimension by enhancing its capabilities and visualizing your data. If you are curious about graphs and would like to learn more, make sure to check out our <a href="https://memgraph.com/blog">blog</a> and join our community on <a href="https://discord.gg/memgraph">Discord</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=298c65764cbf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/memgraph/security-analysis-with-jupiterones-starbase-and-memgraph-298c65764cbf">Security Analysis with JupiterOne’s Starbase and Memgraph</a> was originally published in <a href="https://medium.com/memgraph">Memgraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Memgraph vs. TigerGraph]]></title>
            <link>https://medium.com/memgraph/memgraph-vs-tigergraph-a209145f1069?source=rss-ad8382481555------2</link>
            <guid isPermaLink="false">https://medium.com/p/a209145f1069</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[graph-database]]></category>
            <category><![CDATA[memgraph]]></category>
            <category><![CDATA[tigergraph]]></category>
            <dc:creator><![CDATA[Memgraph]]></dc:creator>
            <pubDate>Fri, 18 Aug 2023 06:46:11 GMT</pubDate>
            <atom:updated>2023-08-18T06:46:11.755Z</atom:updated>
            <content:encoded><![CDATA[<p>In today’s data-driven world, the necessity to process and interpret complex relationships within massive datasets is making organizations continually search for the go-to graph database, leaving the traditional relational database options behind.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OxKr29XmFh7y48Gzux4AQA.png" /></figure><p>After the initial <a href="https://db-engines.com/en/ranking/graph+dbms">DB-Engines</a> consultations, two names commonly arise in conversations: TigerGraph and Memgraph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/0*eZc40hQyqOA4jX6a.png" /></figure><h3>Background on both solutions</h3><p>Founded in 2012 by Dr. Yu Xu, <strong>TigerGraph’s</strong> core objective is to provide a scalable and efficient graph database platform that enables organizations to leverage the power of interconnected data, supporting applications ranging from fraud detection to AI and machine learning.</p><p><strong>Memgraph</strong> is an in-memory, open-source graph database with roots in the UK and Croatia. Founded by Marko Budiselic and Dominik Tomicevic in 2016, and backed by American investors, Memgraph prioritizes high performance and developer accessibility. With a robust community edition, the platform offers a blend of ease of use and practical functionality, all presented through clear and uncomplicated <a href="https://memgraph.com/blog/what-is-an-open-source-license">licensing</a>, making it the backbone of many cybersecurity solutions.</p><h3>Memgraph vs. TigerGraph differences</h3><p>Although both TigerGraph and Memgraph have been developed in C++ and aim to provide performant solutions for <a href="https://memgraph.com/blog/real-time-graph-analytics">real-time data analytics</a>, there exist some important differences between the two platforms that set them apart. Let’s check what those are.</p><h3>Query language</h3><p>The choice of query language plays a significant role in the overall user experience.</p><p>GSQL, TigerGraph’s proprietary query language, does offer an expressive, Turing-complete language tailored for graph pattern-matching and analytics functions but might present a steeper learning curve for those new to graphs. It has been specifically designed for TigerGraph, and the skillset may not be easily transferred from or to other graph database platforms.</p><p>In contrast, Cypher query language is an open-source, declarative language known for its user-friendly syntax. Cypher’s human-readable style has propelled it into a standard for querying graph databases. It has been developed by Neo4j but is utilized by various systems, including Memgraph. Due to its simplicity, and broad community support, it is a preferred choice for many developers who know their applications will need minimum changes if they require a switch to another database vendor.</p><h3>Data storage</h3><p>TigerGraph and Memgraph offer distinctive approaches to handling data in their graph databases, each reflecting a unique strategy to balance performance, scalability, and flexibility.</p><p>TigerGraph employs a hybrid memory-disk approach, leveraging RAM for storing frequently accessed data and disk storage for large graphs that may exceed available memory. This hybrid model allows TigerGraph to achieve real-time analytics, where active datasets are immediately available, while also scaling to handle massive datasets without being constrained by RAM.</p><p>In contrast, Memgraph’s architecture has been built natively for in-memory data analysis and storage, focusing on lightning-fast data processing. Being ACID compliant, it ensures consistency and reliability in its core design. However, Memgraph also offers flexibility. An analytical storage mode that bypasses ACID compliance is available, accelerating analytics and data import operations when absolute consistency is not required. Additionally, an on-disk storage option allows users to weigh performance against budget constraints, thus achieving a balance tailored to specific needs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hqJaqjQIQ9j8t2GK.png" /></figure><p>While TigerGraph’s hybrid approach offers a comprehensive solution for both speed and scalability, Memgraph’s focus on in-memory processing with adaptable options reflects a commitment to performance with versatility to suit various requirements. The distinction between these two models shows the innovation in graph database technology, catering to diverse needs in data management, analysis, and storage.</p><h3>Data isolation level</h3><p>TigerGraph employs a <a href="https://memgraph.com/blog/acid-transactions-meaning-of-isolation-levels">read-committed data isolation level</a>, meaning that a transaction can access data which is committed before and during this transaction’s execution. For example, two same READ queries inside one transaction can return different results because between them another transaction was committed.</p><p>On the other hand, Memgraph uses snapshot isolation by default, where each query operates on a consistent snapshot of the data at the query’s start time, with the option to change the isolation level, but snapshot isolation offers an advantage as it provides a more consistent view of the data, reducing the chance of reading partial or uncommitted changes. This ensures more accurate query results and a smoother transaction experience, making snapshot isolation generally considered a more reliable approach in many scenarios.</p><h3>Pricing models and support</h3><p>TigerGraph has a free version that allows users to work with up to 50GB of data, making it suitable for small projects or initial exploration. Memgraph offers something different with its <a href="https://memgraph.com/pricing">Community Edition</a>, which is not only free but also open-source and packed with features.</p><p>For example, both TigerGraph and Memgraph offer high availability features to ensure that data is consistently accessible and resistant to failures, but Memgraph’s replication is available even in the Community Edition of the product. This means that the Community Edition is not “crippleware” but a fully functional version that allows users to “kick the tires” on the product and properly test it to ensure it meets requirements before deploying it in a production environment.</p><p>By offering this, and a plethora of other features in the Community Edition, Memgraph not only shows a commitment to performance and reliability but also to accessibility, empowering users to explore and validate the capabilities of the software without barriers.</p><p>Due to the lack of complicated layers of management that some larger companies might have, in Memgraph you can talk directly to engineers if you have questions or need help. It’s a more hands-on, direct way of working that puts you closer to the people who built the product, and it can make working with Memgraph a more pleasant and efficient experience.</p><h3>Overview of features</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/743/0*17JzTRqgLju6cKo7.png" /></figure><h3>Takeaways on both graph databases</h3><p>Memgraph and TigerGraph both offer graph database solutions, but Memgraph’s native in-memory design sets it apart. Built for speed without losing stability or ACID compliances, Memgraph provides efficient real-time querying and analytics. Although TigerGraph claims to be “The World’s Fastest Graph Analytics Platform for the Enterprise”, clients have reported increased performance after switching to Memgraph. If speed, reliability, direct interaction, and support from engineers are key priorities, Memgraph may be the more appealing choice.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pBgHSqk9LASlTnug.png" /></figure><p>Check the performance of Memgraph on your own dataset using <a href="https://memgraph.com/blog/benchmark-memgraph-or-neo4j-with-benchgraph">Benchgraph</a>, a graph database performance benchmark, and feel free to <a href="https://memgraph.com/contact-us">contact us</a> about making the switch.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a209145f1069" width="1" height="1" alt=""><hr><p><a href="https://medium.com/memgraph/memgraph-vs-tigergraph-a209145f1069">Memgraph vs. TigerGraph</a> was originally published in <a href="https://medium.com/memgraph">Memgraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Visualize a Social Network in Python with a Graph Database: Flask + Docker + D3.js]]></title>
            <link>https://medium.com/memgraph/how-to-visualize-a-social-network-in-python-with-a-graph-database-flask-docker-d3-js-40f84940d4a?source=rss-ad8382481555------2</link>
            <guid isPermaLink="false">https://medium.com/p/40f84940d4a</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[flask]]></category>
            <category><![CDATA[memgraph]]></category>
            <category><![CDATA[d3js]]></category>
            <dc:creator><![CDATA[Memgraph]]></dc:creator>
            <pubDate>Fri, 18 Aug 2023 06:46:02 GMT</pubDate>
            <atom:updated>2023-08-18T06:46:02.603Z</atom:updated>
            <content:encoded><![CDATA[<p>When you think about a web application, a graph database doesn’t usually spring to mind. Instead, most people just take the familiar route of using an SQL database to store information. While this is perfectly acceptable for most use cases there are sometimes those that would see tremendous benefits by using a graph database. In this tutorial, I will show you how to make a basic web application using Flask that stores all of its information in a graph database. To be more precise we are using <strong>Memgraph DB</strong>, an in-memory database that can easily handle a lot of information and perform read/write instructions quite quickly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xYqa23LGXjtRuE5jb9araQ.png" /></figure><p>Our use case is a <strong>Social Network Graph</strong> (in the code referred to as <strong>SNG</strong> for convenience) representing users and the connections between them. Usually, such a graph would contain millions of relationships and the algorithms that are performed on them don’t do well with data being stored in relational databases.</p><p>In this tutorial, I will show you step by step how to build a simple Python web application from the bottom up so you get a basic understanding of the technologies that are used. You can also find all of the code <a href="https://github.com/g-despot/sng-demo">here</a> if you don’t want to work on it as you go through the tutorial. If at any point in this tutorial you have a question or something is not working for you, feel free to post on <a href="https://stackoverflow.com/questions/tagged/memgraphdb">StackOverflow</a> with the tag memgraphdb.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Wf97S6ZcdNG7PV9W.png" /></figure><h3>Prerequisites</h3><p>Because we are building a complete web application there is a number of tools that you will need to install before we begin:</p><ul><li><a href="https://python-poetry.org/docs/"><strong>Poetry</strong></a>: a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you.</li><li><a href="https://flask.palletsprojects.com/en/1.1.x/quickstart/"><strong>Flask</strong></a>: a very powerful web framework that provides you with tools, libraries and technologies used in web development. A Flask application can be as small as a single web page or as complex as a management interface.</li><li><a href="https://docs.docker.com/get-docker/"><strong>Docker and Compose</strong></a>: an open platform for developing, shipping, and running applications. It enables us to separate our application from our infrastructure (host machine). If you are installing Docker on Windows, Compose will be already included. For Linux and macOS visit <a href="https://docs.docker.com/compose/install/">this site</a>.</li><li><a href="https://docs.memgraph.com/memgraph/quick-start"><strong>Memgraph DB</strong></a>: a native fully distributed in-memory graph database built to handle real-time use-cases at enterprise scale. Follow the <strong>Docker Installation</strong> instructions on the Quick Start page. While it’s completely optional, I encourage you to also install <a href="https://memgraph.com/product/lab"><strong>Memgraph Lab</strong></a> so you can execute <strong>openCypher</strong> queries on the database directly and see visualized results.</li></ul><h3>Creating the Project Structure and Handling Dependencies</h3><p>Sometimes standard packaging systems and dependency management in Python can be confusing for beginners so we decided to use Poetry.<br>To start building our project structure choose a working directory and run:</p><pre>poetry new sng-demo</pre><p>Now you should have a directory with the following content:</p><pre>sng-demo<br>├── pyproject.toml<br>├── README.rst<br>├── sng_demo<br>│  └── __init__.py<br>└── tests<br>   ├── __init__.py<br>   └── test_poetry_demo.py</pre><p>In this tutorial, we won’t use the testing functionalities so go on ahead and delete the directory tests as well as the file README.rst.</p><p>Now we need to add the dependencies for our project. Given that we are going to run the app inside a Docker container we don’t need the dependencies installed locally, only inside the container. Copy the files <a href="https://github.com/g-despot/sng-demo/blob/master/pyproject.toml">project.toml</a> and <a href="https://github.com/g-despot/sng-demo/blob/master/poetry.lock">poetry.lock</a> and place them in the root directory of the project. The only other thing we need to do about dependency management is to tell Docker how to run Poetry on startup so it can install/update all the necessary dependencies inside the container.</p><h3>Dockerizing an Application</h3><p>In the root directory of the project create two files, Dockerfile and docker-compose.yml. At the beginning of the Dockerfile, we specify the Python version and instruct the container to install <strong>CMake</strong>, <strong>poetry</strong>, <strong>mgclient</strong> and <strong>pymgclient</strong>. Poetry is necessary to manage our dependencies inside the container while CMake and mgclient are required for pymgclient, the Python driver for <strong>Memgraph DB</strong>.<br>You don’t have to focus too much on this part just copy the code to your Dockerfile:</p><pre>FROM python:3.7<br> <br>#Install CMake<br>RUN apt-get update &amp;&amp; \<br>  apt-get --yes install cmake<br> <br>#Install poetry<br>RUN pip install -U pip \<br>  &amp;&amp; curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python<br>ENV PATH=&quot;${PATH}:/root/.poetry/bin&quot;<br> <br>#Install mgclient<br>RUN apt-get install -y git cmake make gcc g++ libssl-dev &amp;&amp; \<br>  git clone https://github.com/memgraph/mgclient.git /mgclient &amp;&amp; \<br>  cd mgclient &amp;&amp; \<br>  git checkout 5ae69ea4774e9b525a2be0c9fc25fb83490f13bb &amp;&amp; \<br>  mkdir build &amp;&amp; \<br>  cd build &amp;&amp; \<br>  cmake .. &amp;&amp; \<br>  make &amp;&amp; \<br>  make install<br> <br>#Install pymgclient<br>RUN git clone https://github.com/memgraph/pymgclient /pymgclient &amp;&amp; \<br>  cd pymgclient &amp;&amp; \<br>  python3 setup.py build &amp;&amp; \<br>  python3 setup.py install</pre><p>Next, we define the working directory with:</p><pre>WORKDIR /app<br>COPY poetry.lock pyproject.toml /app/</pre><p>The second command will enable us to cache the project requirements and only reinstall them when pyproject.toml or poetry.lock are changed.</p><pre>RUN poetry config virtualenvs.create false &amp;&amp; \<br>  poetry install --no-interaction --no-ansi</pre><p>We don’t need to create a virtual environment because our application is already isolated by being in a Docker container. To disable it virtualenvs.create needs to be set to false.<br>The second line in the command ensures that Poetry asks us no interactive questions while installing/updating dependencies and it makes the output more log friendly.</p><pre>COPY . /app<br>EXPOSE 5000<br>ENTRYPOINT [ &quot;poetry&quot;, &quot;run&quot; ]</pre><p>This is where we essentially create all the directories and files inside of our container. The EXPOSE command informs Docker that the container listens on the specified network port at runtime.</p><p>Next, we need to create a docker-compose.yml file. <strong>Compose</strong> is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. For our project, we need two services. One is the web application (sng_demo) and the other a database instance (memgraph).</p><p>If you followed the instructions on <a href="https://docs.memgraph.com/memgraph/quick-start">how to setup Memgraph DB with Docker</a> correctly you only need to add the following code to your docker-compose.yml file to run the container:</p><pre>version: &#39;3&#39;<br>services:<br>  memgraph:<br>    image: &quot;memgraph&quot;<br>    ports:<br>      - &quot;7687:7687&quot;<br>  sng_demo:<br>    build: .<br>    volumes:<br>      - .:/app<br>    ports:<br>      - &quot;5000:5000&quot;<br>    environment:<br>      MG_HOST: memgraph<br>      MG_PORT: 7687<br>    depends_on:<br>      - memgraph</pre><p>When it comes to the ports key, there is an important distinction between the <strong>HOST_PORT</strong> and the <strong>CONTAINER_PORT</strong>. The first number in the key is the <strong>HOST_PORT</strong> and it can be used to connect from your host machine to the service (for example with <strong>Memgraph Lab</strong>). The second number specifies the <strong>CONTAINER_PORT</strong> which is used for service-to-service communication. More precisely, our service sng_db can use this port to access the service memgraph and connect to the database.</p><p>The environment key contains MG_HOST and MG_PORT which represent environment variables in the service’s container. They store the memgraph service address and port which are needed to establish a database connection. The depends_on key is used to start services in dependency order because we need the database to start before the web application.</p><p>The build key allows us to tell Compose where to find the build instructions as well as the files and/or folders used during the build process. By using the volumes key, we bypass the need to constantly restart our image to load new changes to it from the host machine.</p><p>Finally, we have a dockerized project that utilizes Poetry! This approach is great for development because it enables us to run our project on completely different operating systems and environments without having to worry about compatibility issues.</p><h3>Web Development with Flask</h3><p>Flask is very simple to use so why not create a <strong>Hello World!</strong> page to try out our Docker+Poetry setup.<br>In the project root directory create a file called app.py with the following code:</p><pre>from flask import Flask<br> <br>app = Flask(__name__)<br> <br> <br>@app.route(&#39;/&#39;)<br>@app.route(&#39;/index&#39;)<br>def index():<br>   return &quot;Hello World&quot;</pre><p>First, we imported the Flask class and then created an instance of it. The route() decorator tells Flask what URL should trigger our function. Now, we need to tell Docker how to run our app. This can be done by creating a simple script in the project root directory. Let’s call it start.sh:</p><pre>#!/bin/bash<br>export FLASK_APP=app.py<br>export FLASK_ENV=development<br>flask run --host 0.0.0.0</pre><p>Setting FLASK_ENV to development will enable the debug mode. This makes Flask use an interactive debugger and reloader.<br>Setting FLASK_APP to app.py specifies how to start the application.<br>We need to tell Docker when and how to run this script so put the following code in your Dockerfile after the line EXPOSE 5000 :</p><pre>ADD start.sh /<br>RUN chmod +x /start.sh</pre><p>The command chmod +x makes the script executable by setting the right permission.<br>To execute the script, add the following command after the line ENTRYPOINT [ &quot;poetry&quot;, &quot;run&quot; ]:</p><pre>CMD [&quot;/start.sh&quot;]</pre><p>That’s it! Our first web page is ready so let’s start our app to make sure we don’t have any errors.<br>In the project root directory execute:</p><pre>docker-compose build</pre><p>The first build will take some time because Docker has to download and install a lot of dependencies.<br>After it finishes run:</p><pre>docker-compose up</pre><p>The URL of our web application is <a href="http://localhost:5000/">http://localhost:5000/</a>. When you open it there should be a message <strong>Hello World!</strong> which means that the app is up and running.</p><p>Now it’s time to create a more complex web page that will contain our Social Network Graph. In the project root directory create a folder called templates and in it a file with the name base.html. This will be our base HTML template for other pages. Copy the code:</p><pre>&lt;!doctype html&gt;<br>&lt;html lang=&quot;en&quot;&gt;<br> <br>&lt;head&gt;<br>    &lt;meta charset=&quot;utf-8&quot;&gt;<br>    &lt;meta name=&quot;viewport&quot; content=&quot;width=device-width, initial-scale=1, shrink-to-fit=no&quot;&gt;<br> <br>    &lt;link rel=&quot;stylesheet&quot; href=&quot;https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css&quot;&gt;<br>    &lt;link rel=&quot;stylesheet&quot; href=&quot;/static/css/style.css&quot;&gt;<br> <br>    &lt;script src=&quot;https://code.jquery.com/jquery-3.3.1.slim.min.js&quot;&gt;&lt;/script&gt;<br>    &lt;script src=&quot;https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js&quot;&gt;&lt;/script&gt;<br>    &lt;script src=&quot;https://d3js.org/d3.v4.min.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;<br> <br>    &lt;title&gt;Social Network Graph Demo&lt;/title&gt;<br>&lt;/head&gt;<br> <br>&lt;body&gt;<br>    {% block content %} {% endblock %}<br>&lt;/body&gt;<br> <br>&lt;/html&gt;</pre><p>We also need to create an HTML file for our actual landing site that utilizes this base file and an accompanying JavaScript file. Create the HTML file in the same location with the name index.html and copy the following code into it:</p><pre>{% extends &#39;base.html&#39; %} {% block content %}<br>&lt;div class=&quot;container&quot;&gt;<br>  Hello World!<br>&lt;/div&gt;<br>&lt;script src=&quot;/static/js/index.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;<br>{% endblock %}</pre><p>In the project root directory create a folder called static with one subfolder called js and another called css. The js folder will contain all of the needed local JavaScript files while the css folder will contain all the CSS stylesheets. In the js folder create a file called index.js and in the css folder one called style.css. Just leave them empty for now.</p><p>If you want to find out more about web development with Flask I suggest you try out <a href="https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world">this tutorial</a>.<br>Your current project structure should like this:</p><pre>sng-demo<br>├── sng_demo<br>│  └── __init__.py<br>├── templates<br>│  ├── base.html<br>│  └── index.html<br>├── static<br>│  ├── css<br>│  │  └── style.css<br>│  └── js<br>│     └── index.js<br>├── app.py<br>├── docker-compose.yml<br>├── Dockerfile<br>├── poetry.lock<br>├── pyproject.toml<br>└── start.sh</pre><h3>The Data Model and Database Connection</h3><p>In the app directory sng-demo create a folder called database. This folder will contain all of the modules that we need to communicate with the database. You can find them <a href="https://github.com/g-despot/sng-demo/tree/master/sng_demo/database">here</a> and just copy their contents. They are closely related to the database driver and if you wish to examine them a bit more I suggest you look up the driver documentation <a href="https://github.com/memgraph/pymgclient">here</a>. In the app directory sng-demo create the module db_operations.py. This is where all the custom database related commands will be located.<br>The sng_demo directory should look like this:</p><pre>sng_demo<br>├── __init__.py<br>├── db_operations.py<br>└── database<br>   ├── __init__.py<br>   ├── memgraph.py<br>   ├── connection.py<br>   └── models.py</pre><p>We will use a very simple data model that can be easily upgraded later on.<br>There is only one node with the label User and each User has two properties, a numerical id and a string name. Nodes are connected with edges of the type FRIENDS:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/737/0*eIgFWN6Hv7dyJg77.png" /></figure><p>There are several methods to populate our database (<a href="https://docs.memgraph.com/memgraph/how-to-guides-overview/import-data">more on that here</a>) but we will be doing it manually by executing <strong>openCypher</strong> queries so you can get a better understanding of how to communicate with the database. You will find all the necessary queries to populate the database in the files <a href="https://github.com/g-despot/sng-demo/blob/master/resources/data_big.txt">data_big.txt</a> and <a href="https://github.com/g-despot/sng-demo/blob/master/resources/data_small.txt">data_small.txt</a>. The former just has a larger dataset than the latter.<br>In the project root directory create a folder called resources and place the files in it. Now you can add an import method to your web application.<br>In the module db_operations.py add the following import and method:</p><pre>import json<br> <br>def clear(db):<br>  command = &quot;MATCH (node) DETACH DELETE node&quot;<br>  db.execute_query(command)<br> <br>def populate_database(db, path):<br>  file = open(path)<br>  lines = file.readlines()<br>  file.close()<br>  for line in lines:<br>      if len(line.strip()) != 0 and line[0] != &#39;/&#39;:<br>          db.execute_query(line)</pre><p>The method clear() deletes any data that might have been left in the database before populating it.<br>The method populate_database() reads all of the <strong>openCypher</strong> queries in the specified file and executes them.<br>In the module app.py change the imports and method index() to:</p><pre>from flask import Flask, render_template, request, jsonify, make_response<br>from sng_demo.database import Memgraph<br>from sng_demo import db_operations<br> <br>app = Flask(__name__)<br> <br> <br>@app.route(&#39;/&#39;)<br>@app.route(&#39;/index&#39;)<br>def index():<br>    db = Memgraph()<br>    db_operations.clear(db)<br>    db_operations.populate_database(db, &quot;resources/data_small.txt&quot;)<br>    return render_template(&#39;index.html&#39;)</pre><p>Now every time we refresh our index page the database is cleared and repopulated with new data. While this is not suitable for the production stage, it is highly useful during development because it will enable us to make changes in the data without having to restart the whole application or working directly on the database.<br>If you want to examine the graph before proceeding, I suggest you open <strong>Memgraph Lab</strong> and run the query MATCH (n1)-[e:FRIENDS]-(n2) RETURN n1,n2,e;.<br>The result should be:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nNsnsho4q5P0qAi0.png" /></figure><p>We also need a method in our app to fetch all the relevant data from the database when a client requests it.<br>Let’s call it get_graph() and place it in the db_operations.py module:</p><pre>def get_graph(db):<br>   command = &quot;MATCH (n1)-[e:FRIENDS]-(n2) RETURN n1,n2,e;&quot;<br>   relationships = db.execute_and_fetch(command)<br> <br>   link_objects = []<br>   node_objects = []<br>   added_nodes = []<br>   for relationship in relationships:<br>       e = relationship[&#39;e&#39;]<br>       data = {&quot;source&quot;: e.nodes[0], &quot;target&quot;: e.nodes[1]}<br>       link_objects.append(data)<br> <br>       n1 = relationship[&#39;n1&#39;]<br>       if not (n1.id in added_nodes):<br>           data = {&quot;id&quot;: n1.id, &quot;name&quot;: n1.properties[&#39;name&#39;]}<br>           node_objects.append(data)<br>           added_nodes.append(n1.id)<br> <br>       n2 = relationship[&#39;n2&#39;]<br>       if not (n2.id in added_nodes):<br>           data = {&quot;id&quot;: n2.id, &quot;name&quot;: n2.properties[&#39;name&#39;]}<br>           node_objects.append(data)<br>           added_nodes.append(n2.id)<br>   data = {&quot;links&quot;: link_objects, &quot;nodes&quot;: node_objects}<br> <br>   return json.dumps(data)</pre><p>First, we need to execute the <strong>openCypher</strong> query MATCH (n1)-[e:FRIENDS]-(n2) RETURN n1,n2,e; and return its results from the database. These results will contain all the edges in the graph as well as all the nodes that are connected to those edges. Nodes that don’t have connections will not be returned and that’s ok for now.</p><p>The results (the object relationships) are in the form of a generator which we can iterate over and access its contents by using the node/edge names specified in our initial query (n1,n2 and e).<br>We also need to check if a node has already been appended to the node_objects list because multiple edges can contain (point to or from) the same node. All of the objects are stored in key-value pairs suitable for later JSON conversion.<br>The final result is a JSON object containing:</p><ul><li>links: all the relationships that are in the graph as pairs of source and target id properties,</li><li>nodes: all the nodes from the graph that form relationships with other nodes.</li></ul><p>In your app.py module add the following method:</p><pre>@app.route(&quot;/get-graph&quot;, methods=[&quot;POST&quot;])<br>def get_graph():<br>   db = Memgraph()<br>   response = make_response(<br>       jsonify(db_operations.get_graph(db)), 200)<br>   return response</pre><p>This method is responsible for responding to POST requests from the client. It returns the graph data that we fetched from the server in the previous method.</p><p>Now let’s do something with this data! Copy the contents for your index.js file <a href="https://github.com/g-despot/sng-demo/blob/master/static/js/index.js">from here</a> and the style.css file <a href="https://github.com/g-despot/sng-demo/blob/master/static/css/style.css">from here</a>.<br>We also need to add the actual SVG graphic to our page so change the index.html file to:</p><pre>{% extends &#39;base.html&#39; %} {% block content %}<br>&lt;div class=&quot;container&quot;&gt;<br>    &lt;svg class=&quot;border rounded mt-3&quot; width=&quot;960&quot; height=&quot;600&quot; style=&quot;background-color:white&quot;&gt;&lt;/svg&gt;<br>&lt;/div&gt;<br>&lt;script src=&quot;/static/js/index.js&quot; charset=&quot;utf-8&quot;&gt;&lt;/script&gt;<br>{% endblock %}</pre><p>I won’t go into much detail about how to use <strong>D3.js</strong> so if you want to find out more I encourage you to visit <a href="https://d3js.org/">their website</a>.</p><p>In short, we fetch all the nodes and edges from the database and add them to an SVG element. The visual representation of the graph is made by simulating how physical forces act on particles (charge and gravity). You can drag and drop the nodes, hover over them to see the value of their name property, zoom in and out of the graph and move the SVG graphic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*-m3esvx4_91ZE7DQ.png" /></figure><h3>Additional Functionalities</h3><p>Go ahead and copy the file <a href="https://github.com/g-despot/sng-demo/blob/master/static/js/query.js">query.js</a> to the directory static/js and <a href="https://github.com/g-despot/sng-demo/blob/master/templates/query.html">query.html</a> to the directory templates. You can find the updated base.html file <a href="https://github.com/g-despot/sng-demo/blob/master/templates/base.html">here</a>. Copy the necessary methods from the <a href="https://github.com/g-despot/sng-demo/blob/master/sng_demo/db_operations.py">db_operations.py</a> module and <a href="https://github.com/g-despot/sng-demo/blob/master/app.py">app.py</a> module.<br>After you made the changes, just open <a href="http://localhost:5000/query/">http://localhost:5000/query/</a> and see the results.<br>This page will make your life easier if you want to debug the data being fetched from the server. It returns all the nodes or edges and shows them in a JSON highlighted format.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/0*XPBN7UIu0VexWIhB.png" /></figure><p>Your current project structure should like this:</p><pre>sng-demo<br>├── resources<br>│  ├── data_big.py<br>│  └── data_small.txt<br>├── sng_demo<br>│  ├── __init__.py<br>│  ├── db_operations.py<br>│  └── database<br>│     ├── __init__.py<br>│     ├── memgraph.py<br>│     ├── connection.py<br>│     └── models.py<br>├── templates<br>│  ├── base.html<br>│  ├── index.html<br>│  └── query.html<br>├── static<br>│   ├── css<br>│   │  └── style.css<br>│   └── js<br>│      ├── index.js<br>│      └── query.js<br>├── app.py<br>├── docker-compose.yml<br>├── Dockerfile<br>├── poetry.lock<br>├── pyproject.toml<br>└── start.sh</pre><h3>Conclusion</h3><p>Even though graph databases have been around for a long time, they are still not considered a mainstream tool in software development. <strong>Relational database-management systems</strong> model data as a set of predetermined structures. Complex joins and self-joins are necessary when the dataset becomes too inter-related. Modern datasets require technically complex queries which are often very inefficient in real-time scenarios.</p><p><strong>Graph databases</strong> offer powerful data modelling and analysis capabilities for many real-world problems such as social networks, business relationships, dependencies, shipping, logistics… and they have been adopted by many of the worlds leading tech companies. With this tutorial, I hope to shed some light on how easy it is to integrate a graph database in your development process and I encourage you to try it out yourself.</p><p>As I said at the beginning, feel free to ask us any questions about this tutorial or Memgraph in general on <a href="https://stackoverflow.com/questions/tagged/memgraphdb">StackOverflow</a> with the tag memgraphdb or on our <a href="https://discord.gg/memgraph">Discord server</a>. <strong>Good luck with your coding!</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=40f84940d4a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/memgraph/how-to-visualize-a-social-network-in-python-with-a-graph-database-flask-docker-d3-js-40f84940d4a">How to Visualize a Social Network in Python with a Graph Database: Flask + Docker + D3.js</a> was originally published in <a href="https://medium.com/memgraph">Memgraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is a Key-Value Database?]]></title>
            <link>https://medium.com/memgraph/what-is-a-key-value-database-1deb5b7444aa?source=rss-ad8382481555------2</link>
            <guid isPermaLink="false">https://medium.com/p/1deb5b7444aa</guid>
            <category><![CDATA[key-value-database]]></category>
            <category><![CDATA[nosql]]></category>
            <category><![CDATA[nosql-database]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Memgraph]]></dc:creator>
            <pubDate>Thu, 03 Aug 2023 09:15:40 GMT</pubDate>
            <atom:updated>2023-08-03T09:16:17.438Z</atom:updated>
            <content:encoded><![CDATA[<p>In the landscape of data management solutions, key-value databases stand apart, offering a unique blend of simplicity and performance tailored to the big data era. By abstracting data into simple key-value pairs and values, this type of database offers an intuitive and flexible data model that can seamlessly scale to accommodate large datasets. Before delving into the complexities of key-value databases, their advantages, use cases, and potential drawbacks, let’s first establish a foundational understanding of these databases and how they represent a distinctive paradigm in data storage and retrieval.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pUCxLBF5qdorMblhratioA.png" /></figure><h3>Understanding a key-value store</h3><p>In the <a href="https://memgraph.com/blog/how-to-choose-a-database-for-your-needs">ever-expanding world of databases</a>, key-value databases have emerged as a vital element. They are designed for storing, retrieving, and managing associative arrays, a data structure commonly known as a dictionary or hash table. The dictionary contains a collection of records, each with different fields containing data. These records are stored and retrieved using a unique key. This unique approach to data management differs from traditional relational databases and makes a key-value database an optimal solution for use cases where simplicity, speed, and scalability are paramount. Key-value databases are the <a href="https://memgraph.com/blog/types-of-nosql-databases-deep-dive">simplest form of NoSQL databases</a>. Each item in the database is stored as an attribute name, or key, together with its value.</p><h3>Defining key-value stores</h3><p>A key-value database, or key-value store, uses a simple key-value method to store data. Each key-value pair represents a specific piece of data. The ‘key’ serves as a unique identifier that is used to find the data within the database. Key-value databases treat the data as a single opaque collection, which may have different fields for every record. This offers considerable flexibility and more closely aligns with modern concepts like object-oriented programming.</p><p>The primary features of a key-value database include simplicity, high performance, and the ability to handle large volumes of data efficiently. Because placeholders or input parameters do not represent optional values, key-value databases often use far less memory to store the same data, which can lead to significant performance gains in certain workloads.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HV-dPaNr5QCJlRaj.png" /></figure><h3>Working with a key-value database</h3><p>Key-value databases are known for their ease of use, making them a good starting point for beginners in the database realm. Thanks to their straightforward data structure, they provide a simple, user-friendly interface. The process involves storing unique keys associated with their corresponding data values and retrieving the associated value using its unique key.</p><p>Key-value databases excel in <a href="https://databasetown.com/key-value-database-use-cases/">use cases that involve frequently accessed data</a>. For instance, in e-commerce platforms, a key-value database could be used to store product details. The unique identifier or ‘key’ could be the product ID, with all the related product data stored as the associated ‘value’. Another scenario is user logs, where key-value stores prove to be efficient. Popular key-value database examples include Redis, DynamoDB, Riak, and RocksDB have active community support and extensive documentation, making it easier for developers to get started and troubleshoot issues.</p><h3>Benefits and advantages of a key-value database</h3><p>Developers and organizations <a href="https://www.geeksforgeeks.org/key-value-data-model-in-nosql/">choose key-value databases for several reasons</a>.</p><h3>Speed</h3><p>One of the main reasons is their speed. Key-value databases can handle large volumes of reads and writes with minimal latency. The simplicity of the data structures used in a key-value store makes it easy for developers to quickly store and retrieve data. They offer impressive speed and performance, handling large volumes of data and providing rapid, random data access.</p><h3>Horizontal scaling</h3><p>Scalability is a critical factor for modern applications dealing with ever-growing data volumes and increasing user demands. Key-value stores offer an advantage in this area, as they can easily scale horizontally. Horizontal scaling involves adding more servers to the existing network to distribute the data and workload across multiple nodes. As a result, key-value databases can maintain their performance levels even as data grows, ensuring that applications can handle a high number of concurrent users and massive data storage needs. This ability to scale out effectively makes them a preferred choice for applications that require elasticity and flexibility.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pSBJOsEARE_T5p3U.png" /></figure><h3>Ease of use and development</h3><p>Key-value databases provide a simple and straightforward interface for data storage and retrieval. Developers can easily interact with the database using basic operations like “get” and “put” based on the associated keys. The straightforward API and data model reduce the complexity of the application code, making it easier to develop and maintain. Additionally, the ease of integration with various programming languages and frameworks allows developers to quickly incorporate key-value stores into their applications without a steep learning curve.</p><h3>Flexible data models</h3><p>Key-value databases are schema-less, meaning that they do not enforce a fixed data structure or data types. This flexibility allows developers to store and manage various types of data within the same database without strict predefined schemas. This is particularly advantageous for applications dealing with diverse and evolving data formats. Whether it’s storing user profiles, session data, configurations, or complex objects, key-value databases can handle different data formats efficiently, eliminating the need for complex and costly data transformations.</p><h3>Potential drawbacks of a key-value database</h3><p>While key-value databases are powerful, they <a href="https://www.techtarget.com/searchdatamanagement/tip/NoSQL-database-types-explained-Key-value-store">aren’t always the ideal choice</a>.</p><h3>Complex queries</h3><p>They lack the ability to perform complex queries or handle sophisticated relationships between data, which relational databases excel at. If your use case involves complex queries or requires understanding the relationships between different data entities, then a relational database or a graph database might be a better fit. Also, they may not be suitable for applications that require multi-record transactions or complex data manipulation.</p><h3>Lack of data relationships</h3><p>Key-value databases do not inherently support relationships between data items. While data can be organized and retrieved efficiently based on keys, establishing complex relationships between different pieces of data requires additional application logic. This can lead to potential data inconsistencies and increased complexity in managing relationships outside the database layer. For applications heavily reliant on data relationships, such as social networks or recommendation systems, graph databases may provide more natural and performant solutions, as they are specifically designed to handle and traverse complex data relationships efficiently.</p><h3>Handling complex data types</h3><p>Another challenge with key-value databases is the handling of complex data types. While they can handle structured, semi-structured, and unstructured data, they lack the advanced capabilities of a document database in managing complex objects.</p><h3>Data size and indexing</h3><p>As the data size grows significantly, key-value databases may face challenges with indexing and maintaining performance. While they excel at handling fast key-based lookups, large datasets can impact the efficiency of these operations. Proper index design and tuning are crucial to ensure optimal performance. Additionally, some key-value databases may not offer sophisticated indexing options, limiting their capabilities to efficiently handle specific types of queries or data access patterns.</p><h3>Overcoming limitations of a key-value database</h3><p>Despite these limitations, there are strategies and tools that can help overcome them. For example, secondary indexing and query languages like Redis’s RediSearch and Amazon DynamoDB’s Query Language can help perform more complex queries. Hybrid models, combining a key-value database with other database types like document databases, can handle complex queries or manage complex relationships between data entities. Additional technologies that can be incorporated into key-value databases to enhance functionality include secondary indexing to create efficient index structures to accelerate application responses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-QDPacXciJkLv5Io.png" /></figure><h3>Takeaways</h3><p>Key-value databases offer a compelling mix of simplicity, speed, and scalability that can be highly beneficial in certain situations. As with any technology, it’s important to understand its strengths and weaknesses to make the most of it. The key-value database world is constantly evolving, and it’s an exciting area to explore for any developer or organization looking to optimize their data storage and retrieval needs.</p><p>They are a fantastic choice for use cases that involve handling large volumes of frequently accessed data or where horizontal scalability is required. However, it’s crucial to understand the capabilities and limitations of key-value databases before choosing them as a solution. Overall, with ongoing evolution and innovation in the database world, the future looks promising for the key-value database scene.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1deb5b7444aa" width="1" height="1" alt=""><hr><p><a href="https://medium.com/memgraph/what-is-a-key-value-database-1deb5b7444aa">What is a Key-Value Database?</a> was originally published in <a href="https://medium.com/memgraph">Memgraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When to Use a NoSQL Database]]></title>
            <link>https://medium.com/memgraph/when-to-use-a-nosql-database-5fa9afebacdd?source=rss-ad8382481555------2</link>
            <guid isPermaLink="false">https://medium.com/p/5fa9afebacdd</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[nosql]]></category>
            <category><![CDATA[memgraph]]></category>
            <dc:creator><![CDATA[Memgraph]]></dc:creator>
            <pubDate>Fri, 21 Jul 2023 13:52:13 GMT</pubDate>
            <atom:updated>2023-07-21T13:52:13.210Z</atom:updated>
            <content:encoded><![CDATA[<p>How we manage and process data changes rapidly over the years, and staying on track with the newest technology trends is important. Traditional relational database management systems, such as MySQL, Oracle, and SQL Server, have always been the first choice to maintain data integrity and for fast querying, but with the rise of big data, NoSQL (Not Only SQL) databases emerged and became more popular due to their speed, flexibility, and scalability. Now, the question that pops up naturally is when is the right time to use a NoSQL database?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qfR9IDGmBwN7zxSvWAv8gg.png" /></figure><p>This blog post will explore NoSQL databases and how they differ from traditional databases. Besides, you will learn when and why to utilize NoSQL databases. Whether you’re a developer, data engineer, or decision maker, this blog post will provide insights into modern databases that will come in handy to boost your projects.</p><h3>Understanding NoSQL databases</h3><p>NoSQL databases are non-relational databases with flexible schema designed for high performance at a massive scale. Unlike traditional relational databases, which use tables and predefined schemas, NoSQL databases use a variety of data models. There are 4 main types of NoSQL databases — document, graph, key-value, and column-oriented databases. NoSQL databases generally are well-suited for unstructured data, large-scale applications, and agile development processes. The most popular examples of NoSQL databases are <a href="https://www.mongodb.com/">MongoDB</a> (document), <a href="https://memgraph.com/">Memgraph</a> (graph), <a href="https://redis.io/">Redis</a> (key-value store) and <a href="https://hbase.apache.org/">Apache HBase</a> (column-oriented).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mzvAebX2JX6zqkzK.png" /></figure><p>These <a href="https://memgraph.com/blog/types-of-nosql-databases-deep-dive">types of NoSQL databases</a> store data in their own unique ways offering pros and cons, which are presented below:</p><h3>Advantages</h3><ul><li><strong>Flexibility</strong>: If you’ve used an SQL database before, you are probably familiar with its strict schema. That can be a hassle with data rapidly changing. NoSQL databases, with their dynamic and flexible schemas, have a huge advantage in handling unstructured and semi-structured data, making an excellent choice for diverse data types.</li><li><strong>Scalability</strong>: Data volumes can grow exponentially, and scalability becomes crucial, especially with the rise of big data. Many NoSQL databases offer horizontal scalability, allowing you to add more servers to increase the capacity. On the other hand, SQL databases usually scale only vertically.</li><li><strong>Performance</strong>: Many NoSQL databases are optimized to deliver high performance, even when dealing with large data volumes or data streams.</li></ul><h3>Disadvantages</h3><ul><li><strong>Maturity</strong>: SQL (standing for Structured Query Language) databases have been popular in the database world for a long time, and because of that, they offer robustness, tools and community that NoSQL databases have not managed to match yet. That does not mean that NoSQL will not overtake over the years, just like it happens often with modern technologies.</li><li><strong>Complexity</strong>: The learning curve with NoSQL databases is steeper than with SQL databases, especially because there is no standard query language. This follows from the maturity and the fact that NoSQL databases are more complex to design, implement and manage. But, the results you gain by storing your data in a proper database might be worth it.</li><li><strong>ACID compliance</strong>: Opposite to NoSQL databases, many SQL databases are usually ACID-compliant. However, as an exception, Memgraph is a NoSQL graph database that is ACID compliant. If you’re unsure what that means for your application, you are invited to check out our recent blog on <a href="https://memgraph.com/blog/acid-transactions-meaning-of-isolation-levels">ACID transactions and isolation levels</a>.</li></ul><h3>5 reasons to choose a NoSQL database</h3><p>Based on the above-mentioned advantages and disadvantages of NoSQL databases, there are various scenarios where NoSQL databases are the optimal solution for your project. Let’s take a look at 5 scenarios where NoSQL databases are great:</p><h3>1. Handling large volumes of data at scale seamlessly</h3><p>NoSQL databases can handle large amounts of data by spreading it across multiple servers. Hence, the NoSQL database is a perfect solution if you have large amounts of growing data. You don’t have to worry that you don’t have enough resources to scale vertically; instead, you should design your database system properly to get the most insights from your data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lf1jkWQBQIOCM3BS.png" /></figure><h3>2. Working easily with unstructured or semi-structured data</h3><p>Predefined schemas can be good sometimes, but if you’ve always found that a hassle for your unstructured or semi-structured data which is often changing, NoSQL databases are the correct approach. They are built with flexible data schema, which speeds up the development process and lowers the efforts on database management. Problems with data types like XML, JSON and others have become issues of the past with NoSQL databases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4DFrITb0vG4QxQbS.png" /></figure><h3>3. Enabling rapid development</h3><p>Again, because of the NoSQL flexibility, your data model can rapidly change, meaning you can update your application on the fly without schema updates. Nowadays, development is fast, and iterations are quick, so removing the task of updating schemas constantly saves you valuable time that can be spent on even faster development.</p><h3>4. High read/write speed</h3><p>If you’re working on applications that require real-time data processing, you might be stuck with traditional databases that don’t offer the required speed. NoSQL databases are optimized for high-speed read and write operations and are ideal for chat, IoT, gaming, fraud detection, and similar applications.</p><h3>5. Managing complex relationships</h3><p>Complex data relationships, such as parent-child or many-to-many relationships, arise in relational databases when data from different tables is related or somehow interconnected. Querying it requires hopping from one table to another and joining it with slow and resource-intensive join operations. A graph database is a type of NoSQL database that handles highly-connected data especially well and is a good choice for social networks, recommendation engines, fraud detection, or any application where you need to traverse through interconnected data points.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_xfcwAzRdULr2ELa.png" /></figure><h3>Concluding thoughts</h3><p>If you still don’t know whether you need an NoSQL or a relational database management system, we prepared <a href="https://memgraph.com/blog/how-to-choose-a-database-for-your-needs">another article</a> on that topic that may speed up your decision process. Keep in mind that it doesn’t always have to be a fight, SQL vs NoSQL, you can surely consider a hybrid approach and get the best out of both SQL and NoSQL databases. But if you did find that your use case fits the description above, then NoSQL is a database for you! We are always open to discussions on this and other topics in the database scene, so <a href="https://www.discord.gg/memgraph">join the conversation</a> on our Discord server to be a part of the community.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5fa9afebacdd" width="1" height="1" alt=""><hr><p><a href="https://medium.com/memgraph/when-to-use-a-nosql-database-5fa9afebacdd">When to Use a NoSQL Database</a> was originally published in <a href="https://medium.com/memgraph">Memgraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Build a Graph Web Application With Python, Flask, Docker & Memgraph]]></title>
            <link>https://medium.com/memgraph/how-to-build-a-graph-web-application-with-python-flask-docker-memgraph-f47f96d7fa35?source=rss-ad8382481555------2</link>
            <guid isPermaLink="false">https://medium.com/p/f47f96d7fa35</guid>
            <category><![CDATA[flask]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[flask-framework]]></category>
            <category><![CDATA[memgraph]]></category>
            <dc:creator><![CDATA[Memgraph]]></dc:creator>
            <pubDate>Thu, 20 Jul 2023 11:31:50 GMT</pubDate>
            <atom:updated>2023-07-20T11:31:50.319Z</atom:updated>
            <content:encoded><![CDATA[<p>The goal is straightforward (or at least it seems simple enough). Let’s build a web application in <strong>Python</strong> that can visualize a graph and run some cool graph algorithms out of the box. Maybe it’s not your flavor, but I prefer the <strong>Flask</strong> web framework for such occasions, so bear with me through this tutorial.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KBulZmIW70SlNHPyJE5bJA.png" /></figure><p>Now, I am going to show you an example of how to accomplish this. You can also take a look at the <a href="https://github.com/g-despot/graph-web-application">finished app</a> on GitHub if you want to see the complete code.</p><p>The general outline of the tutorial is:</p><ol><li><strong>Create a Flask server</strong></li><li><strong>Dockerize your application</strong></li><li><strong>Import the data into Memgraph</strong></li><li><strong>Query the database</strong></li></ol><p>Graph visualizations will be covered in <strong>part two</strong> of the tutorial so stay tuned! Spoiler alert, we are going to use D3.js to draw our graph.</p><h3>Prerequisites</h3><p>For this tutorial, you will need to install:</p><ul><li><a href="https://docs.docker.com/get-docker/">Docker</a></li><li><a href="https://docs.docker.com/get-docker/">Docker Compose</a> <em>(which is included with Docker on Windows and macOS)</em></li></ul><p>With Docker, we don’t need to worry about installing Python, Flask, Memgraph… essentially anything. Everything will be installed automatically and run smoothly inside Docker containers!</p><blockquote><em>Disclaimer: Docker fanboy alert</em></blockquote><h3>1. Create a Flask server</h3><p>I included comments in the code to make it more understandable, but if at any point you feel like something is unclear, <a href="https://memgr.ph/discord">join our Discord Server</a> and share your thoughts. First, create the file app.py with the following contents:</p><pre>import json<br>import logging<br>import os<br>from argparse import ArgumentParser<br>from flask import Flask, Response, render_template<br>from gqlalchemy import Memgraph<br> <br>log = logging.getLogger(__name__)<br> <br>def init_log():<br>    logging.basicConfig(level=logging.DEBUG)<br>    log.info(&quot;Logging enabled&quot;)<br>    # Set the log level for werkzeug to WARNING because it will print out too much info otherwise<br>    logging.getLogger(&quot;werkzeug&quot;).setLevel(logging.WARNING)</pre><p>Other than the imports, the first few lines focus on setting up the logging. No web application is complete without logging, so we will add the bare minimum and disable the pesky <em>werkzeug</em> logger, which sometimes prints too much info.</p><p>Now, let’s create an argument parser. This will enable you to easily change the behavior of the app on startup using arguments.</p><pre># Parse the input arguments for the app<br>def parse_args():<br>    &quot;&quot;&quot;<br>    Parse command line arguments.<br>    &quot;&quot;&quot;<br>    parser = ArgumentParser(description=__doc__)<br>    parser.add_argument(&quot;--host&quot;, default=&quot;0.0.0.0&quot;, help=&quot;Host address.&quot;)<br>    parser.add_argument(&quot;--port&quot;, default=5000, type=int, help=&quot;App port.&quot;)<br>    parser.add_argument(&quot;--template-folder&quot;, default=&quot;public/template&quot;, help=&quot;Flask templates.&quot;)<br>    parser.add_argument(&quot;--static-folder&quot;, default=&quot;public&quot;, help=&quot;Flask static files.&quot;)<br>    parser.add_argument(&quot;--path-to-input-file&quot;, default=&quot;graph.cypherl&quot;, help=&quot;Graph input file.&quot;)<br>    parser.add_argument(&quot;--debug&quot;, default=True, action=&quot;store_true&quot;, help=&quot;Web server in debug mode.&quot;)<br>    print(__doc__)<br>    return parser.parse_args()<br> <br>args = parse_args()</pre><p>It’s time to create your server instance:</p><pre># Create the Flask server instance<br>app = Flask(<br>    __name__,<br>    template_folder=args.template_folder,<br>    static_folder=args.static_folder,<br>    static_url_path=&quot;&quot;,<br>)</pre><p>You can finally create the view functions that will be invoked from the browser via HTTP requests. In layman’s terms, the homepage is called by:</p><pre># Retrieve the home page for the app<br>@app.route(&quot;/&quot;, methods=[&quot;GET&quot;])<br>def index():<br>    return render_template(&quot;index.html&quot;)</pre><p>The only thing that’s left is to implement and call the main() function:</p><pre># Entrypoint for the app that will be executed first<br>def main():<br>    # Code that should only be run once<br>    if os.environ.get(&quot;WERKZEUG_RUN_MAIN&quot;) == &quot;true&quot;:<br>        init_log()<br>    app.run(host=args.host,<br>            port=args.port,<br>            debug=args.debug)<br> <br>if __name__ == &quot;__main__&quot;:<br>    main()</pre><p>The somewhat strange statement os.environ.get(&quot;WERKZEUG_RUN_MAIN&quot;) == &quot;true&quot;</p><p>will make sure that this code is only executed once. Confused? A problem arises when working with Flask in development mode because each code change triggers a reload of the server, which in turn could result in parts of your code executing multiple times (for example, like the main function).</p><p>So, if you need to execute something only once in Flask at the beginning like loading data, this is the perfect place for it.</p><p>The next step is to create the following files, which we will work on in the next tutorial:</p><ul><li>index.html in public/template</li><li>index.js in public/js</li><li>style.css in public/css</li></ul><p>One more file is needed and this one will specify all the Python dependencies that need to be installed. Create requirements.txt with the following contents:</p><pre>gqlalchemy==1.0.6<br>Flask==2.0.2</pre><p>Your current project structure should look like this:</p><pre>app<br>├── public<br>│  ├── css<br>│  │   └── style.css<br>│  ├── js<br>│  │   └── index.js<br>│  └── templates<br>│      └── index.html<br>├── app.py<br>└── requirements.txt</pre><h3>2. Dockerize your application</h3><p>This is much simpler than you might think. Most often, you will need a Dockerfile in which you will specify how your Docker image should be created. Let’s take a look at our Dockerfile :</p><pre>FROM python:3.9<br> <br># Install CMake<br>RUN apt-get update &amp;&amp; \<br>  apt-get --yes install cmake &amp;&amp; \<br>  rm -rf /var/lib/apt/lists/*<br> <br># Install Python packages<br>COPY requirements.txt ./<br>RUN pip3 install -r requirements.txt<br> <br># Copy the source code<br>COPY public /app/public<br>COPY app.py /app/app.py<br>WORKDIR /app<br> <br># Set the environment variables<br>ENV FLASK_ENV=development<br>ENV LC_ALL=C.UTF-8<br>ENV LANG=C.UTF-8<br> <br># Start the web application<br>ENTRYPOINT [&quot;python3&quot;, &quot;app.py&quot;]</pre><p>The first line indicates that we are basing our image on a Linux image that has Python 3.9 preinstalled. The next step is to install CMake (which is needed for the Memgraph Python driver) with RUN and the standard Linux installation command apt-get ... .</p><p>We copy the requirements.txt file and install the Python packages with <strong>pip</strong>. The source code also needs to be copied to the image in order for us to start the web application. The ENTRYPOINT command is responsible for starting the desired process inside the container.</p><p>But we are not finished with Docker yet. We need to create a docker-compose.yml file that will tell Docker which containers to start.</p><pre>version: &quot;3&quot;<br>services:<br>  server:<br>    build: .<br>    volumes:<br>      - .:/app<br>    ports:<br>      - &quot;5000:5000&quot;<br>    environment:<br>      MEMGRAPH_HOST: memgraph<br>      MEMGRAPH_PORT: &quot;7687&quot;<br>    depends_on:<br>      - memgraph<br> <br>  memgraph:<br>    image: &quot;memgraph/memgraph&quot;<br>    ports:<br>      - &quot;7687:7687&quot;</pre><p>There are two services/containers in our app:</p><ol><li><strong>Server</strong>: Uses the Dockerfile to build a Docker image and run it.</li><li><strong>Memgraph</strong>: This is our database. Docker will automatically download the image and start it.</li></ol><p>Because we are supplying environment variables, let’s load them in app.py right after the imports:</p><pre>MEMGRAPH_HOST = os.getenv(&quot;MEMGRAPH_HOST&quot;, &quot;memgraph&quot;)<br>MEMGRAPH_PORT = int(os.getenv(&quot;MEMGRAPH_PORT&quot;, &quot;7687&quot;))</pre><p>Your current project structure should look like this:</p><pre>app<br>├── public<br>│  ├── css<br>│  │   └── style.css<br>│  ├── js<br>│  │   └── index.js<br>│  └── templates<br>│      └── index.html<br>├── app.py<br>├── docker-compose.yml<br>├── Dockerfile<br>└── requirements.txt</pre><p>Now, we can even start our app with the following commands:</p><pre>docker-compose build<br>docker-compose up</pre><h3>3. Import the data into Memgraph</h3><p>This task will be done inside the main() function because it only needs to be executed once:</p><pre>memgraph = None<br> <br>def main():<br>    if os.environ.get(&quot;WERKZEUG_RUN_MAIN&quot;) == &quot;true&quot;:<br>        init_log()<br>        global memgraph<br>        memgraph = Memgraph(MEMGRAPH_HOST,<br>                            MEMGRAPH_PORT)<br>        load_data(args.path_to_input_file)<br>    app.run(host=args.host,<br>            port=args.port,<br>            debug=args.debug)</pre><p>How do we import the data into Memgraph? I prepared a file with the Cypher queries that need to be executed in order to populate the database. You just need to download the file in your root directory and add the following load_data() function:</p><pre>def load_data(path_to_input_file):<br>    &quot;&quot;&quot;Load data into the database.&quot;&quot;&quot;<br>    try:<br>        memgraph.drop_database()<br>        with open(path_to_input_file, &quot;r&quot;) as file:<br>            for line in file:<br>                memgraph.execute(line)<br>    except Exception as e:<br>        log.info(f&quot;Data loading error: {e}&quot;)</pre><p>First, we clear everything in the database, and then we go over each line in the file graph.cypherl and execute them. And that’s it. Once we start the web application, Memgraph will import the dataset.</p><h3>4. Query the database</h3><p>We will create a function that will execute a Cypher query and return the results. It returns the whole graph, but we will limit ourselves to 100 nodes:</p><pre>def get_graph():<br>    results = memgraph.execute_and_fetch(<br>        f&quot;&quot;&quot;MATCH (n)-[]-(m)<br>                RETURN n as from, m AS to<br>                LIMIT 100;&quot;&quot;&quot;<br>    )<br>    return list(results)</pre><p>The view function get_data() which fetches all the nodes and relationships from the database, filters out the most important information, and returns it in JSON format for visualization. To can the network load at a minimum, you will send a list with every node id (no other information about the nodes) and a list that specifies how they are connected to each other.</p><pre>@app.route(&quot;/get-graph&quot;, methods=[&quot;GET&quot;])<br>def get_data():<br>    &quot;&quot;&quot;Load everything from the database.&quot;&quot;&quot;<br>    try:<br>        results = get_graph()<br> <br>        # Sets for quickly checking if we have already added the node or edge<br>        # We don&#39;t want to send duplicates to the frontend<br>        nodes_set = set()<br>        links_set = set()<br>        for result in results:<br>            source_id = result[&quot;from&quot;].properties[&#39;name&#39;]<br>            target_id = result[&quot;to&quot;].properties[&#39;name&#39;]<br> <br>            nodes_set.add(source_id)<br>            nodes_set.add(target_id)<br> <br>            if ((source_id, target_id) not in links_set and<br>                    (target_id, source_id,) not in links_set):<br>                links_set.add((source_id, target_id))<br> <br>        nodes = [<br>            {&quot;id&quot;: node_id}<br>            for node_id in nodes_set<br>        ]<br>        links = [{&quot;source&quot;: n_id, &quot;target&quot;: m_id} for (n_id, m_id) in links_set]<br> <br>        response = {&quot;nodes&quot;: nodes, &quot;links&quot;: links}<br>        return Response(json.dumps(response), status=200, mimetype=&quot;application/json&quot;)<br>    except Exception as e:<br>        log.info(f&quot;Data loading error: {e}&quot;)<br>        return (&quot;&quot;, 500)</pre><h3>What’s next?</h3><p>As you can see, it’s very easy to connect to Memgraph and run graph algorithms, even from a web application. While this part of the tutorial focused on the backend, in the next one, we will talk about graph visualizations and the D3.js framework.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f47f96d7fa35" width="1" height="1" alt=""><hr><p><a href="https://medium.com/memgraph/how-to-build-a-graph-web-application-with-python-flask-docker-memgraph-f47f96d7fa35">How to Build a Graph Web Application With Python, Flask, Docker &amp; Memgraph</a> was originally published in <a href="https://medium.com/memgraph">Memgraph</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>