<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Berk Torun on Medium]]></title>
        <description><![CDATA[Stories by Berk Torun on Medium]]></description>
        <link>https://medium.com/@berktorun.dev?source=rss-759f402bb906------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*B0Eis2epoSvR6KsWShF4oA.jpeg</url>
            <title>Stories by Berk Torun on Medium</title>
            <link>https://medium.com/@berktorun.dev?source=rss-759f402bb906------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 17:15:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@berktorun.dev/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Understanding Milvus: Key Concepts and Potential Applications]]></title>
            <link>https://medium.com/n11-tech/understanding-milvus-key-concepts-and-potential-applications-1fe4dedd7412?source=rss-759f402bb906------2</link>
            <guid isPermaLink="false">https://medium.com/p/1fe4dedd7412</guid>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[n11]]></category>
            <category><![CDATA[milvus]]></category>
            <category><![CDATA[data-insights]]></category>
            <category><![CDATA[vector-database]]></category>
            <dc:creator><![CDATA[Berk Torun]]></dc:creator>
            <pubDate>Mon, 02 Sep 2024 11:00:40 GMT</pubDate>
            <atom:updated>2024-09-02T11:00:40.097Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3U7e9xx86MVIMfAr" /></figure><p><em>Milvus is an advanced open-source vector database designed for handling complex data operations, particularly those involving unstructured data. This article delves into the underlying architecture of Milvus, exploring its key components such as the Access Layer, Coordinator Service, Worker Nodes. We will also examine the algorithms Milvus employs, including ANNOY and HNSW.</em></p><p><em>In the context of n11, we will discuss how Milvus can be utilized for specific scenarios, such as category prediction. By leveraging Milvus’s robust capabilities, n11 can enhance its data processing and analysis strategies, driving more accurate and scalable solutions.</em></p><h3>Key Concepts</h3><p>In this section, we’ll explore the foundational ideas behind Milvus, including the nature of unstructured data, the role of embedding vectors in representing this data, and how vector similarity search is used to find and analyze similar data points.</p><h4>Unstructured Data</h4><p>Unstructured data refers to information that doesn’t fit into a traditional, organized database structure. This includes formats like images, videos, audio files, and natural language text, which lack a predefined model or schema. Unstructured data is highly prevalent, making up approximately 80% of the world’s data. To make sense of this data, it can be transformed into vectors using artificial intelligence (AI) and machine learning (ML) models, enabling more advanced analysis and search capabilities.</p><h4><strong>Embedding Vectors</strong></h4><p>An embedding vector is a numerical representation of unstructured data, such as emails, IoT sensor data, images, or even protein structures. Essentially, it’s an array of floating-point numbers or binary values that captures the features of this data. By converting unstructured data into embedding vectors, modern techniques make it easier to analyze and compare complex data types in a structured manner.</p><h4>Vector Similarity Search</h4><p>Vector similarity search involves comparing a vector to a database of vectors to identify those most similar to the query vector. This process is accelerated by using Approximate Nearest Neighbor (ANN) search algorithms. When two embedding vectors are found to be very similar, it indicates that the original data sources they represent are also closely related.</p><h3>Why Milvus</h3><p>Basically Milvus stands out for its high performance, particularly when handling vector searches across massive datasets, ensuring quick and efficient retrieval of relevant data. It’s designed with developers in mind, offering extensive support for multiple programming languages and a robust toolchain that simplifies integration and usage. Milvus is also built for scalability in cloud environments, maintaining high reliability even during system disruptions. Additionally, it excels in hybrid search capabilities by seamlessly combining scalar filtering with vector similarity search, allowing for more precise and versatile data queries.</p><p>To achieve this level of precision, Milvus supports various similarity metrics that cater to different types of data and search requirements.</p><h4><strong>Similarity Metrics</strong></h4><p>In Milvus, similarity metrics are essential for measuring how closely vectors match each other. By selecting the appropriate metric, you can greatly improve the accuracy of tasks like classification and clustering. Milvus supports a variety of metrics tailored to different types of data. For floating-point embeddings, commonly used metrics include Euclidean distance, Inner product, and Cosine similarity. For binary embeddings, Hamming and Jaccard distances are frequently utilized. Each of these metrics plays a vital role in optimizing vector search performance, ensuring that similar data points are effectively and efficiently identified.</p><h3>Milvus Architecture</h3><p>Milvus is designed for similarity search on large-scale dense vector datasets, handling millions to trillions of vectors efficiently. It supports advanced features like data sharding, streaming data ingestion, dynamic schema, and hybrid search, making it highly adaptable to various embedding retrieval scenarios. Milvus adopts a shared-storage architecture with separate layers for access, coordination, computation, and storage, ensuring scalability, availability, and disaster recovery. For optimal performance, deploying Milvus with Kubernetes is recommended.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*J7nEQu2ALQQv4sLM.png" /><figcaption>Milvus Architecture Diagram</figcaption></figure><p>Milvus follows a disaggregated architecture, with four independent layers that ensure scalability and disaster recovery.</p><h4><strong>Access Layer</strong></h4><p>The access layer is the front-end of the system, composed of a group of stateless proxies that serve as the main user interface. It validates client requests and optimizes the results returned to users:</p><ul><li>The proxy is stateless and provides a unified service address through load balancing tools like Nginx, Kubernetes Ingress, NodePort, and LVS.</li><li>In Milvus’s massively parallel processing (MPP) architecture, the proxy aggregates and post-processes intermediate results before delivering the final output to the client.</li></ul><h4><strong>Coordinator Service</strong></h4><p>The coordinator service functions as the system’s brain, assigning tasks to worker nodes and managing cluster operations. It handles cluster topology, load balancing, timestamp generation, and data management.</p><ul><li><strong>Root Coordinator (Root Coord):</strong> Manages data definition (DDL) and data control (DCL) requests, such as creating or deleting collections, partitions, or indexes, and oversees timestamp management.</li><li><strong>Query Coordinator (Query Coord):</strong> Manages query node topology, load balancing, and segment handoffs from growing to sealed segments.</li><li><strong>Data Coordinator (Data Coord):</strong> Oversees data and index node topology, maintains metadata, and triggers background operations like flushing, compaction, and index building.</li></ul><h4><strong>Worker Nodes</strong></h4><p>Worker nodes execute tasks as directed by the coordinator service, handling data manipulation language (DML) commands from the proxy. These nodes are stateless, thanks to the separation of storage and computation, which allows for easy scaling and disaster recovery when deployed on Kubernetes. There are three types of worker nodes:</p><ul><li><strong>Query Node:</strong> Retrieves incremental log data, converts it into growing segments, loads historical data from object storage, and performs hybrid searches between vector and scalar data.</li><li><strong>Data Node:</strong> Processes mutation requests, retrieves log data, and packages it into log snapshots for storage in the object storage.</li><li><strong>Index Node:</strong> Builds indexes, which can be managed using a serverless framework without needing to be memory resident.</li></ul><h4><strong>Storage</strong></h4><p>Storage is the backbone of the system, ensuring data persistence. It consists of three main components:</p><ul><li><strong>Meta Storage:</strong> Stores snapshots of metadata, such as collection schemas and message checkpoints, using etcd for high availability and strong consistency. Milvus also relies on etcd for service registration and health checks.</li><li><strong>Object Storage:</strong> Handles the storage of log snapshots, index files, and intermediate query results. Milvus utilizes MinIO and can be deployed on AWS S3 or Azure Blob. To improve performance and reduce costs, Milvus plans to implement cold-hot data separation using memory or SSD-based caching.</li><li><strong>Log Broker:</strong> Acts as a pub-sub system for streaming data persistence and event notifications. It ensures data integrity during system recovery, using Pulsar for clusters and RocksDB for standalone setups. The log broker’s pub-sub mechanism, as illustrated, supports system scalability and reliability by maintaining a log sequence that subscribers use to update local data.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*x3rN3j1v5RfYB6Fl.png" /><figcaption>Log mechanism</figcaption></figure><p>Also Milvus operates in two modes: Standalone and Cluster. Both modes offer the same core features, allowing you to choose the one that best suits your dataset size, traffic, and operational needs. The Standalone mode is ideal for smaller-scale applications and includes core components like Milvus, Meta Store, and Object Storage, all working together to ensure efficient data management and persistence. On the other hand, the Cluster mode is designed for larger, more complex deployments, featuring a microservice architecture with components such as Root Coord, Proxy, and Query Node, along with third-party dependencies like etcd for metadata storage, S3 for object storage, and Pulsar or Kafka for log management. While Standalone mode is suitable for simpler setups, Cluster mode is preferred for scalable, distributed environments that require high availability and resilience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Eq-B-382aUzB1p2SLFcgNg.jpeg" /><figcaption>Standalone and Cluster modes</figcaption></figure><h4><strong>Message Storage / WAL</strong></h4><p>Milvus employs a Write-Ahead Logging (WAL) mechanism to ensure the secure processing of data. This mechanism records every operation before it is executed, allowing the system to recover by replaying these logs in the event of unexpected failures. For this critical function, Milvus supports two powerful messaging systems: Apache Kafka and Apache Pulsar.</p><ul><li><strong>Apache Kafka: </strong>Apache Kafka is a widely used distributed streaming platform known for its high throughput, fault tolerance, and scalability. Kafka operates as a publish-subscribe messaging system, where data is written to a log and then consumed by subscribers. In the context of Milvus, Kafka ensures the persistence of streaming data, preventing data loss in the event of a system failure. Kafka’s straightforward and direct architecture makes it a reliable choice for scenarios where message ordering and durability are crucial. Kafka’s advantages include its ability to handle high throughput and large-scale data processing. When working with massive datasets, Kafka enables rapid data processing. However, Kafka’s scalability is typically achieved by adding more brokers, which can complicate management. Additionally, Kafka is primarily a log-based storage system, which limits its use cases mainly to streaming data.</li><li><strong>Apache Pulsar: </strong>Apache Pulsar, on the other hand, offers a more flexible and powerful solution. Pulsar is a distributed messaging and streaming platform designed for high-performance workloads. One of its standout features is its multi-layered architecture, which separates the serving and storage layers. This design not only enhances scalability but also simplifies cluster management. Pulsar supports dynamic scaling, allowing the system to adapt to changing workloads seamlessly. Pulsar’s scalability is further enhanced by its integration with Apache BookKeeper, a robust management tool that ensures the persistent storage and management of data. This is particularly important when dealing with large datasets and scenarios that require high fault tolerance. Moreover, Pulsar can function both as a message queue and a message stream, making it a versatile option that supports a broader range of messaging patterns.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7A1Y7qcNBRsld9RORdaJ5A.jpeg" /></figure><p>Milvus supports both Kafka and Pulsar, offering flexibility in choosing the messaging system that best fits operational needs. Kafka is ideal for scenarios prioritizing high throughput and data durability. In contrast, Pulsar stands out with its fine-grained scalability, simplified management, and versatility, making it particularly advantageous for large-scale deployments.</p><p>Pulsar’s integration with Apache BookKeeper and its scalable architecture ensures that Milvus can maintain high availability and reliability, even as data volume and query load increase. This flexibility is especially beneficial in enterprise environments like n11, where complex and evolving data requirements must be met efficiently.</p><p>In summary, while Kafka and Pulsar are both robust options for managing WAL in Milvus, Pulsar’s advanced features and flexibility make it a compelling choice for optimizing Milvus for diverse and future-proof applications.</p><p>Now that we’ve thoroughly explored the architecture and the integral components that form the backbone of Milvus, let’s shift our focus to the algorithms that power its performance. Understanding these algorithms is key to grasping how Milvus efficiently handles the complexity of large-scale, unstructured datasets.</p><h3>Algorithms</h3><h4><strong>ANNOY!</strong>🥱</h4><p>ANNOY (Approximate Nearest Neighbors Oh Yeah!) is an algorithm designed for efficient nearest neighbor search in high-dimensional spaces. It’s particularly useful when dealing with large datasets where an exact search would be computationally expensive. ANNOY works by building multiple trees, each one splitting the data in a different way, to approximate the nearest neighbors of a given point.</p><p>When a query is made, ANNOY traverses these trees to quickly find candidate neighbors. While it doesn’t always guarantee the exact nearest neighbors, it achieves a balance between speed and accuracy, which is critical in large-scale applications. ANNOY can work with various distance metrics, such as Euclidean distance, to determine how close two vectors are in the high-dimensional space. Euclidean distance, for example, measures the straight-line distance between points and is commonly used in tasks that require assessing the similarity between data points.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6rvP6BSC6MVxEEYo7rwvsw.png" /><figcaption>ANNOY! Algorithm Flow</figcaption></figure><p>Now, if we look at the flow diagram you provided, it illustrates a basic approach that mirrors the steps ANNOY takes to find the nearest neighbors:</p><p>• <strong>Starting Point</strong>: Similar to how ANNOY begins with a set of data points, the process starts by selecting an initial point to begin the search.</p><p>• <strong>Tree Traversal</strong>: The diagram shows how the algorithm systematically searches for the nearest unvisited points, akin to how ANNOY navigates through its trees to identify potential neighbors.</p><p>• <strong>Approximation</strong>: Just as ANNOY may not visit every single point but instead focuses on approximating the nearest neighbors quickly, the diagram reflects this by prioritizing certain points while marking others as visited.</p><p>By combining the principles shown in the diagram with a distance metric like Euclidean distance, ANNOY is able to efficiently perform approximate nearest neighbor searches, making it a powerful tool in scenarios where performance is critical, such as real-time recommendation systems or large-scale image retrieval in Milvus.</p><h4><strong>HNSW</strong>🌐</h4><p>HNSW (Hierarchical Navigable Small World) is an advanced algorithm for approximate nearest neighbor search, similar to ANNOY but with key differences. While ANNOY uses random trees to partition data, HNSW builds a multi-layer graph where higher layers provide a coarse overview and lower layers offer finer detail. This hierarchical structure allows HNSW to navigate the search space more intelligently, often resulting in faster and more accurate searches, especially in high-dimensional or complex data distributions.</p><ul><li><strong>Multi-Layer Graph Structure</strong>: HNSW organizes data into multiple layers, where each layer is a graph connecting points that are close to each other according to some distance metric, such as Euclidean distance. The higher layers have fewer points and provide a broad, less detailed view, whereas the lower layers have more points and provide detailed connections.</li><li><strong>Search Process: </strong>The search in HNSW starts at the topmost layer, where the algorithm quickly identifies a rough approximation of the nearest neighbors. It then navigates down through the layers, refining the search with each step, until it reaches the bottom layer, where it performs a more precise search to identify the nearest neighbors.</li><li><strong>Efficiency: </strong>The hierarchical structure allows HNSW to efficiently narrow down the search space by progressively refining the search from a broad overview to a detailed examination. This makes it highly efficient for large-scale datasets where exhaustive search would be computationally prohibitive.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*r5mjE143rfNANwsZ.png" /><figcaption>Layers in the HNSW Algorithm</figcaption></figure><p>The diagram you provided visually represents the layers in the HNSW algorithm. Here’s how it relates to the HNSW process:</p><ul><li><strong>Layer 2 (Top Layer)</strong>: This is the highest layer with the fewest points, providing a broad overview of the data space. The algorithm begins the search here, identifying the nearest neighbor in this coarse layer. The dashed lines represent the connections between layers, indicating the transition from a broad search to a more detailed one.</li><li><strong>Layer 1 (Middle Layer)</strong>: After identifying the nearest neighbor in the top layer, the algorithm moves to the next layer down. Here, it refines the search by finding the nearest neighbor within a more populated set of points. The process repeats as the algorithm navigates down through the layers.</li><li><strong>Layer 0 (Bottom Layer)</strong>: This is the most detailed layer, where the search is finally narrowed down to find the exact nearest neighbors. The connections in this layer are dense, allowing for a more precise search that considers all possible candidates.</li></ul><p>Having explored the fundamental concepts, architecture, and key algorithms that make Milvus a powerful vector database, let’s now turn our attention to how these elements come together in a real-world application. Specifically, we’ll delve into how n11 leverages Milvus for category prediction, ensuring that products are accurately classified based on both their titles and images. We’ll walk through a detailed scenario that illustrates how Milvus processes data, applies advanced algorithms like ANNOY and HNSW, and ultimately helps n11 maintain a high standard of accuracy and consistency in its marketplace.</p><h3><strong>Using Milvus for Category Prediction in n11: A Detailed Walkthrough</strong></h3><p>In the n11 platform, category prediction is essential to ensure that products are correctly classified, which significantly impacts the user experience and the accuracy of search results. By leveraging Milvus and its robust vector search capabilities, n11 can automate and enhance this process, particularly in scenarios where product titles and images may not align perfectly. Below, we’ll explore how Milvus processes the data, from ingestion to final categorization, using advanced algorithms like ANNOY and HNSW.</p><h4><strong>1.Data Ingestion and Vectorization</strong></h4><p><strong>Text Data (Product Title)</strong></p><ul><li>When a seller inputs a product title, the text undergoes preprocessing, including tokenization, stop-word removal, and possibly stemming or lemmatization to distill the text to its core components.</li><li>This processed text is then converted into a high-dimensional vector using an embedding model such as Word2Vec or BERT. These models map the text into a vector space where semantically similar phrases are positioned closely.</li><li>For instance, a title like “Sunglasses for Women” might be transformed into a 300-dimensional vector, with each dimension representing a specific feature learned from the model’s training data.</li></ul><p><strong>Image Data (Product Images)</strong></p><ul><li>Uploaded product images are processed using a convolutional neural network (CNN), such as ResNet or VGG, which extracts visual features from the images and generates embedding vectors.</li><li>These vectors represent the image in a high-dimensional space, capturing visual aspects like shapes, colors, and textures. Similar images will thus be located near each other in this vector space.</li></ul><h4><strong>2. Attribute Assignment and Metadata</strong></h4><p>After vectorization, Milvus assigns attributes to each vector:</p><ul><li><strong>Vector ID</strong>: A unique identifier assigned to each vector for retrieval purposes.</li><li><strong>Category Labels</strong>: Initial labels provided by the seller, such as “Sunglasses,” used as metadata.</li><li><strong>Timestamp</strong>: The time the vector was created, useful for tracking and data management.</li><li><strong>Additional Metadata</strong>: Other relevant information, like seller ID, product ID, or tags, to facilitate filtering or further categorization.</li></ul><p>Here’s an example JSON structure that represents the attribute assignment and metadata for a vector in Milvus:</p><pre>{<br>  &quot;vector_id&quot;: &quot;1234567890abcdef&quot;,<br>  &quot;attributes&quot;: {<br>    &quot;category_label&quot;: &quot;Sunglasses&quot;,<br>    &quot;timestamp&quot;: &quot;2024-09-01T10:15:30Z&quot;,<br>    &quot;metadata&quot;: {<br>      &quot;seller_id&quot;: &quot;seller_789&quot;,<br>      &quot;product_id&quot;: &quot;product_12345&quot;,<br>      &quot;tags&quot;: [&quot;fashion&quot;, &quot;accessory&quot;, &quot;summer&quot;]<br>    }<br>  },<br>  &quot;vector_data&quot;: [<br>    0.132, 0.567, -0.234, 0.789, -0.456, 0.101, 0.923, 0.654, -0.214, 0.783<br>    // Additional dimensions of the vector...<br>  ]<br>}</pre><h4><strong>3. Data Storage in Milvus</strong></h4><p>After vectorization and attribute assignment, the vectors are ingested into Milvus, where they are indexed for fast retrieval. The process involves several steps within Milvus’s architecture:</p><p><strong>Access Layer</strong></p><ul><li>The data first passes through the Access Layer, comprising stateless proxies that handle client requests. The proxies validate the data, ensuring it meets format and attribute requirements.</li><li>The Access Layer also balances the load across multiple proxies, ensuring efficient processing even with large volumes of data.</li></ul><p><strong>Coordinator Service</strong></p><ul><li>After validation, the data is managed by the Coordinator Service, which acts as the system’s brain, distributing data across worker nodes.</li><li>The Root Coordinator manages metadata and assigns data to appropriate partitions, triggering necessary background tasks like index building.</li></ul><p><strong>Worker Nodes</strong></p><p>These nodes are responsible for the actual data processing:</p><ul><li><strong>Data Nodes</strong>: Handle the ingestion of vectors, ensuring they are stored correctly and creating log snapshots for persistence.</li><li><strong>Index Nodes</strong>: Index the data using algorithms like ANNOY or HNSW:</li></ul><blockquote><strong>ANNOY</strong> creates random projection trees that partition the vector space, facilitating fast approximate nearest neighbor searches.</blockquote><blockquote><strong>HNSW</strong> builds a hierarchical graph, where each vector connects to its nearest neighbors across multiple layers, allowing more accurate searches by refining results through the graph layers.</blockquote><ul><li><strong>Query Nodes</strong>: Manage search queries, retrieving indexed vectors and performing similarity searches to find the closest matches.</li></ul><p><strong>Final Storage</strong></p><p>Indexed vectors are stored in Milvus’s Object Storage, backed by systems like MinIO, AWS S3, or Azure Blob Storage, ensuring data persistence. Metadata management is handled by the Meta Storage, using etcd for high availability and consistency.</p><h4><strong>4. Data Retrieval and Similarity Search</strong></h4><p>When a new product is listed, Milvus processes the query through the following steps:</p><ul><li><strong>Query Processing</strong>: The query is received by the Access Layer and forwarded to a Query Node.</li><li><strong>Index Traversal</strong>: The Query Node uses the indexes built by ANNOY or HNSW to navigate the vector space and identify nearest neighbors:</li></ul><blockquote><strong>ANNOY</strong> provides fast approximate results using its random trees.</blockquote><blockquote><strong>HNSW</strong> refines the search through hierarchical layers, offering more accurate results.</blockquote><ul><li><strong>Result Compilation</strong>: The closest matches are compiled and returned for further processing.</li></ul><h4><strong>5. Final Categorization and Action</strong></h4><p><strong>Cross-Validation: </strong>If the image vectors and title vector suggest different categories (e.g., the title suggests “Sunglasses,” but the image resembles “Women’s Clothing”), the system flags this as a mismatch. The seller may be prompted to correct the images or title.</p><p><strong>Blocking Nonexistent Products</strong>: For titles indicating nonexistent products (e.g., “iPhone 16”), Milvus can block the listing if no similar products exist in the database or if the title refers to a product not yet available.</p><p><strong>Final Decision</strong>: Based on the similarity search and cross-validation, the product is either categorized correctly, flagged for review, or blocked if it doesn’t meet the platform’s standards.</p><h4><strong>Conclusion</strong></h4><p>Milvus, with its advanced data processing and indexing capabilities, empowers n11 to effectively handle large volumes of unstructured data. By leveraging powerful algorithms like ANNOY and HNSW, Milvus ensures that similarity searches are not only fast but also highly accurate, enabling precise categorization of products. This precision is crucial for n11, as it directly contributes to improving the accuracy of product listings, minimizing the risk of misclassification, and maintaining the overall integrity of the marketplace.</p><p>Incorporating Milvus into our processes allows us to stay ahead in managing complex data challenges, ensuring that our platform continues to offer a reliable and seamless experience for both sellers and buyers. As we continue to refine and expand our use of Milvus, we anticipate even greater efficiency and accuracy in how we categorize and present products, ultimately enhancing the quality and trustworthiness of the n11 marketplace.</p><p>Thank you for taking the time to explore how we are utilizing Milvus to optimize our operations at n11. We’re excited to continue innovating and providing the best possible experience for our users, and we look forward to sharing more of our journey with you in the future.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1fe4dedd7412" width="1" height="1" alt=""><hr><p><a href="https://medium.com/n11-tech/understanding-milvus-key-concepts-and-potential-applications-1fe4dedd7412">Understanding Milvus: Key Concepts and Potential Applications</a> was originally published in <a href="https://medium.com/n11-tech">n11 Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Apache Kafka: From LinkedIn’s Data Streams to Worldwide Communication]]></title>
            <link>https://medium.com/@berktorun.dev/understanding-apache-kafka-from-linkedins-data-streams-to-worldwide-communication-8e199d7140f7?source=rss-759f402bb906------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e199d7140f7</guid>
            <dc:creator><![CDATA[Berk Torun]]></dc:creator>
            <pubDate>Fri, 15 Mar 2024 08:52:19 GMT</pubDate>
            <atom:updated>2024-03-15T08:52:19.788Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yfiFYoGvR-FaB4Rq.png" /></figure><p>Welcome to the fascinating world of Apache Kafka, a powerhouse in the realm of real-time data streaming! This guide aims to shed light on Kafka’s core concepts, starting from its inception at LinkedIn to its evolution into a global communication technology. Whether you’re a seasoned developer or just curious about data architectures, join us as we unravel the mysteries of Kafka, making complex ideas accessible and engaging. Let’s dive into the journey of Kafka, understanding its purpose, mechanics, and impact on the tech world today.</p><h3>1. Introduction to Apache Kafka</h3><h4>What is Apache Kafka?</h4><p>Apache Kafka is an open-source stream-processing software platform developed by the Apache Software Foundation, written in Scala and Java. It is designed to provide a high-throughput, low-latency platform for handling real-time data feeds. Kafka enables the processing of streams of records in a fault-tolerant, durable way and is widely used for building real-time streaming data pipelines and applications.</p><h4>The Origin of Kafka at LinkedIn: Addressing a Specific Need</h4><p>Kafka was originally developed at LinkedIn to solve the problem of handling large amounts of activity data across the site. The platform needed a system that could efficiently process the huge volumes of data generated by the website’s user interactions, system logs, and operational metrics. LinkedIn needed a scalable and reliable solution to process these continuous data streams in real-time, which led to the creation of Kafka. It was designed to be scalable, fault-tolerant, and capable of handling millions of messages per second, eventually becoming a fundamental part of LinkedIn’s data architecture. Kafka’s success at LinkedIn led to its adoption as an open-source project under the Apache Software Foundation, where it has continued to evolve and gain popularity across various industries for its robust streaming capabilities.</p><h3>2. Message Structure in Apache Kafka</h3><h4>Basics of Messages in Kafka</h4><p>In Apache Kafka, a message is the basic unit of data that is transported between producers and consumers. Each message consists of a key, a value, and a timestamp. The key is used to determine the partition within a topic where the message will be sent, facilitating message ordering and categorization. The value, on the other hand, is the actual data payload of the message. The timestamp records the time at which the message was produced, aiding in time-sensitive data processing and retention policies.</p><p>Messages in Kafka are sent to topics, which act as categories or feeds to which messages are published. Topics in Kafka are split into partitions, allowing the system to scale by distributing the data across multiple nodes in the cluster, thereby enhancing performance and reliability.</p><h4>Key-Value Pairs and Serialization</h4><p>Key-value pairs in Kafka messages are a fundamental concept, allowing for structured data processing and efficient message routing. The key typically represents the identifier that determines how data is partitioned and stored across the Kafka cluster, while the value holds the message’s content.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s39Uw0UaSpiIfUxTMGc_NQ.png" /></figure><p>Serialization is the process of converting the key and value objects into a binary format for storage and transmission over the network. Kafka uses serializers to convert message keys and values into bytes and deserializers to convert bytes back into keys and values on the receiving end. This process is crucial for Kafka’s performance and scalability, as it ensures that messages are compact and network transfer is optimized. Common serialization formats in Kafka include String, JSON, and Avro, each offering different benefits in terms of readability, schema evolution, and efficiency.</p><p>Understanding the message structure and the serialization process in Kafka is essential for designing effective data streaming applications, as it directly affects how data is organized, processed, and stored within the Kafka ecosystem.</p><h3>3. Batching in Apache Kafka</h3><h4>What is Batching?</h4><p>Batching in Apache Kafka refers to the process of grouping multiple messages together into a single batch for processing and transmission. This technique is fundamental in Kafka’s design, allowing it to efficiently handle large volumes of data messages. Instead of sending messages individually, which can incur significant overhead and latency, Kafka combines them into batches, reducing the number of network requests and improving overall throughput.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rpQBctIrq-CuPRJCtwmNgg.png" /></figure><h4>How Batching Optimizes Throughput</h4><p>Batching optimizes throughput in several ways. Firstly, it reduces the network I/O operations by sending a larger, combined payload rather than multiple smaller ones, which minimizes the per-message overhead and improves the utilization of network resources. Secondly, it allows Kafka to make better use of disk I/O operations when storing messages to the log by writing chunks of data at once rather than individual messages, leading to faster write speeds and better disk throughput.</p><p>Furthermore, batching can be tuned through configuration settings in Kafka, such as the batch size and the linger time, which dictate the maximum size of the batch and the maximum time to wait before sending the batch, respectively. These settings allow developers to strike a balance between latency and throughput, optimizing performance based on the specific requirements of their application.</p><p>In summary, batching is a critical feature in Apache Kafka that enhances data processing efficiency by grouping messages together, thereby optimizing network and disk utilization and significantly increasing the system’s throughput.</p><h3>4. Topics and Partitions in Apache Kafka</h3><h4>Understanding Topics</h4><p>In Apache Kafka, a topic is a category or feed name to which messages are published by producers. Topics are the core abstraction Kafka uses to allow data producers and consumers to communicate. They act like channels where messages flow from producers to consumers. Each topic is identified by a unique name within a Kafka cluster, and producers send messages to specific topics while consumers subscribe to topics to receive messages.</p><p>Topics are designed to be multi-subscriber; that is, they allow multiple consumers to read messages independently and at their own pace without interfering with each other. This design makes Kafka highly suitable for building scalable and distributed data processing systems where multiple applications or services need to process the same set of messages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KvA6_EIHF3RwW9OS0GJueA.png" /></figure><h4>The Role of Partitions in Scalability and Fault Tolerance</h4><p>Partitions are a fundamental aspect of Kafka’s scalability and fault tolerance. A topic in Kafka is divided into one or more partitions. These partitions allow the messages for a topic to be spread across multiple nodes in the Kafka cluster, enabling horizontal scaling of processing. Each partition can be placed on a different server, thus spreading the data load and allowing for more messages to be processed concurrently.</p><p>Partitions also play a crucial role in fault tolerance. In Kafka, each partition can be replicated across multiple nodes, meaning that copies of the partition’s data are maintained on different servers. This replication ensures that if a node fails, the data can be retrieved from another node that has a replica of the partition, thereby minimizing data loss and downtime.</p><p>The number of partitions in a topic directly influences the degree of parallelism in data processing, as more partitions mean more parallel consumers can read from a topic concurrently. However, the partition count should be chosen wisely, as too many partitions can lead to overhead in management and decreased performance due to the increased number of file handles and network connections.</p><p>In summary, topics and partitions are central to Kafka’s architecture, providing the mechanisms for organizing messages and enabling the system to scale out and handle failures gracefully, thus ensuring reliable data delivery at scale.</p><h3>5. Producer and Consumer Architecture in Apache Kafka</h3><h4>The Producer: Generating and Sending Messages</h4><p>In Apache Kafka, a producer is an application or process that creates and sends messages to a Kafka topic. The producer is responsible for determining which record to send to which partition within the topic. This can be done either automatically by Kafka, where it evenly distributes messages across all partitions, or by specifying a key in the message, which Kafka uses to consistently route messages with the same key to the same partition.</p><p>Producers can send messages synchronously or asynchronously. In synchronous transmission, the producer waits for an acknowledgment from the broker after each message is sent, ensuring that the message is successfully stored. In asynchronous transmission, the producer sends messages in batches without waiting for acknowledgments, which can lead to higher throughput but with a risk of message loss in case of a failure.</p><p>The producer’s performance and reliability are crucial for the overall efficiency of the Kafka ecosystem. Producers also have features like message buffering and compression, which help in enhancing throughput and reducing the load on the network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZtiY5qBUWqFf6mPLg2lm2A.png" /></figure><h4>The Consumer: Retrieving and Processing Messages</h4><p>A consumer in Apache Kafka is an application or process that subscribes to one or more topics and reads messages from them. Consumers pull data from Kafka topics; they do not receive data pushed to them. This pull model allows consumers to control the pace at which they process messages, enabling them to handle varying loads of data efficiently.</p><p>Kafka consumers can work in groups to consume data from multiple partitions of a topic, facilitating distributed processing and enhancing scalability. Each consumer in a group is assigned a set of partitions from the topics they subscribe to, and each message from those partitions is delivered to only one consumer in the group. Kafka manages the distribution of partitions across consumers dynamically, rebalancing them as consumers join or leave the group.</p><p>Consumers keep track of the messages they have processed using an offset, which is a sequential id that uniquely identifies each message within a partition. Consumers can commit these offsets to Kafka, which allows them to resume reading from where they left off in case of a failure or restart, ensuring no message is lost or processed twice.</p><p>Together, the producer and consumer architecture in Kafka provides a robust framework for reliable, scalable, and efficient real-time data processing, enabling a wide range of streaming applications in various industries.</p><h3>6. Brokers and Clusters in Apache Kafka</h3><h4>The Broker: A Closer Look</h4><p>In the world of Apache Kafka, a broker refers to a single Kafka server instance. Brokers are the backbone of the Kafka ecosystem, responsible for storing data and serving client requests. Each broker can handle thousands of reads and writes per second and can store terabytes of data without significantly impacting performance. Brokers manage the data storage for topics in partitions and serve as the communication hub between producers and consumers.</p><p>A Kafka broker receives messages from producers, assigns offsets to them, and then stores the messages in a log file on disk. The broker also serves consumer requests to read messages, delivering the messages from the log files to the consumers. In addition to these tasks, brokers handle tasks such as topic and partition management, request processing, and replication to ensure data durability and high availability.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IURGQfUYEsKkqHN-2L5lAQ.png" /></figure><h4>Clustering for High Availability and Scalability</h4><p>Kafka’s architecture is designed to run in a cluster configuration, where multiple brokers work together to form a distributed system. This clustering enhances Kafka’s high availability and scalability. If one broker fails, others in the cluster can take over the workload, ensuring that the system remains available and that data is not lost.</p><p>The clustering mechanism allows Kafka to distribute the data across multiple machines, balancing the load and scaling out to accommodate more data and processing. Partitions of a topic are distributed across different brokers in the cluster, and each partition can be replicated across multiple brokers. This means that even if a broker in the cluster fails, another broker that has a replica of the same partition can continue to serve data, ensuring no interruption in service.</p><p>Clustering also allows Kafka to balance load dynamically. As the data volume or the processing needs grow, more brokers can be added to the cluster to increase capacity. Kafka’s design enables it to handle massive data streams by scaling horizontally, adding more machines to the cluster to increase throughput and storage capacity as needed.</p><p>In conclusion, brokers and clusters form the core of Kafka’s operational capabilities, providing the infrastructure needed for reliable, scalable, and efficient data streaming. Their robust design ensures that Kafka can serve as a central platform for processing and analyzing large streams of real-time data, making it a critical component in modern data architectures.</p><h3>Closing Remarks</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*WTgS7ZyVZkeBvV6Q.gif" /></figure><p>As we wrap up our exploration of Apache Kafka, it’s clear that this powerful platform is more than just a messaging system; it’s the backbone of modern data streaming, capable of handling vast volumes of real-time data with ease. From its origins at LinkedIn to becoming a critical tool in global data infrastructure, Kafka has proven its worth in scalability, reliability, and efficiency. Whether you’re dealing with big data or building complex applications, Kafka offers the robustness and flexibility needed to meet the challenges of today’s data-driven world. So here’s to Kafka, a true pioneer in streaming technology, continuously evolving to power our data-centric future.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e199d7140f7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[JWT Authentication with Spring Boot 3 and Java 17]]></title>
            <link>https://medium.com/@berktorun.dev/the-grand-tour-spring-security-and-jwt-authentication-dfd20040fbd6?source=rss-759f402bb906------2</link>
            <guid isPermaLink="false">https://medium.com/p/dfd20040fbd6</guid>
            <dc:creator><![CDATA[Berk Torun]]></dc:creator>
            <pubDate>Sun, 18 Feb 2024 10:32:08 GMT</pubDate>
            <atom:updated>2024-02-22T21:59:36.821Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/705/1*bjjP5iaBNGpr5tZS4RKx3g.png" /></figure><p>Welcome to our exploration of Spring Security and JWT Authentication within a Spring Boot framework. In this session, we’ll delve into the foundational elements that construct a secure environment for web applications. Our journey will be comprehensive, covering critical aspects from dependencies to configuration, ensuring a robust understanding of the mechanisms at play. Prepare to enhance your knowledge of securing applications effectively using industry-standard practices. Let’s begin our focused and detailed examination.</p><h3>Spring Security and JWT Dependencies: The Cornerstones of Security</h3><p>In any Spring Boot application, security is paramount, and integrating JWT for authentication adds a robust layer of protection. But before we can harness this power, we need to introduce some key players in the form of Maven dependencies.</p><h4><strong><em>Spring Boot Starter Security</em></strong></h4><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;org.springframework.boot&lt;/groupId&gt;<br>    &lt;artifactId&gt;spring-boot-starter-security&lt;/artifactId&gt;<br>&lt;/dependency&gt;</pre><p>This dependency is the backbone of our security setup. It integrates Spring Security into our Spring Boot application, bringing with it a suite of authentication, protection, and configuration mechanisms.</p><h4>JJWT (JSON Web Token for Java)</h4><p>The JJWT library is our go-to toolkit for working with JWTs in Java. It simplifies the creation, parsing, and validation of JWTs. However, JJWT is modular, so we’ll need a few pieces to make it fully functional:</p><ul><li><strong>API Module</strong></li></ul><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;io.jsonwebtoken&lt;/groupId&gt;<br>    &lt;artifactId&gt;jjwt-api&lt;/artifactId&gt;<br>    &lt;version&gt;0.11.5&lt;/version&gt;<br>&lt;/dependency&gt;</pre><p>This is where all the JWT standards and interfaces live. It’s the core that defines how we interact with JWTs</p><ul><li><strong>Implementation Module</strong></li></ul><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;io.jsonwebtoken&lt;/groupId&gt;<br>    &lt;artifactId&gt;jjwt-impl&lt;/artifactId&gt;<br>    &lt;version&gt;0.11.5&lt;/version&gt;<br>&lt;/dependency&gt;</pre><p>While the API gives us the rules, the implementation is where the action happens. This module contains the machinery that processes JWTs according to the definitions in the API.</p><ul><li><strong>Jackson Integration</strong></li></ul><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;io.jsonwebtoken&lt;/groupId&gt;<br>    &lt;artifactId&gt;jjwt-jackson&lt;/artifactId&gt;<br>    &lt;version&gt;0.11.5&lt;/version&gt;<br>&lt;/dependency&gt;</pre><p>For JWTs to work smoothly with JSON objects (which they inherently are), this module integrates Jackson into the mix, allowing seamless conversion between JWTs and JSON.</p><p>By integrating these dependencies into our Spring Boot application, we establish a solid foundation for implementing JWT-based authentication. This setup not only secures our application but also aligns it with modern security standards, ensuring our application’s defenses are as formidable as a well-fortified castle.</p><p>With our dependencies in place, we’re now equipped to build upon this foundation, introducing entities and configurations that interact with these security mechanisms. Let’s prepare to delve deeper into the world of Spring Security and JWT, turning our focus towards the pivotal roles played by users and role entities in our next section.</p><h3>The Heart of the Realm: User and Role Entities in Spring Security</h3><p>In the realm of Spring Security, the core entities that uphold the kingdom’s order are the User and Role classes. These are not just mere data structures; they are the very backbone of our security model, ensuring that each individual within our system is properly identified and granted the correct level of access.</p><ul><li><strong>User Class</strong></li></ul><pre>@Data<br>@Builder<br>@Entity<br>@NoArgsConstructor<br>@AllArgsConstructor<br>@Table(name = &quot;users&quot;)<br>public class User implements UserDetails {<br><br>    @Id<br>    @GeneratedValue(strategy = GenerationType.UUID)<br>    private UUID id;<br><br>    @Column(unique = true)<br>    private String username;<br><br>    @Column(unique = true)<br>    private String email;<br><br>    private String password; // Note: In real scenarios, exclude from JSON serialization with @JsonIgnore<br><br>    private Set&lt;Role&gt; authorities; // Roles are simplified for illustration<br><br>    private boolean accountNonExpired;<br>    private boolean isEnabled;<br>    private boolean accountNonLocked;<br>    private boolean credentialsNonExpired;<br>}</pre><p>In this revised User class, the attributes are tailored to fit the needs of Spring Security. By implementing UserDetails, we provide Spring Security with essential information like whether the account is expired or locked, and what authorities (or roles) the user has. This allows for a seamless integration with Spring Security&#39;s authentication and authorization mechanisms.</p><ul><li><strong>Role Enum Class</strong></li></ul><pre>public enum Role implements GrantedAuthority {<br>    ROLE_USER(&quot;USER&quot;),<br>    ROLE_ADMIN(&quot;ADMIN&quot;);<br><br>    private String value;<br><br>    Role(String value) {<br>        this.value = value;<br>    }<br><br>    public String getValue() {<br>        return this.value;<br>    }<br><br>    @Override<br>    public String getAuthority() {<br>        return name();<br>    }<br>}</pre><p>The Role enum defines the different roles available within our system. By implementing GrantedAuthority, we enable Spring Security to recognize these roles as authorities that can be assigned to users. This is crucial for the enforcement of role-based access control throughout our application.</p><h3>The Sentinel at the Gate: JwtAuthFilter in Action</h3><p>As we continue our journey through the secure realms of Spring Security and JWT, we meet an essential guardian — the JwtAuthFilter. This component acts as the vigilant sentinel at the gates of our application, scrutinizing every incoming request to ensure only the worthy pass.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/0*vX6JlwMsDXgO-8HO.gif" /></figure><p>The JwtAuthFilter is a specialized filter extending OncePerRequestFilter, ensuring that our security checks are applied once per request. Here’s a distilled version of what this steadfast guardian looks like:</p><pre>@Component<br>@RequiredArgsConstructor<br>public class JwtAuthFilter extends OncePerRequestFilter {<br><br>    private final JwtService jwtService;<br>    private final UserService userService;<br><br>    @Override<br>    protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)<br>            throws ServletException, IOException {<br>        String authHeader = request.getHeader(&quot;Authorization&quot;);<br>        if (authHeader != null &amp;&amp; authHeader.startsWith(&quot;Bearer &quot;)) {<br>            String token = authHeader.substring(7);<br>            String userName = jwtService.extractUsername(token);<br>            if (userName != null &amp;&amp; SecurityContextHolder.getContext().getAuthentication() == null) {<br>                UserDetails user = userService.loadUserByUsername(userName);<br>                if (jwtService.validateToken(token, user)) {<br>                    UsernamePasswordAuthenticationToken authToken = new UsernamePasswordAuthenticationToken(<br>                            user, null, user.getAuthorities());<br>                    authToken.setDetails(new WebAuthenticationDetailsSource().buildDetails(request));<br>                    SecurityContextHolder.getContext().setAuthentication(authToken);<br>                }<br>            }<br>        }<br>        filterChain.doFilter(request, response);<br>    }<br>}</pre><p>In this compact yet powerful class, the JwtAuthFilter serves multiple critical roles:</p><ol><li><strong>Token Extraction:</strong> It peeks into the Authorization header of each request, looking for the bearer token that signifies a user&#39;s credentials.</li><li><strong>Identity Verification:</strong> By parsing the token, the filter retrieves the username and then employs the UserService to fetch the corresponding user details.</li><li><strong>Authentication:</strong> If the token is valid and the user is not already authenticated, the filter crafts a UsernamePasswordAuthenticationToken and registers it with Spring Security&#39;s SecurityContextHolder. This step effectively signs the user into the application for the duration of the request.</li><li><strong>Access Control:</strong> By inserting this filter into the security chain, we ensure that no request bypasses our strict token-based authentication mechanism.</li></ol><p>In essence, the JwtAuthFilter stands as the unwavering guardian of our application, a testament to the robust security afforded by Spring Security and JWT integration. It ensures that no unwelcome intruders can penetrate our domain, keeping our digital fortress safe and secure.</p><h3>The Master of Ceremonies: JwtService Unveiled</h3><p>In the grand ballroom of Spring Security and JWT Authentication, the JwtService stands as the master of ceremonies. This pivotal component orchestrates the validation and creation of JWT tokens, ensuring that only legitimate and validated users can access the protected areas of our application.</p><pre>@Service<br>public class JwtService {<br><br>    @Value(&quot;${application.security.jwt.secret-key}&quot;)<br>    private String secretKey;<br><br>    @Value(&quot;${application.security.jwt.expiration}&quot;)<br>    private long expiration;<br><br>    public String generateToken(User user) {<br>        Map&lt;String, Object&gt; claims = new HashMap&lt;&gt;();<br>        claims.put(&quot;id&quot;, user.getId());<br>        claims.put(&quot;email&quot;, user.getEmail());<br>        claims.put(&quot;roles&quot;, user.getAuthorities());<br>        return createToken(claims, user.getUsername());<br>    }<br><br>    public Boolean validateToken(String token, UserDetails userDetails) {<br>        Date expirationDate = extractExpiration(token);<br>        if (expirationDate.before(new Date())) {<br>            return false;<br>        }<br>        String username = extractUsername(token);<br>        return userDetails.getUsername().equals(username) &amp;&amp; !expirationDate.before(new Date());<br>    }<br><br>    private String createToken(Map&lt;String, Object&gt; claims, String username) {<br>        return Jwts.builder()<br>                .setClaims(claims)<br>                .setSubject(username)<br>                .setIssuedAt(new Date(System.currentTimeMillis()))<br>                .setExpiration(new Date(System.currentTimeMillis() + expiration))<br>                .signWith(getSignKey(), SignatureAlgorithm.HS256)<br>                .compact();<br>    }<br><br>    private Key getSignKey() {<br>        byte[] keyBytes = Decoders.BASE64.decode(secretKey);<br>        return Keys.hmacShaKeyFor(keyBytes);<br>    }<br>}</pre><p>In this class, the generateToken method crafts the JWT token, embedding critical information such as the user&#39;s ID, email, and roles. These claims ensure that each token is personalized and secure. Meanwhile, the validateToken method acts as the gatekeeper, verifying the token&#39;s integrity and relevance against the user&#39;s details and ensuring that expired tokens are rejected, maintaining the security of the application.</p><p>The creation and validation processes hinge on the secretKey and expiration values, which determine the token&#39;s durability and safety. By utilizing the JJWT library&#39;s parsing and signing capabilities, the JwtService ensures that each token is a reliable representation of the user&#39;s identity and permissions.</p><pre>application.security.jwt.secret-key=secretkey<br># a day<br>application.security.jwt.expiration=86400000</pre><p>Thus, as the master of ceremonies, the JwtService ensures that every token — much like an invite to an exclusive event — is valid, up-to-date, and correctly issued, maintaining the integrity and security of the entire application.</p><h3>The Architect’s Blueprints: Securing the Kingdom with SecurityConfig</h3><p>As we conclude our grand tour, we find ourselves in the architect’s chamber, the place where the structural integrity of our kingdom’s security is conceived and forged. This is where SecurityConfig comes into play, acting as the master blueprint for our Spring application&#39;s security framework.</p><p>Let’s delve into the simplified version of the SecurityConfig class and understand how it cements the defenses of our digital realm:</p><pre>@Configuration<br>@EnableWebSecurity<br>@EnableMethodSecurity<br>@RequiredArgsConstructor<br>public class SecurityConfig {<br><br>    private final JwtAuthFilter jwtAuthFilter;<br>    private final UserService userService;<br>    private final PasswordEncoder passwordEncoder;<br><br>    // Reduced whitelist for simplicity<br>    private static final String[] WHITE_LIST_URL = {<br>            &quot;/api/v1/auth/login&quot;,<br>            &quot;/api/v1/auth/register&quot;<br>    };<br><br>    @Bean<br>    SecurityFilterChain filterChain(HttpSecurity http) throws Exception {<br>        return http<br>                .csrf(AbstractHttpConfigurer::disable) // Disabling CSRF as we use JWT which is immune to CSRF<br>                .authorizeHttpRequests(auth -&gt; auth<br>                        .requestMatchers(WHITE_LIST_URL).permitAll() // Whitelisting some paths from authentication<br>                        .anyRequest().authenticated()) // All other requests must be authenticated<br>                .sessionManagement(session -&gt; session<br>                        .sessionCreationPolicy(SessionCreationPolicy.STATELESS)) // Stateless session management<br>                .addFilterBefore(jwtAuthFilter, UsernamePasswordAuthenticationFilter.class); // Registering our JwtAuthFilter<br>    }<br><br>    @Bean<br>    AuthenticationProvider authenticationProvider() {<br>        DaoAuthenticationProvider provider = new DaoAuthenticationProvider();<br>        provider.setUserDetailsService(userService); // Setting our custom user details service<br>        provider.setPasswordEncoder(passwordEncoder); // Setting the password encoder<br>        return provider;<br>    }<br>}</pre><p>In this SecurityConfig, we start by marking the configuration with @Configuration, @EnableWebSecurity, and @EnableMethodSecurity, which signal to Spring Framework to consider this class for security configurations.</p><h4><strong>Key Components of SecurityConfig:</strong></h4><ol><li><strong>JwtAuthFilter Integration:</strong> We incorporate JwtAuthFilter into the security chain to ensure that every request is checked for a valid JWT, except for the whitelisted URLs.</li><li><strong>Whitelist URLs: </strong>The WHITE_LIST_URL array defines which endpoints are accessible without authentication. In our streamlined version, we&#39;ve limited this to essential authentication-related paths to ensure clarity and security. This approach minimizes the surface area for unauthorized access while allowing users to register and log in.</li><li><strong>Security Filter Chain:</strong> Here, we define the rules governing which requests require authentication and which don’t. We enforce a stateless session management strategy, apt for JWT-based authentication, ensuring no session state is stored server-side.</li><li><strong>Custom Authentication Provider:</strong> By defining a custom AuthenticationProvider, we integrate Spring Security with our user service and password encoder. This allows us to use our user database and password standards for authentication.</li><li><strong>CSRF Protection:</strong> By default, CSRF (Cross-Site Request Forgery) protection is enabled in Spring Security. However, in a stateless API (typical with JWT usage), we disable CSRF protection as it’s not required and could complicate our API interactions.</li></ol><p>In the grand design of SecurityConfig, every detail from user authentication to request filtering is meticulously planned and executed. This ensures that only authenticated users can access secured endpoints, while also providing a clear path for user registration and login. With this blueprint, our Spring application&#39;s security is not just an afterthought, but a well-integrated fortress, ready to defend against the threats lurking in the shadows of the digital world.</p><h3>Securing the Future: Final Reflections</h3><p>In conclusion, our journey through the integration of Spring Security and JWT in a Spring Boot framework has provided a foundation on which you can build robust security mechanisms for your web applications. Armed with the knowledge of configuring security dependencies, defining user and role entities, implementing authentication filters, and establishing secure configurations, you are now equipped to extend this structure with your own service and controller classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/0*33ezrm1P8kvqU3zc" /></figure><p>Remember, the security of your application is as strong as the commitment you put into maintaining and updating these components. As you move forward, leverage this framework to protect your users’ data and ensure a safe, secure user experience. Happy coding, and may your applications remain secure and resilient against threats!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dfd20040fbd6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Swagger Like a Pro with Spring Boot 3 and Java 17]]></title>
            <link>https://medium.com/@berktorun.dev/swagger-like-a-pro-with-spring-boot-3-and-java-17-49eed0ce1d2f?source=rss-759f402bb906------2</link>
            <guid isPermaLink="false">https://medium.com/p/49eed0ce1d2f</guid>
            <dc:creator><![CDATA[Berk Torun]]></dc:creator>
            <pubDate>Thu, 15 Feb 2024 11:47:45 GMT</pubDate>
            <atom:updated>2024-02-15T11:47:45.855Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*Avh-XF-lc2mtXYL4G0pr1w.png" /></figure><h3>Greetings, Developers!</h3><p>Dive into the exciting world of API development as we embark on a journey to integrate Swagger UI with Spring Boot 3.2.2 and Java 17. Swagger isn’t just any tool; it’s your gateway to creating interactive, understandable, and testable API documentation effortlessly. It’s about making your API more accessible and your development process smoother.</p><p>So, buckle up! We’re about to simplify your API adventure and make documentation a breeze. Let’s get our hands dirty and transform how you build and document APIs with Swagger and Spring Boot.</p><h3>Let’s Get Rolling!</h3><p>Before we jump into the deep end, let’s make sure you’re all set up. You’ll need Java 17 and Spring Boot 3.2.2 warming up in your development environment. These are the building blocks we’ll be using to construct our Swagger-infused masterpiece.</p><p>Now, let’s talk about a game-changer: the springdoc-openapi-starter-webmvc-ui dependency. This little gem is the bridge between Spring Boot and Swagger UI, making your API&#39;s documentation look like a piece of art. It automatically generates and serves your API docs in a user-friendly web interface, making your API&#39;s functionalities clear and accessible.</p><h4>Spring into Action: Setting Up the Maven Dependency</h4><p>Adding the springdoc-openapi-starter-webmvc-ui to your project is like inviting a superhero into your code. Simply pop open your pom.xml and include the following lines:</p><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;org.springdoc&lt;/groupId&gt;<br>    &lt;artifactId&gt;springdoc-openapi-starter-webmvc-ui&lt;/artifactId&gt;<br>    &lt;version&gt;2.0.2&lt;/version&gt;<br>&lt;/dependency&gt;</pre><p>This tells Maven, “Hey, we’re doing some serious Swaggering here!” It ensures that your Spring Boot 3.2.2 application gets all the Swagger goodness it needs. Remember, version harmony is key here; 2.0.2 of the Springdoc dependency plays nicely with Spring Boot 3.2.2.</p><h4>Configuring Security with Swagger in Mind</h4><p>In the digital world, a castle is only as good as its defenses. Similarly, an API is only as good as its security measures. But don’t worry, we won’t be sacrificing our Swagger UI’s accessibility in the name of security. It’s all about finding that sweet balance.</p><p>To do this, we’ll tweak our SecurityConfig class a bit. Think of this class as the guardian of your API kingdom. Here’s a simple guide to update your security settings:</p><ol><li>Open your SecurityConfig class.</li><li>Add the Swagger UI’s URLs to your whitelist, just like adding VIP guests to an exclusive party list:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IrQ48GFDAG7svUsLnrXOJA.png" /><figcaption>Basic SecurityConfig class</figcaption></figure><p>Each URL here serves a purpose:</p><ul><li>/swagger-ui/**: This is your API&#39;s public face, the interactive documentation that Swagger UI provides.</li><li>/v3/api-docs/**: Think of this as the backbone of your Swagger UI, where all the API details live.</li></ul><h4>Unleashing Swagger UI</h4><p>Now, let’s set the stage for your API’s grand debut by configuring the SwaggerConfig class. This class is like the director of a play, ensuring that every scene (or API endpoint) is presented perfectly.</p><ol><li>In your SwaggerConfig class, we create the publicApi bean. It&#39;s like organizing your API&#39;s public appearances, making sure they&#39;re grouped under the &quot;public-apis&quot; banner for all to see</li><li>Next up, the customOpenAPI bean. This is where you get to tell your API&#39;s story, setting the title, version, and even adding some security flair with JWT</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jOc8v5l4o5O5ROi_PVwI9g.png" /><figcaption>Basic SwaggerConfig class</figcaption></figure><p>Think of these beans as your API’s resume, highlighting its best features and how to access them securely.</p><h4>Final Touches and Swagger Exploration</h4><p>You’re almost at the finish line! Let’s bring your API onstage:</p><ol><li>Run your Spring Boot application. Just like starting a car, this powers up your API’s engine.</li><li>Navigate to http://localhost:8080/swagger-ui.html – this is the VIP entrance to your API documentation.</li><li>Explore! Swagger UI allows you to see your API’s endpoints, try them out in real-time, and understand their responses. It’s like having a conversation with your API.</li></ol><p>Encourage your readers to play around with Swagger UI, experiment with different endpoint parameters, and see the immediate feedback. It’s a fantastic way to learn about your API’s capabilities and ensure it’s performing as expected.</p><h4>Wrapping Up</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*I4QEgi4x28dj_kyjGZpUmw.gif" /></figure><p>And that’s a wrap, developers! We’ve navigated through integrating Swagger with Spring Boot, highlighting the route to better API documentation. Remember, quality documentation is essential for seamless communication between your API and its users.</p><p>I encourage you to incorporate Swagger into your Spring Boot projects, enhancing your APIs with clarity and efficiency. It’s an investment in your project’s future and usability.</p><p>As we conclude this guide, I’m eager to hear from you. Feel free to share your experiences, questions, or feedback. Let’s keep the conversation going and support each other in our development journeys.</p><p>Keep coding, keep documenting, and most importantly, keep growing. Until next time, happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49eed0ce1d2f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Navigating the Complex World of Singletons in Multithreaded Applications]]></title>
            <link>https://medium.com/@berktorun.dev/navigating-the-complex-world-of-singletons-in-multithreaded-applications-867bb03eed02?source=rss-759f402bb906------2</link>
            <guid isPermaLink="false">https://medium.com/p/867bb03eed02</guid>
            <dc:creator><![CDATA[Berk Torun]]></dc:creator>
            <pubDate>Mon, 15 Jan 2024 11:57:40 GMT</pubDate>
            <atom:updated>2024-01-15T11:57:40.228Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/284/1*u5TpGQ2tui8GqlcqTv-BEg.png" /></figure><p><em>Dive into the world of software design where the Singleton pattern emerges as a crucial player, especially in the complex landscape of multithreaded applications. This journey explores the Singleton pattern — a design principle ensuring a single instance of a class with global access. We’ll examine its role, challenges, and strategies for effective implementation in environments where multiple threads intersect and interact.</em></p><h3>Lone Wolf in the Pack: What’s a Singleton Pattern?</h3><p>Imagine you have a unique object in your software, a bit like a lone wolf in a pack, distinct and singular. This is where the Singleton Pattern comes into play in the world of software design. But what exactly is this Singleton Pattern?</p><p>In essence, the Singleton Pattern is a design principle used in software development to ensure that a class has only one instance while providing a global point of access to this instance. It’s like having a single, exclusive club member who’s accessible to everyone but remains the only member of its kind.</p><p>Why is this useful, you might wonder? Well, there are several reasons:</p><ol><li><strong><em>Consistency:</em></strong> By having a single instance, the Singleton ensures that all operations are performed on the same instance, maintaining consistency across your application.</li><li><strong><em>Controlled Access:</em></strong> It provides controlled access to the sole instance. This way, you can manage its use and ensure it’s used appropriately throughout your software.</li><li><strong><em>Resource Management:</em></strong> Sometimes, creating multiple instances of a class can be resource-intensive. A Singleton can help conserve resources by limiting the number of instances to just one.</li><li><strong><em>Global State Management:</em></strong> Singleton patterns are often used to manage global states in applications, acting as a central point for configuration and data management.</li></ol><p>Think of it as having one remote control for your smart home. Instead of needing separate remotes for each device, you have one central control point that manages everything.</p><p>In the next sections, we’ll dive into the nitty-gritty of how the Singleton pattern navigates the tricky waters of multithreaded environments. Stay tuned to learn about the challenges and solutions when Singleton meets multithreading!</p><h3>When Threads Tangle: The Singleton and Multithreading Conundrum</h3><p>Now that we’ve met our lone wolf, the Singleton, let’s introduce it to a forest full of threads. In the world of software, threads are like individual paths of execution, and in a multithreaded environment, these paths often cross and intertwine. This is where our Singleton faces a new challenge: staying unique in a land of concurrent threads.</p><h4>The Challenge</h4><p>Imagine if our lone wolf had to deal with a pack all trying to befriend it at the same time. In a similar sense, when multiple threads try to access or create the instance of a singleton class simultaneously, chaos ensues. If not handled properly, each thread might end up creating its version of the singleton object. This defeats the whole purpose of the Singleton pattern, which is to have one and only one instance.</p><h4>Why It’s Tricky</h4><p>Singletons are straightforward in a single-threaded environment: you create an instance if it doesn’t exist, and if it does, you just use the existing one. But in a multithreaded scenario, two threads can simultaneously check if the singleton instance exists, find it doesn’t, and each creates a new instance.</p><h4>Thread Safety in Singleton</h4><p>To prevent such a scenario, we need to make our Singleton thread-safe. Thread safety ensures that our singleton maintains its uniqueness, even when multiple threads attempt to create an instance at the same time. This typically involves synchronization techniques, ensuring that only one thread can create the singleton instance at a time.</p><h4>Common Solutions</h4><ul><li><strong><em>Synchronized Access:</em></strong> One common approach is to synchronize the method that creates the instance. However, this can lead to performance issues as threads wait for their turn to access the method.</li><li><strong><em>Double-Checked Locking:</em></strong> A more performance-friendly technique involves double-checked locking. It first checks if an instance exists, then synchronizes the creation process, and checks again, ensuring that only one instance gets created.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AbNXvjC31PfI00vHmIY_eA.png" /><figcaption>Java Code for Thread-Safe Singleton: This snippet shows a thread-safe Singleton pattern using double-checked locking in Java, ensuring efficient and safe instance creation in multithreaded environments.</figcaption></figure><ul><li><strong><em>Eager Initialization:</em></strong> Another approach is eager initialization, where the singleton instance is created beforehand. This can ensure thread safety but might lead to resource wastage if the instance is never used.</li><li><strong><em>Using Atomic References:</em></strong> Modern programming languages offer atomic references and operations, providing another way to ensure that the instance is created only once, even in a multithreaded environment.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4ClXxLGKCl3j6gvBaAeyuA.png" /><figcaption>Java Singleton with Lazy Initialization and Atomic Reference: This code demonstrates a thread-safe Singleton pattern using an AtomicReference for lazy initialization in a multithreaded environment, ensuring thread safety without explicit synchronization blocks.</figcaption></figure><h4>The Trade-offs</h4><p>While ensuring thread safety is crucial, it’s also important to consider the trade-offs in performance and complexity. The more complex the synchronization mechanism, the harder it is to maintain and understand the code. And with stringent synchronization, performance can take a hit, especially if the singleton is accessed frequently.</p><h3><strong>Safely Juggling Threads: Making Singleton Thread-Safe</strong></h3><p>After navigating the complex terrain where singletons meet multithreading, it’s time to master the craft of keeping our singleton thread safe amidst the bustling activity of multiple threads. This is akin to a juggler deftly handling several balls in the air — each ball representing a thread that could potentially disrupt our singleton’s uniqueness.</p><h4>Implementing Thread-Safe Singletons</h4><p>Ensuring that a singleton remains the only one of its kind in a multithreaded environment involves more than just recognizing potential problems; it requires practical solutions. Here are some effective strategies:</p><ol><li><strong><em>Lazy Initialization with Volatile Keyword:</em></strong> In languages like Java, you can use the volatile keyword with lazy initialization. This ensures that the singleton instance is created only when it&#39;s needed, and the volatile keyword guarantees that the instance is correctly handled during concurrent access.</li><li><strong><em>Static Inner Class:</em></strong> This method leverages the Java Classloader mechanism. The Singleton Holder pattern utilizes a static inner class that holds the singleton instance. This class is not loaded into memory until the instance is explicitly requested, providing a thread-safe solution without explicit synchronization.</li><li><strong><em>Using Thread Local Storage:</em></strong> For some use cases, Thread Local Storage (TLS) can be an effective way to ensure thread safety. It allows storing data that is accessible only by a specific thread, though it doesn’t provide a global singleton but a thread-contained singleton.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vtg_nlJzymlbni-GiGvQKQ.png" /><figcaption>Java Singleton Using Static Inner Class: Demonstrates creating a thread-safe Singleton with lazy initialization, utilizing the classloader for safe instance creation in multithreaded environments.</figcaption></figure><h4>Best Practices for Thread-Safe Singleton</h4><ul><li><strong><em>Minimize Synchronization Overhead:</em></strong> Overuse of synchronization can lead to performance bottlenecks. Use it judiciously and only when necessary.</li><li><strong><em>Immutable State:</em></strong> If the state of the singleton does not change after its creation, it naturally becomes easier to manage in a multithreaded environment.</li><li><strong><em>Lazy Loading:</em></strong> Lazy loading of the singleton instance can help in resource management, especially when the instance creation is resource-intensive.</li><li><strong><em>Consider Scalability:</em></strong> Think about how your singleton will perform as your application scales. Will the thread safety mechanism still hold up under increased load?</li></ul><h4>Testing for Thread Safety</h4><ul><li><strong><em>Stress Testing:</em></strong> Put your singleton through rigorous stress tests to ensure it maintains its integrity under concurrent access.</li><li><strong><em>Code Reviews and Analysis:</em></strong> Regularly review and analyze your code for potential thread safety issues. Tools and static code analyzers can aid in this process.</li></ul><h4>The Balancing Act</h4><p>In conclusion, making a singleton thread safe is a delicate balancing act. It’s about ensuring safety and uniqueness while maintaining performance and scalability. Like a skilled juggler, a developer must keep all these aspects in harmony to create a robust and efficient singleton in a multithreaded environment.</p><h3><strong>Designing with Care: Singleton in the Multithreaded World</strong></h3><p>In the intricate dance of software design, introducing a singleton into a multithreaded environment is like adding a soloist to an orchestra. It requires careful consideration to ensure harmony. Let’s explore the key design principles and practices to gracefully integrate singletons in multithreaded applications.</p><h4>Key Design Considerations</h4><ol><li><strong><em>Understand the Use Case:</em></strong> Before implementing a singleton, ask yourself if it’s truly necessary. Singletons are powerful but can lead to issues if used inappropriately. They are best suited for scenarios where a single, shared resource or service is required.</li><li><strong><em>Evaluate the Impact on Application State:</em></strong> Singletons, by their nature, hold state. In a multithreaded environment, this can lead to unpredictable behavior if the state is not managed carefully. Ensure that any state change is thread-safe and predictable.</li><li><strong><em>Dependency Injection (DI):</em></strong> Consider using dependency injection to manage your singleton instances. DI frameworks can handle the lifecycle and dependencies of singletons, making them easier to test and maintain.</li></ol><h4>Best Practices in Implementation</h4><ol><li><strong><em>Avoid Global State:</em></strong> While singletons can be used to store global state, this practice is generally discouraged. Global states can make testing difficult and lead to code that’s hard to debug and maintain.</li><li><strong><em>Ensure Lazy Initialization:</em></strong> Lazy initialization, where the instance is created when it’s first needed, can improve performance and resource management. This is particularly important in multithreaded applications where resources are at a premium.</li><li><strong><em>Design for Testability:</em></strong> Ensure that your singleton design doesn’t hinder testing. Consider patterns that allow you to replace singleton instances during testing, such as using interfaces or dependency injection.</li><li><strong><em>Document Usage and Access Patterns:</em></strong> Document how and when the singleton should be used, especially in a team environment. This helps to avoid misuse and makes the codebase more maintainable.</li></ol><h4>Thread Safety Strategies</h4><ul><li><strong><em>Immutable Singletons:</em></strong> If possible, make your singleton immutable after initialization. Immutability is a natural defense against many multithreading issues.</li><li><strong><em>Consider Using Advanced Concurrency Controls:</em></strong> In some programming languages, there are advanced concurrency control mechanisms such as locks, semaphores, or atomic operations. Utilizing these can help manage access to the singleton in a more refined way, particularly in complex multithreaded scenarios.</li></ul><h4>Scalability and Performance</h4><ul><li><strong><em>Monitor Performance Impact:</em></strong> Regularly monitor the impact of the singleton on your application’s performance. In a multithreaded environment, the overhead of accessing a singleton, especially if it involves synchronization, can become a bottleneck.</li><li><strong><em>Scalability Concerns:</em></strong> Consider how the singleton will scale with your application. Will it remain efficient and reliable as the number of threads increases? If scalability becomes an issue, it might be time to reconsider the use of a singleton.</li></ul><h4>Handling Singleton Lifecycle</h4><ul><li><strong><em>Manage Instance Lifecycle:</em></strong> Pay attention to the lifecycle of your singleton instance. In some environments, like web applications, you might need to handle scenarios where the application is restarted or reloaded.</li><li><strong><em>Singleton Destruction: </em></strong>In languages like C++, where you have control over object destruction, think about when and how the singleton should be destroyed. Proper cleanup is essential to avoid resource leaks.</li></ul><h3><strong>Singletons in Action: Real-World Tales</strong></h3><p>In the world of software engineering, theory and practice often meet in surprising ways. Let’s explore some real-world examples where the singleton pattern has not only been implemented but has thrived within multithreaded environments, shedding light on its practical uses and effectiveness.</p><h4>Case Study 1: Database Connection Pooling</h4><ul><li><strong><em>Scenario:</em></strong> A web application requires access to a database, and creating a database connection is resource-intensive.</li><li><strong><em>Implementation:</em></strong> The singleton pattern is used to create a database connection pool. This pool manages a finite number of connections that can be reused by multiple threads.</li><li><strong>Outcome:</strong> Improved performance and resource utilization. The singleton ensures that connections are efficiently managed and shared among multiple threads, reducing the overhead of creating and closing connections frequently.</li></ul><h4>Case Study 2: Logging Mechanism</h4><ul><li><strong><em>Scenario:</em></strong> An application needs a consistent way to log messages for debugging and monitoring purposes.</li><li><strong><em>Implementation:</em></strong> A logger class is implemented as a singleton, providing a centralized mechanism to log messages from different parts of the application.</li><li><strong><em>Outcome:</em></strong> Consistent and centralized logging. The singleton pattern ensures that all components of the application use the same logging service, which simplifies maintenance and analysis.</li></ul><h4>Case Study 3: Configuration Management</h4><ul><li><strong><em>Scenario:</em></strong> A distributed system requires access to shared configuration settings, like API keys or service URLs.</li><li><strong><em>Implementation:</em></strong> The singleton pattern is used to encapsulate the configuration settings, providing a single point of access.</li><li><strong><em>Outcome:</em></strong> Simplified configuration management. The singleton ensures that configuration settings are loaded once and distributed consistently across the application, preventing discrepancies and repeated I/O operations.</li></ul><h4>Insights from Real-World Applications</h4><p>These examples demonstrate the versatility and practicality of the singleton pattern in various scenarios. From managing shared resources to ensuring consistent access to critical services, singletons have proven to be invaluable in multithreaded environments. However, it’s also evident that careful consideration must be given to ensure that the singleton does not become a bottleneck or a point of failure.</p><h4>Key Takeaways</h4><ul><li><strong><em>Appropriate Use:</em></strong> Singleton should be used judiciously, only when there is a clear need for a single instance.</li><li><strong><em>Design Flexibility:</em></strong> The implementation of singletons can vary based on the specific requirements of the application, showing the pattern’s flexibility.</li><li><strong><em>Performance Considerations:</em></strong> While singletons can offer performance benefits, it’s essential to monitor and optimize their usage to prevent potential issues in a multithreaded context.</li></ul><h3><strong>The Double-Edged Sword: Pros and Cons of Singleton in Threads</strong></h3><p>The singleton pattern, like any tool in software engineering, comes with its own set of benefits and challenges, especially when employed in multithreaded scenarios. Let’s dissect this double-edged sword to understand the trade-offs involved.</p><h4>Pros of Singleton in Multithreaded Environments</h4><ol><li><strong><em>Controlled Access and Consistency:</em></strong> Singleton provides a controlled and consistent access point to a resource across multiple threads. This is crucial in maintaining state consistency and avoiding conflicts in resource usage.</li><li><strong><em>Resource Management:</em></strong> By ensuring only one instance of a resource-intensive class, singletons help in efficient resource management, reducing memory footprint and load time.</li><li><strong><em>Global Access:</em></strong> Singletons offer a globally accessible instance, which can be an advantage for resources that need to be accessed throughout the application, like configuration settings or logging mechanisms.</li></ol><h4>Cons of Singleton in Multithreaded Environments</h4><ol><li><strong><em>Complexity in Thread Safety:</em></strong> Ensuring thread safety in singletons can add complexity to the code. Techniques like double-checked locking or using volatile variables require careful implementation to avoid subtle bugs.</li><li><strong><em>Scalability Concerns:</em></strong> Singletons can become a bottleneck in high-performance, multithreaded applications. When multiple threads need to access the singleton instance, it can lead to performance issues due to synchronization overhead.</li><li><strong><em>Testing Difficulties:</em></strong> Singletons can pose challenges in unit testing because they carry states across different tests and can lead to dependencies that are hard to mock or isolate.</li><li><strong><em>Potential for Misuse:</em></strong> The ease of access to a singleton can lead to its misuse, such as using it to store global state inappropriately, leading to code that is difficult to maintain and debug.</li></ol><h4>Balancing the Pros and Cons</h4><p>Understanding these pros and cons is key to deciding when and how to use singletons. The singleton pattern should be employed when there is a clear and justified need for a single instance of a class. It’s also crucial to mitigate the cons through careful design and implementation, such as using dependency injection for better testability or avoiding unnecessary global states.</p><h3><strong>Beyond Singleton: Exploring Alternatives</strong></h3><p>While the singleton pattern has its place, it’s not always the best solution, especially in multithreaded applications where its drawbacks can become more pronounced. Let’s explore some alternative design patterns and solutions that can offer similar benefits without some of the pitfalls associated with singletons.</p><h4>1. Dependency Injection (DI)</h4><ul><li>Concept: Instead of having global access to an instance, objects receive their dependencies from an external source (typically a DI framework).</li><li>Benefits: DI enhances the modularity, testability, and maintainability of the code. It allows for better management of dependencies and decouples the object creation from the business logic.</li></ul><h4>2. Service Locator Pattern</h4><ul><li>Concept: Similar to a singleton in providing a central access point, but instead of a single instance, it locates services and provides them on demand.</li><li>Benefits: This pattern reduces coupling compared to singletons and improves the flexibility of the application.</li></ul><h4>3. Factory Pattern</h4><ul><li>Concept: Use a factory class to create instances of objects, rather than directly calling a constructor or using a singleton.</li><li>Benefits: Provides more control over the creation process and can support scenarios where multiple instances are needed, or instances need to be created based on certain conditions.</li></ul><h4>4. Thread-Local Storage</h4><ul><li>Concept: Data is stored on a per-thread basis. Each thread has its instance of a variable.</li><li>Benefits: Eliminates the need for synchronization and prevents shared-state issues, making it suitable for certain multithreading scenarios.</li></ul><h4>5. Object Pool Pattern</h4><ul><li>Concept: Pre-instantiate and maintain a pool of objects that can be borrowed and returned rather than creating and destroying them on demand.</li><li>Benefits: This pattern is useful in managing resources in multithreaded environments, reducing the overhead of resource creation and garbage collection.</li></ul><p>Each of these alternatives has its own set of trade-offs and is suitable for different scenarios. The choice largely depends on the specific requirements of the application, the problem being solved, and the overall architecture. By considering these alternatives, developers can choose the most appropriate pattern that aligns with their application’s needs and constraints.</p><h3><strong>Wrapping It Up: Singleton in a Nutshell</strong></h3><p>As we conclude our exploration of the Singleton pattern in multithreaded structures, let’s encapsulate the essential insights and lessons we’ve gathered along this journey. The Singleton pattern, a stalwart in the design pattern landscape, has both its fervent advocates and critics, especially when it comes to multithreaded applications.</p><h4>Key Takeaways</h4><ol><li><strong><em>Singleton Basics:</em></strong> Singleton is a design pattern that ensures a class has only one instance, providing a global point of access to it. It’s like having one authoritative manager in a system.</li><li><strong><em>Multithreading Challenges:</em></strong> In a multithreaded environment, ensuring that a class remains a true singleton requires careful handling to avoid issues like creating multiple instances by different threads.</li><li><strong><em>Thread Safety Techniques:</em></strong> Ensuring thread safety in singletons involves strategies like synchronized access, lazy initialization, double-checked locking, and the use of volatile variables, each with its trade-offs.</li><li><strong><em>Design Considerations:</em></strong> When implementing a singleton, consider the impact on application design, including aspects like resource management, scalability, and testing.</li><li><strong>Real-World Applications:</strong> Singletons find their use in various scenarios like database connection pools, logging mechanisms, and configuration managers, but they should be used judiciously.</li><li><strong><em>Pros and Cons:</em></strong> The pattern offers controlled access and resource efficiency but can introduce complexity, hinder scalability, and pose challenges in testing.</li><li><strong><em>Alternatives to Singleton:</em></strong> Other design patterns, such as Dependency Injection, Factory Pattern, and Service Locator, offer different ways to manage instances and can sometimes be more suitable in complex multithreaded environments.</li></ol><h4>Final Words</h4><p>The Singleton pattern, with its unique characteristics and challenges, continues to be a valuable tool in certain situations, particularly where a single instance of a resource or service is necessary. However, its usage in multithreaded environments demands careful consideration and often, creative solutions to ensure thread safety, maintainability, and efficiency. Remember, the choice to use a singleton should be driven by specific needs and contexts. It’s not a one-size-fits-all solution but a strategic decision that can have significant implications for your application’s architecture and performance.</p><h4>Embracing Flexibility and Prudence</h4><p>As technology evolves and software design paradigms shift, the Singleton pattern, like any other pattern, should be adapted and employed with a clear understanding of its strengths and limitations. Always be open to alternatives and continuously evaluate whether a pattern fits the evolving needs of your application.</p><p>In the end, the art of software design lies in choosing the right tool for the right job, and sometimes, the most powerful tool is the knowledge and discernment to use or not use a pattern like Singleton.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=867bb03eed02" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Couchbase: Sizi şöyle kanepeye alalım!️]]></title>
            <link>https://medium.com/@berktorun.dev/couchbase-sizi-%C5%9F%C3%B6yle-kanepeye-alal%C4%B1m-%EF%B8%8F-8db8e3b22009?source=rss-759f402bb906------2</link>
            <guid isPermaLink="false">https://medium.com/p/8db8e3b22009</guid>
            <category><![CDATA[couchbase]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Berk Torun]]></dc:creator>
            <pubDate>Fri, 08 Dec 2023 06:01:54 GMT</pubDate>
            <atom:updated>2023-12-08T06:01:54.580Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/430/1*xzX3Kc0NCyxNPT0Sqt7LHA.jpeg" /></figure><p><em>Sizlere son zamanlarda oldukça popüler olan ve günden güne popülaritesini arttıran Couchbase’den bahsedeceğim. Nedir? Ne değildir? Neden bu kadar tercih edilmeye başlandı? Bu soruları temel düzeyde ve kendimce önemli olduğunu düşündüğüm şekilde anlatmaya çalışacağım. Hazırsanız sizi şöyle kanepeye alalım!</em></p><h3>Nedir bu Couchbase?🤔</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/1*BtPbMilBDklRJhoB98Wmkg.gif" /></figure><p>Couchbase, etkileşimli web uygulamaları için bir NoSQL veritabanıdır. Esnek veri modeline sahiptir, kolayca ölçeklendirilebilir, tutarlı ve yüksek performans sağlar aryıca uygulama verilerini %100 çalışma süresiyle sunma kapasitesine sahiptir.</p><h3><strong>Neden Couchbase?</strong></h3><h4>1. PERFORMANS KREM!!!🚀</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/374/1*Ozymgb55ZVdO4BGWaAVQ1g.gif" /></figure><p>Couchbase’in tercih edilmesinin en önemli sebeplerinin başında sahip olduğu “<em>memory-first”</em> mimari geliyor. Bu mimariyi basitçe birkaç maddede anlatalım.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*JtcxKk5rtPCYnlNWM7kZBg.png" /></figure><ol><li><strong><em>Yüksek Performans:</em></strong> Verilerin çoğu bellekte saklandığından, sık erişilen verilere hızlı erişim sağlanır. Bellek, diskten çok daha hızlıdır, bu da veriye erişim sürelerini dramatik bir şekilde azaltır.</li><li><strong><em>Düşük Gecikme Süresi:</em></strong> Verilerin bellekte bulunması, uygulamaların taleplerine hızlı cevap verebilmesini sağlar. Bu da kullanıcı deneyimini olumlu yönde etkiler.</li><li><strong><em>Ölçeklenebilirlik:</em></strong> Bellek odaklı mimari, sistemin daha fazla bellek eklenerek ölçeklendirilebilir olmasını sağlar. Bu, artan kullanıcı taleplerine veya veri hacmine kolayca adapte olmayı mümkün kılar.</li><li><strong><em>Yüksek Erişilebilirlik:</em></strong> Verilerin diskte de kalıcı olarak depolanması, sistem yeniden başladığında veya beklenmedik bir durumda oluşan kesintilerde bile veri kaybı yaşanmamasını sağlar.</li><li><strong><em>Performansın Kalıcılığı:</em></strong> Bellek ve disk arasındaki bu entegrasyon, hızlı erişim sağlarken aynı zamanda verilerin kalıcı olarak saklanmasını da mümkün kılar. Bu, hızlı yanıtlarla birlikte veri güvenliğini de sağlar.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*49bawA_nu0wePOA3DFbp_g.png" /></figure><p>Ayrıyeten Couchbase’in sahip olduğu asenkron yapı aynı memory-first mimarisi gibi performansa olumlu katkıda bulunuyor. Bu mimaride diske yazma işlemi asenkron olarak yapılır. Bu değiştirilebilir, ancak kritiktir. Önemli veriler veya kaybolması göze alınamayacak veriler (para transferi veya sipariş kaydı gibi) varsa, dayanıklılık(durability) seçeneğinin ayarlanması gerekebilir. Bunun nedeni, Couchbase’de verinin belleğe yazıldığı anda client’a bir commit göndermesi ve verinin gerçekten işlendiğini düşünmenizi sağlamasıdır. Daha sonra asenkron olarak çoğaltma ve diske yazma işlemi başlar. Ancak bu sırada elektrik kesintisi, sunucunun kapalı kalması veya hizmetin çökmesi durumunda bellekte bulunan ve diske yazılmayan veriler kaybolur. En nihayetinde bu bize tehlikeli bir takas sunmuş oluyor ancak kritik durumlar söz konusuysa mimari buna göre tekrardan tasarlanabilir.</p><p>İşlemler ilk önce memcache’de yürütülür ve işlemler gerçekleştirilir gerçekleştirilmez client’a bir commit gönderilir. Diğer tüm işlemler iç düzeyde asenkron olarak gerçekleştirilir. Sonuç olarak hem okuma hem de yazma hızları çok hızlıdır.</p><h4>2. XDCR(Cross Data Center Replication)🔄</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qAJhWwQQdeWMRo3HS60kUg.png" /></figure><p>Bu özellk en temelde verilerin potansiyel olarak farklı veri merkezlerinde bulunan clusterlar arasında veri replikasyonuna olanak tanır. Bu özellik, verilerin bir clusterdan diğerine otomatik olarak kopyalanmasını ve senkronize edilmesini mümkün kılar. Birkaç maddede XDCR özelliğinin sağladığı faydalar şunlardır:</p><ol><li><strong><em>Yüksek Erişilebilirlik ve Kurtarma Senaryoları:</em></strong> Verilerin farklı coğrafi bölgelerdeki sunucular arasında otomatik olarak kopyalanması, bir sunucunun çökmesi durumunda bile veri kaybını engeller. Bu, felaket kurtarma süreçlerinde veri kaybını önler ve hızlı bir şekilde geri dönüş sağlar.</li><li><strong><em>Yük Dengesi ve Performans:</em></strong> XDCR, yüksek trafikli veriler için farklı sunucular arasında dağıtılmasını sağlar. Bu, yükü dengeler ve genel performansı artırır. Ayrıca, kullanıcıların farklı coğrafi bölgelerden veriye erişmesini hızlandırır.</li><li><strong><em>Küresel Veri Dağıtımı:</em></strong> Farklı coğrafi bölgelerdeki kullanıcıların veriye erişimini iyileştirir. Örneğin, bir kullanıcı Asya’da veriye erişmeye çalışırken, veri kopyaları yakın bir sunucuda bulunduğundan daha hızlı erişebilir.</li><li><strong><em>Yedekleme ve Veri Koruma:</em></strong> Verilerin farklı coğrafi bölgelerde yedeklenmesi, veri kaybını önler. Bu, veri güvenliğini ve korumasını artırır.</li><li><strong><em>Esneklik:</em></strong> XDCR, yapılandırılabilir ve esnektir. İhtiyaçlara ve kullanım senaryolarına göre yapılandırılabilir.</li></ol><p>Bu faydalar, özellikle büyük ölçekli uygulamalar için önemlidir ve veri güvenliği, yedekleme ve kullanılabilirlik konularında Couchbase’in sağladığı avantajları artırır.</p><h4>3. N1QL: JSON + SQL🫂</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/497/1*9OaztrgqhimR5Q_11cBSsw.gif" /></figure><p>N1QL (Not One Query Language), Couchbase için SQL benzeri bir sorgu dili ve arabirimdir. N1QL, Couchbase üzerindeki JSON dökümanları üzerinde SQL benzeri sorgular yapılmasını sağlar. Bu sayede geleneksel ilişkisel veritabanlarında olduğu gibi, kolayca anlaşılabilir ve kullanılabilir sorgular ile belgeler arasında sorgu yapabilirsiniz.</p><p>N1QL’in sağladığı bazı avantajlar şunlardır:</p><ol><li><strong><em>SQL-Like Sorgular:</em></strong> N1QL, SQL benzeri bir yapıya sahiptir, bu da geleneksel veritabanlarındaki gibi sorgular yapmayı kolaylaştırır. Bu sayede verileri sorgulamak ve analiz etmek için SQL bilen kişilerin kolayca adapte olmasını sağlar.</li><li><strong><em>JSON Dökümanları ile Uyumlu:</em></strong> Couchbase, JSON tabanlı dökümanlar üzerinde çalışır ve N1QL bu dökümanlara yönelik sorgular yapmayı mümkün kılar. Bu, geliştiricilerin verileri daha doğal ve rahat bir şekilde sorgulamasına olanak tanır.</li><li><strong><em>Esneklik ve Performans:</em></strong> N1QL, esnek bir sorgulama dilidir ve performanslı sorgular yapılmasına imkan tanır. Belge tabanlı veritabanlarının sunduğu avantajları, sorgulama dilinin esnekliğiyle birleştirir.</li><li><strong><em>Veri Manipülasyonu ve Analizi:</em></strong> N1QL, verileri sorgulama, filtreleme, eşleştirme ve analiz etme yetenekleri sunar. Bu, veriler üzerinde geniş kapsamlı operasyonlar gerçekleştirilmesini mümkün kılar.</li><li><strong><em>Entegrasyon Kolaylığı:</em></strong> Couchbase’in N1QL özelliği, mevcut SQL bilgisini ve sistemlerini kullanarak Couchbase ile entegrasyonu kolaylaştırır. Bu, var olan altyapıların Couchbase’e geçişini kolaylaştırır.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hHFumFZLs6-O7UMDkMgSHA.png" /><figcaption>SQL ve N1QL kullanımına bir örnek</figcaption></figure><p>N1QL, Couchbase’in döküman odaklı mimarisini destekleyen güçlü bir sorgu dili olarak, geliştiricilerin verileri işleme ve analiz etme süreçlerini optimize etmelerini sağlar. Bu da ölçeklenebilirlik, performans ve esneklik açısından büyük avantajlar sunar.</p><p>Bu yazımda anlatacaklarım bu kadardı. Bildiklerimi ve değerli gördüklerimi anlatmaya çalıştım. Buraya kadar okuduysanız şimdiden teşekkürlerimi sunuyorum. Umarım beğenmişsinizdir. Bir başka yazıda görüşmek üzere.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*ieERZTdCBvMTd3Obtjwmxw.gif" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8db8e3b22009" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Base, ACID ve CAP: Veritabanı Yaklaşımları]]></title>
            <link>https://medium.com/@berktorun.dev/base-acid-ve-cap-veritaban%C4%B1-yakla%C5%9F%C4%B1mlar%C4%B1-ce5d329b8d7e?source=rss-759f402bb906------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce5d329b8d7e</guid>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Berk Torun]]></dc:creator>
            <pubDate>Mon, 04 Dec 2023 06:22:57 GMT</pubDate>
            <atom:updated>2023-12-04T06:28:04.401Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*dREVUpXenaBg6hHpsx-lcQ.jpeg" /></figure><p><em>Veritabanları, bilginin depolanması, erişilmesi ve yönetilmesi açısından modern teknolojik dünyanın temel taşıdır. Ancak, bu veri yönetimi dünyasında hangi prensiplerin öne çıktığını, ne zaman kullanılması gerektiğini ve nasıl bir denge sağladığını anlamak önemlidir. İşte bu yazıda, veritabanı yönetimindeki önemli prensiplerden bazılarını, yani Base, ACID ve CAP’i yakından inceleyeceğiz.</em></p><h4>Base: Veri Yönetimindeki “Yumuşak Dokunuş” 🌟</h4><p>Veritabanı dünyasında, Base (Basically Available, Soft State, Eventually Consistent) prensibi, esneklik ve kullanılabilirlik üzerine odaklanır. Bu prensip, veri tutarlılığını anında sağlamak yerine, zamanla tutarlı hale getirme fikrine dayanır.</p><p>Base prensibi, anında kesinlik ve tutarlılık yerine, kullanılabilirlik ve esneklik sunar. Özellikle dağıtık sistemlerde, ağdaki gecikmeler veya bağlantı kopmaları nedeniyle anında tutarlılığın zor olabileceği durumlarda tercih edilir. Veri kopyaları arasındaki zaman içindeki uyumsuzluk, sonunda bir tutarlılık haline gelir.</p><p>Bu prensip, belirli durumlarda anında güncel veriye ihtiyaç duyulmadığında veya bazı aksaklıklara rağmen sistemin çalışabilirliği ön planda olduğunda kullanılır. Örneğin, sosyal medya platformları veya ağ tabanlı uygulamalar, bazı durumlarda anında güncel veriye ihtiyaç duymayabilir, bunun yerine zamanla tutarlı bir durumu tercih edebilirler.</p><p>Base prensibi, veri yönetiminde esneklik ve performansın dengeli bir şekilde sağlanmasına odaklanırken, kullanılabilirlik ve performans arasında bir denge kurmayı hedefler.</p><h4>ACID: Veri Dünyasının Güvenilir Kumandası 🔒</h4><p>Veritabanı yönetiminde, ACID (Atomicity, Consistency, Isolation, Durability) prensibi, veri işlemlerinin sağlamlığını ve güvenilirliğini temsil eder. Bu prensipler, veri işlemlerinin güvenilir ve tutarlı olmasını sağlamak için kritik öneme sahiptir.</p><ul><li>⚛️<strong>Atomicity (Atomiklik): </strong>İşlem ya tamamıyla gerçekleşir ya da hiç gerçekleşmez. Parçalanamaz bir bütün gibi düşünebiliriz. Örneğin, bir ödeme işlemi, para çekme ve para yatırma işlemlerinin her ikisi de tamamlandığında başarılı olmalıdır.</li><li>🧩<strong>Consistency (Tutarlılık):</strong> Veritabanı işlemleri, belirlenen kurallara uygun olmalı ve veri bütünlüğünü korumalıdır. Bu, bir işlem sonrası verinin bir anlık tutarlılık içinde olmasını ifade eder.</li><li>🛡️<strong>Isolation (İzolasyon):</strong> Eş zamanlı işlemler birbirlerinden bağımsız olmalı ve bir işlem diğer işlemlerden etkilenmemelidir. Bu, aynı anda gerçekleşen işlemlerin birbirlerini etkilememesi gerektiği anlamına gelir.</li><li>💎<strong>Durability (Kalıcılık):</strong> Veritabanı işlemleri kalıcı olmalı ve bir defa gerçekleştiğinde geri dönülmez olmalıdır. Veri kaybı olmamalıdır, hatta sistem çökse bile veri kaybolmamalıdır.</li></ul><p>ACID prensipleri, veritabanı yönetiminde güvenilirlik, tutarlılık ve bütünlük sağlar. Bu prensipler, veritabanı işlemlerinin doğruluğunu ve güvenilirliğini artırarak sistemlerin sağlıklı bir şekilde çalışmasını sağlar.</p><h4>Base ve ACID’in Karşılaştırılması: Hangisi Ne Zaman Tercih Edilmelidir?🤔</h4><p>Veritabanı yönetiminde, Base ve ACID prensipleri arasında bir denge kurmak önemlidir. Hangi prensibin kullanılacağı, ihtiyaçlara, sistem gereksinimlerine ve önceliklere bağlıdır.</p><ul><li><strong><em>Base (Basically Available, Soft State, Eventually Consistent)</em></strong>, genellikle dağıtık sistemlerde kullanılabilirlik ve esneklik için tercih edilir. Veriler arasında zaman içinde uyumsuzluk olabilir ancak zamanla tutarlılık sağlar. Özellikle anında tutarlılığın zor olduğu durumlarda tercih edilir.</li><li><strong><em>ACID (Atomicity, Consistency, Isolation, Durability)</em></strong> ise, işlemlerin kesinlikle güvenilir ve tutarlı olmasını gerektiren durumlarda önemlidir. Özellikle finansal işlemler gibi doğruluğun ve bütünlüğün kritik olduğu sistemlerde tercih edilir.</li></ul><p>Hangi prensibin tercih edilmesi gerektiği, projenin gereksinimlerine bağlıdır. Örneğin, sosyal medya platformları genellikle Base prensibini benimseyerek esneklik ve kullanılabilirliği ön planda tutarken, bankacılık sistemleri ACID prensibini tercih ederek güvenilirlik ve doğruluğu öne çıkarır.</p><p>Önemli olan, her iki prensibin de kendine özgü avantajları ve dezavantajları olduğu için projenin ihtiyaçlarına en uygun olanının seçilmesidir. Genellikle, performans ve tutarlılık arasında bir denge kurulması gereken durumlarda Base, veri bütünlüğü ve doğruluğun kritik olduğu senaryolarda ACID prensipleri daha çok tercih edilir.</p><h4>CAP Teoremi ve Dağıtık Sistemlerde Etkisi🌐</h4><p>Dağıtık sistemler, günümüzde pek çok büyük ölçekli uygulama ve hizmet için temel altyapıyı oluşturuyor. CAP Teoremi (Consistency, Availability, Partition Tolerance), bu dağıtık sistemlerin tasarımında belirleyici bir faktör olarak karşımıza çıkıyor.</p><ul><li>🧩<strong>Consistency (Tutarlılık):</strong> Veri bütünlüğünün ve tutarlılığın sağlanması anlamına gelir. Tüm kullanıcılar, aynı anda aynı veriyi görmelidir.</li><li>🚀<strong>Availability (Kullanılabilirlik):</strong> Sistem her zaman erişilebilir olmalıdır. Kullanıcılar istedikleri zaman sistemle etkileşime geçebilmelidir.</li><li>🧊<strong>Partition Tolerance (Bölünme Toleransı):</strong> Ağda oluşabilecek kesintiler veya bölünmeler durumunda bile sistem çalışmaya devam etmelidir.</li></ul><p>CAP Teoremi, bu üç prensibin aynı anda sağlanamayacağını iddia eder. Yani, bir dağıtık sistem sadece iki prensibi aynı anda sağlayabilir; üçüncüsünden biri tercih edilmelidir. Bu teorem, dağıtık sistemlerin tasarımında bir denge kurulması gerektiğini vurgular.</p><p>Örneğin, e-ticaret uygulamaları gibi kullanıcılara sürekli erişim sağlanması gereken sistemler, genellikle Availability (Kullanılabilirlik) üzerinde odaklanırken, bankacılık sistemleri gibi veri doğruluğunun kritik olduğu alanlar Consistency (Tutarlılık) prensibini ön planda tutarlar.</p><p>Bu teorem, dağıtık sistemlerin tasarımında hangi önceliğin belirleneceğini ve hangi durumda hangi prensibin tercih edilebileceğini anlamak açısından büyük önem taşır.</p><h4>Base, ACID ve CAP: Gerçek Hayat Uygulamaları ve Senaryoları📊</h4><p>Base, ACID ve CAP prensipleri, veritabanı yönetimi ve sistem tasarımı alanında farklı ihtiyaçlara ve senaryolara cevap verir. Bu prensiplerin gerçek hayattaki kullanımı, çeşitli sektörlerde ve uygulama senaryolarında farklılık gösterir.</p><ul><li>🌱<strong>Base:</strong> Dağıtık sistemlerde kullanılabilirlik ve esneklik ön plandadır. Sosyal medya platformları, bulut tabanlı uygulamalar veya ağ üzerindeki paylaşılan veriler gibi senaryolarda Base prensibi sıklıkla tercih edilir. Örneğin, Twitter gibi platformlarda anlık veri akışı, anında güncellik yerine zamanla tutarlılık üzerine odaklanabilir.</li><li>🔐<strong>ACID:</strong> Finansal işlemler, bankacılık sistemleri veya envanter yönetimi gibi alanlarda doğruluk ve tutarlılık ön plandadır. ACID prensipleri, veri bütünlüğünün kritik olduğu bu tür uygulamalarda tercih edilir. Örneğin, bir banka hesabı işlemi, kesinlikle tutarlı olmalı ve her adımda doğrulanabilir olmalıdır.</li><li>🌐<strong>CAP:</strong> Dağıtık sistemlerin karmaşıklığına ve gereksinimlerine bağlı olarak, CAP prensibi farklı senaryolarda rol oynar. Örneğin, çevrimiçi bir alışveriş platformu, kullanıcılara sürekli erişim sağlamak için Availability (Kullanılabilirlik) üzerinde durabilirken, aynı anda tutarlılık garantisini de verebilir.</li></ul><p>Her bir prensibin gerçek hayatta farklı senaryolarda nasıl kullanıldığını anlamak, veritabanı yönetimindeki çeşitliliği ve esnekliği kavramak adına önemlidir. Hangi prensibin hangi senaryoda daha uygun olduğunu belirlemek, projenin ihtiyaçlarına bağlıdır.</p><p>Bu benim ilk yazım ve umarım keyif almışsınızdır! Daha fazla yazı ve keşif için bir sonraki yazılarda görüşmek üzere!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce5d329b8d7e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>