<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Marco Bürckel on Medium]]></title>
        <description><![CDATA[Stories by Marco Bürckel on Medium]]></description>
        <link>https://medium.com/@mbue?source=rss-20611e19fdf7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*R4Jh8kEuEfwbzZLvvpJtGw.png</url>
            <title>Stories by Marco Bürckel on Medium</title>
            <link>https://medium.com/@mbue?source=rss-20611e19fdf7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 18:02:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mbue/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[4 Reasons why SaaS Startups should not consider Microservices for their product development]]></title>
            <link>https://medium.com/@mbue/4-reasons-why-saas-startups-should-not-consider-microservices-for-their-product-development-c3843adace1e?source=rss-20611e19fdf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/c3843adace1e</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Marco Bürckel]]></dc:creator>
            <pubDate>Fri, 20 Aug 2021 08:32:18 GMT</pubDate>
            <atom:updated>2021-08-20T08:32:18.488Z</atom:updated>
            <content:encoded><![CDATA[<p>“As a startup, we use a microservices-based architecture to allow our product to scale.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2aiQBr0SZwHZit6wDpk5Dg.png" /></figure><p>I often hear statements like that. Especially young tech companies that are eager to grow often choose this architectural style. After all, microservices seem to be the preferred choice of many big players and they promise a scalable development process and product.</p><p>Unfortunately, it is not as easy as that:</p><p>First and foremost, microservices are a way of coping with complexity. When projects become large, they allow us to keep things manageable. However, this advantage does not come for free. For example, microservices also introduce a higher system complexity, which initially means a slower development velocity.</p><p>For big companies, this trade-off makes sense: More developers are able to work on their products at the same time. It is easier to set up multiple autonomous teams and there is less friction when it comes to software releases. However, startups usually do not suffer from problems caused by organizational complication or product complexity. Their focus is not internal process optimization but rather a high pace of innovation. As a startup, we want to validate our product ideas on the market as fast as we can. In this context, microservices might soon feel more like a millstone around the neck than an enabler of growth.</p><p>Today, I want to share four considerations that you and your team should discuss before choosing a microservice-based architecture as the foundation of your product development process:</p><ol><li><strong>Wrong service boundaries cause a lot of problems</strong><br>When it comes to microservices, we must be very careful in terms of how we decompose functionality into services. Wrong decisions can greatly influence implementation complexity, result in unstable system behaviors and communication overhead. A good decomposition of services can be derived from the product domain (its “business logic”). By analyzing entities and their relationships in context of product functionalities, we can identify which application parts need to belong together and which parts might reside in different services. However, when it comes to startups, product ideas are often not very clear yet. The product domain might change over time, which makes it difficult to define a proper set of microservices. Is your product domain stable enough to allow a healthy decomposition into services?</li><li><strong>Your product might not really benefit from fine-grained scalability</strong><br>Microservices allow us to scale different parts of our software individually. This can be a great benefit when defining scaling strategies for large software systems, e.g. when they run resource-intensive workflows or background tasks. However, many applications do not really need a sophisticated scaling strategy. In many cases, a simple horizontal scaling mechanism (e.g. running multiple identical instances of a monolith, see <a href="https://paulhammant.com/2011/11/29/cookie-cutter-scaling/">Cookie-Cutter-Scaling</a>) is sufficient. For example, if your backend mostly consists of APIs acting upon incoming requests in a typical request/response lifecycle, it will not make a big difference whether you scale-out all APIs in a single application process (or container) or scale-out individual parts. Looking at your product functionalities and your implementation strategies, would the system really benefit from scaling parts individually?</li><li><strong>Higher operational complexity</strong><br>Microservices form a distributed system — with all the usual consequences. The deployment process, monitoring and troubleshooting tend to become more complex. After the go-live of your product, it means that you will have to invest more resources into operational infrastructure to deal with that complexity. On the other hand, you might only have a few paying customers after the initial go-live. Did you ever make a thorough cost estimation for managing the planned microservice setup in production?</li><li><strong>Higher development complexity</strong><br>The distributed nature of microservices also implies a higher workload throughout development. Your development team has to tackle data consistency issues, a bunch of topics regarding communication infrastructure and asynchronous messaging, as well as a more complicated top-level architecture. On development machines, running, debugging and testing the application becomes more complicated. This might not feel like a big deal, but the time spent for these “everyday” tasks quickly sum up and make a significant difference in the end. The MVP development using microservices might be significantly more expensive compared to a monolithic approach. Therefore, you should ask yourself whether microservices are an option in context of the financial scope of the project.</li></ol><p>Of course, some startups have good reason to choose a microservices-based architecture and above considerations must be carefully evaluated in context of the individual situation. However, I believe that the majority of SaaS startups benefit from a monolithic approach. In most cases, the advantage of a fast-paced MVP development outweighs any advantages offered by microservices. Once there is a large number of customers, gradually shifting towards a microservice architecture is still a possible option.</p><p>In case you have further questions or need advice for your own project, do not hesitate to get in touch with me.</p><p>— Marco</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c3843adace1e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Practical Tips on Software Architecture Design, Part Two]]></title>
            <link>https://medium.com/@mbue/practical-tips-on-software-architecture-design-part-two-37c2b61d7b6d?source=rss-20611e19fdf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/37c2b61d7b6d</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-architect]]></category>
            <dc:creator><![CDATA[Marco Bürckel]]></dc:creator>
            <pubDate>Mon, 17 Feb 2020 17:58:05 GMT</pubDate>
            <atom:updated>2020-02-17T17:58:05.634Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8Ohwr1ttq5NJNrmbx7hnrg.jpeg" /></figure><p>In this series, I present practical tips and ideas on how to design a software architecture from scratch. As mentioned in the previous part, these ideas do not try to form a comprehensive design methodology that can be followed step-by-step. Instead, they aim to be a helpful set of tools along your journey from an empty whiteboard to a reasonable, architectural design for your software.</p><p>These articles are intended to be a short read with a practical focus. Thus, I want to skip rather theoretical questions such as a possible definition of the term software architecture or the difference between software architecture and software design. There are plenty of good articles and papers which already cover these questions¹. For the purpose of this series, we look at any design considerations and decisions that allow your software system to grow healthy in terms of a certain set of quality criteria.</p><p>In case you have not read part one of this series, you can find it here:<br><a href="https://medium.com/@mbue/practical-tips-on-software-architecture-design-part-one-1c6bb0167157">Practical Tips on Software Architecture Design, Part One</a></p><h3>More on Requirements …</h3><h4>#7 Identify significant functional requirements</h4><p>In the previous article, we already talked about the importance of functional and non-functional requirements. While <em>functional requirements</em> specify the behaviors and use cases of the system, the <em>non-functional requirements</em> specify characteristics of a system, such as its quality attributes (performance, portability, scalability, …) or technical constraints.</p><p>As architects, our mission is to find a system design which</p><ol><li>enables and supports the implementation of the given functional requirements</li><li>complies with the given non-functional requirements</li></ol><p>While there are other factors that have an influence on our design decisions, such as organizational constraints or risk considerations, the aforementioned mission is our main game: The requirements will be the drivers of our design considerations. They basically shape the problem that we try to find a solution for.</p><p>In Tip #5, I already mentioned that non-functional requirements have a significant influence on the architectural design of a system. But what about functional requirements?</p><p>In case of functional requirements, usually not all of them are important for our design considerations. However, it is quite obvious that functional aspects of a system do greatly affect its structure, technological building blocks and technical behavior. The architecture of an industrial conveyor control system will most certainly look different compared to a distributed HR management platform, and both look different compared to a game engine.</p><p>In many cases, functional requirements suggest or even demand certain design and technology choices. If our architecture does not take them into account, features become hard or even impossible to implement, or we introduce accidental complexity by providing a suboptimal technical environment for implementation.</p><p>As a trivial example, an application driven by data forms and reports might suggest the usage of a (relational) database. Of course, we could use a different approach and use plain file I/O instead. But as soon as the reports require some sort of aggregated data or sophisticated projection, we would start to implement parts of the data management functionality which a database would provide out of the box. This is a simple example demonstrating <strong>accidental complexity</strong>, a term coined by Fred Brooks in his famous “No Silverbullet” paper. It is the technical complexity that is not relevant to the problem, often resulting from the wrong choice of language and tools.</p><p>Usually, we can identify clusters of functional requirements which are enabled or supported by the same design consideration:</p><blockquote><strong>Example.</strong><br>Consider a CRM application that has the following functional requirements:</blockquote><blockquote>1) Data forms, such as the creation of a new customer, can be visually adapted towards the organization’s corporate design by specifying custom colors, a custom font as well as a custom brand image.<br>2) Obligatory fields in data forms are shaded in a soft color tone to highlight their importance.</blockquote><blockquote>As soon as we found a system design that enables and supports the implementation of requirement 1, it is unlikely that requirement 2 will have a significant impact on our system design considerations. Both requirements demand a rich user interface, and the possibility to visually customize it, based on rules or some kind of configuration.</blockquote><p>During functional requirement analysis, we need to identify the requirements that challenge or question our current design idea. This allows us to alter our solution step by step, to support the implementation of the functional requirements in the best way. That does not mean that we should think ahead the implementation of all functional aspects in an early design phase. Of course, plenty of software design decisions will be made during implementation of individual requirements. However, we need to watch out for requirements that push the limits of what our architectural building blocks and technology choices are able to offer.</p><p>This is usually an iterative approach: Both the requirements as well as your design ideas will change and evolve over time. Nevertheless, at any stage, you should understand the current state of requirements well, as well as their impact on the architectural design.</p><p>Start by looking at individual requirements and tag them with keywords indicating possible solution ideas or design considerations the requirement could benefit from. As stated in tip #1, I recommend to start with abstract concepts and not introduce concrete technology choices too early. In the beginning, you can work with conceptual keywords such as <em>graph database</em>, <em>push communic</em>at<em>ion, report engine, rich UI, realtime OS, </em>gp<em>u accelerat</em>io<em>n, etc.</em></p><p>After tagging individual requirements, we can easily identify requirement clusters that are dependant on the same design or technology consideration. We can pick the most challenging representatives in each cluster and further discuss them with developers and stakeholders. This allows us to iteratively refine possible solution approaches and gives us a good foundation for documenting and justifying certain design decisions.</p><p>Once you feel like you got a fair grasp on the functional requirements and possible solution concepts, you can of course dig deeper into technical details and begin to shape a technology stack. As mentioned before, the whole design process should be seen as an iterative and evolving process, not as a strict waterfall-like design flow.</p><h4>#8 Define Quality Attribute Scenarios</h4><p>As mentioned before, non-functional requirements usually belong to one of the following categories:</p><ul><li>Technical constraints</li><li>Quality attributes, such as availability, extensibility or portability</li></ul><p>When it comes to technical constraints, their meaning and influence on our solution design is usually pretty obvious. They normally require the integration of certain systems, the usage of a specific programming language or the compatibility to a certain environment.</p><p>Much more interesting (and challenging) is the analysis of quality attributes.</p><p>The first challenge is to write them down properly. From my experience, quality attributes are usually the requirements with the most ambiguous formulation. In many projects, they are not even written down at all, but rather mentioned verbally. In case they are written down, you will most certainly find a rather vague enumeration such as:</p><ul><li>The application performance is crucial, because users expect a good experience</li><li>The system must be extensible, because we need to be able to adapt it to new situations</li><li>The application must be reliable, because it is critical to our success</li></ul><p>While the meaning of quality attributes such as Extensibility or Reliability are standardized², they surely leave plenty of room for different interpretations. In tip #6, I already mentioned that it is a good idea to discuss the scope of quality attributes. This can be a first helpful step to better understand the ideas and motivations behind them.</p><p>However, in the next step, it is crucial to enrich each required quality attribute with concrete, measurable scenarios. Otherwise, we will most likely end up discussing vague requirements without ever being able to tell whether they are fulfilled or not.</p><p>A quality attribute scenario basically describes a measurable system response to a certain interaction (the stimulus). It consists of six parts:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SZf6NAPPEQBvdCsf4hdSxA.png" /><figcaption>Elements of a Quality Attribute Scenario</figcaption></figure><ul><li><strong>Source of Stimulus</strong>: An entity (a human being or system) that generates the stimulus.</li><li><strong>Stimulus</strong>: The interaction that occurs.</li><li><strong>Environment</strong>: The condition in which the stimulus occurs. For example, the system may be in an overload condition.</li><li><strong>Artifact</strong>: The receiver of the stimulus. This is either the system, or a part of it.</li><li><strong>Response</strong>: The activity that occurs after the arrival of the stimulus.</li><li><strong>Response measure</strong>: Defines the measurable outcome of the scenario.</li></ul><p>The above template makes it easy to describe concrete expectations for each quality attribute while keeping the description simple and short. Let’s take a look at two example scenarios for the quality attributes performance and extensibility:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s7z6pod67xFd9szAf4f6pg.png" /><figcaption>Quality Attribute Sample Scenarios</figcaption></figure><p>Quality attribute scenarios help us to define some sort of “benchmark” in measuring whether the architecture complies to the non-functional requirements. They are a common agreement between stakeholders and the development and serve as a great foundation for a later QA process.</p><h3>On Component Design …</h3><h4><strong>#10 Design with Segregation in Mind</strong></h4><p>Keeping the building blocks of a system separated from each other is one of the software designers’s most valuable tools. By implementing different system parts as individual class libraries (“modules”), or by keeping them in separated namespaces or packages, we make them manageable. On an implementation level, we gain a clear view on each building block and its interfaces. From an organizational point of view, different people or teams can focus on their respective building block, without being distracted from the implementation details of other system parts.</p><p>However, as long as those building blocks are not designed as separated processes, they will never be truly independent of each other. How so?</p><p>Let’s consider a developer making changes to a certain component of a system. The component is nicely isolated as a dedicated package or class library module, so during implementation, the work can be focused on that particular package. However, as soon as the developer wants to make a first smoke test by running the system, she or he still has to be aware of</p><ul><li>the proper configuration of all the system parts</li><li>fulfilling the dependencies of all the system parts, be it libraries or 3rd party systems like database systems or message brokers</li><li>and the initialization phase of all the system parts during startup</li></ul><p>Otherwise, the system may crash at a random point before our developer’s code is even executed. In smaller systems, this may sound like a trivial problem. However, in any medium-sized or bigger system, maintaining the configuration and dependency needs of different components tend to be a quite complex task.</p><p>The only way to keep system parts completely isolated from each other, is to design them to run on different processes. These days, the most well-known architecture style that utilizes process separation might be the microservice architecture style. When using microservices, we decompose an application into small, loosely-coupled, independently deployable processes.</p><p>These days, I often come across discussions whether a system should be designed as a monolith or as a set of microservices. In my opinion, the question is not that simple, as these are not the only possibilities we have. At any time in the design phase, we should be aware of all the graduations and variations between a single-process monolith and a fine-grained service decomposition, e.g. microservices.</p><p>The possibility of decomposing a system into independent processes should be understood as a general tool, that comes with certain benefits and complexities. When we define the structural building blocks of a system, we can examine at any stage whether it could be beneficial to isolate individual building blocks as separated processes.</p><p>Of course, in comparison to a single-process monolith, we introduce some new challenges such as inter-process communication or additional deployment complexity. However, applying process separation can lead to easier manageable systems, a cleaner software design and a better interchangeability of individual system parts.</p><blockquote><strong>Example.</strong><br>Let’s suppose we want to build a desktop application for editing vector graphics.<br>When designing the architecture for a vector graphics editor, chance is high that one of our system components will be called a “Rasterizer”. The rasterizer turns vector graphics, described by geometric shapes, into a raster image, described by pixels. So basically, it turns a vector graphics file into a bitmap file. The rasterizer might be a good candidate to deploy as a process apart from the graphics editor itself:</blockquote><blockquote>- It processes files, which already are a very natural inter-process communication mechanism.</blockquote><blockquote>- The algorithm is complex and might utilize some very specific dependencies (e.g. libraries for dealing with bitmap I/O), which might not be a concern to the core of our editor.</blockquote><blockquote>- The rasterization algorithm can be quite complex and surely is tailored towards one special task. Process isolation leads to a less cluttered code base and also enables problem-specific implementation choices for the component. For example, we could choose a different programming language to benefit from performance advantages or better libraries.</blockquote><p>In part one, I already mentioned a possible initial design phase of identifying system parts as abstract components (tip #1). After identifying the top-level system parts as well as their rough inner structure, it is a good idea to start thinking about a possible process separation and evaluate where it could be beneficial. As with any architectural considerations, such an evaluation is of course an iterative process throughout the design phase.</p><p>It is also worth noting that such a decomposition of a system can also be done at a later time. If we apply it to an existing system, the amount of work might be higher, as we usually have to refactor individual system parts in order to allow them to run as individual processes. On the other hand, our insights on a mature, existing system will most certainly be better than on any system during some early design phase. From that point of view, a decomposition at a later stage is more likely to feel natural and is based on real experience with the system.</p><h3>Final Words</h3><p>That’s all for the second part. In case you have any questions or feedback regarding the tips above, drop a comment. I’d love to read about your opinion or your own experiences. 😊</p><p>In case you need further advise or personal help with your current project, do not hesitate to get in touch with me. Together with my team at <a href="https://www.axioconcept.com/">axio concept GmbH</a>, I help companies to design and build software architectures and software products.</p><p>❤️ Follow me here on Medium, <a href="https://twitter.com/mbue86">Twitter</a> or <a href="https://www.linkedin.com/in/marco-b%C3%BCrckel-0b2575133/">LinkedIn</a> to get notified about new articles.</p><p>— Marco</p><p>[1] A good overview on different definitions can be found here: <a href="https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=513807">https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=513807</a></p><p>[2] ISO 25010<br>“Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models”<br><a href="https://www.iso.org/standard/35733.html">https://www.iso.org/standard/35733.html</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37c2b61d7b6d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Processing highly connected Data using Azure Cosmos DB and Gremlin ]]></title>
            <link>https://medium.com/@mbue/processing-highly-connected-data-using-azure-cosmos-db-and-gremlin-f8b613d071f7?source=rss-20611e19fdf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/f8b613d071f7</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[cosmosdb]]></category>
            <category><![CDATA[graph-database]]></category>
            <category><![CDATA[azure]]></category>
            <dc:creator><![CDATA[Marco Bürckel]]></dc:creator>
            <pubDate>Sun, 28 Oct 2018 14:20:08 GMT</pubDate>
            <atom:updated>2018-10-28T14:20:08.330Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>In this article, we take a look at processing graph-oriented data using Azure Cosmos DB. A graph allows us to represent connections between entities in a convenient and natural way. In many data processing scenarios, we want to efficiently store and analyze huge amounts of these connections.</p><p>For example, think about social networks like Twitter or Facebook: They basically consist of a vast number of people that are connected to each other in various ways. By analyzing the connections between them, we can infer social relationships, optimize content delivery and make meaningful content suggestions to users.</p><p>There are a lot of other problems that can be solved using graphs: Route optimization in logistics, automated product recommendation or monitoring of network topologies, to name a few.</p><p>Azure Cosmos DB enables us to efficiently store and analyze highly connected data using graph structures. Graphs can be read and manipulated using <strong>Gremlin</strong>, a popular graph traversal language originated from the Apache TinkerPop project. As Cosmos DB supports seamless scaling and multi-region replication of graphs, it is well-suited for large-scale data processing scenarios.</p><p>In this article, we will do the following:</p><ul><li>We will learn more about graph databases and Cosmos DB</li><li>We utilize Gremlin queries to construct a graph and store it inside Cosmos DB</li><li>We take a look at some of Gremlin’s query capabilities</li><li>Finally, I want to discuss the usage of graph databases alongside other database systems. This is a very important aspect when designing larger systems, as you may want to take advantage of graph database capabilities but still use other databases, e.g. a relational database.</li></ul><p>Let’s jump right in. ✨</p><h3>About Vertices, Edges and Graph Databases</h3><p>A graph consists of objects and a set of relationships between them. From an abstract, mathematical point of view, the objects are called <strong>vertices</strong> and the relationships are called <strong>edges</strong>.</p><p>An edge connects a pair of vertices and defines a relationship between them. In an <strong>undirected graph</strong>, the edges do not have a direction. Thus, when we have a pair of vertices named x and y, the edge (x, y) is identical to the edge (y, x).</p><p>In a <strong>directed graph</strong>, edges do have a direction. Below are two graph visualizations as an example:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/644/1*c4uQlCbSaBA_Bg04JIvW_Q.png" /><figcaption>A simple, undirected graph</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/584/1*3poiW9i_lRoi-9I2FXzIzA.png" /><figcaption>A simple, directed graph visualizing a sociogram</figcaption></figure><p>By traversing (or following) the edges of these graphs, we are able to answer the following questions:</p><ul><li>Is there any path from A to F in the undirected graph?<br> What is the shortest path from C to E?</li><li>Which person is most popular (is liked most) in the above sociogram?</li></ul><p>A <strong>graph database</strong> is a database that represents and stores data using graphs, that is to say vertices and edges. The graph can be enriched with arbitrary semantic information, stored as a set of properties. <strong>Properties</strong> are key/value-elements that are either attached to a vertex or an edge. Just like in a document-oriented database, the properties do not follow a predetermined schema.</p><p>Queries against a graph database are expressed in a language which is suitable for traversing graphs. The efficient execution of graph traversals is one big strength of graph databases. By contrast, in a relational database, this is a task that does not scale very well.</p><p>If we were to implement a sociogram describing friendships using an RDBMS, we might come up with a table containing people and a table containing friend-relationships:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*78wEI1XtawPkIHD26iDJIw.png" /><figcaption>The representation of a sociogram within an RDBMS</figcaption></figure><p>Traversing the graph now basically results in recursive JOINs: Do any friends of Alice, living in Springfield, cultivate a friendship with people living in Shelbyville? To answer this question, we have to fetch all relationship records involving Alice, fetch the records of her friends living in Springfield, fetch relationship records involving them, … and so on.</p><p>The query execution time in such scenarios easily gets unpredictable, let alone the complexity of expressing them. Of course we could invest some time to optimize these queries a bit by altering above table structure. However, we had to invest that work due to the fact that relational databases do not offer us any good tools to handle highly connected data. The above scenario is a typical example of <strong>accidental complexity, </strong>a term coined by Fred Brooks in his famous “No Silverbullet” paper. It is the technical complexity that is not relevant to the problem, often resulting from the wrong choice of language and tools.</p><p>By contrast, using Gremlin, fetching Shelbyville-based friends of Springfield-based friends of Alice might look like this:</p><pre>g.V()<br>    .has(&#39;name&#39;, &#39;Alice&#39;).out()<br>    .has(&#39;city&#39;, &#39;Springfield&#39;).out()<br>    .has(&#39;city&#39;, &#39;Shelbyville&#39;)</pre><h3>First Steps with Cosmos DB and Gremlin</h3><p>Microsoft Azure Cosmos DB is a globally distributed, multi-model database platform that supports storage of documents, key-value collections and graphs. It offers elastic scaling for storage and throughput as well as multi-region replication.</p><p>In this section, we will create a Cosmos DB graph database and use the Data Explorer from the Azure Portal to run some basic Gremlin commands.</p><h4>Creating a Graph Database</h4><p>First, start by creating a new Cosmos DB account within the Azure Portal. Do not forget to select the Gremlin API as the instance API:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cJ7ajgfrIRRb-WDKn0fYzg.png" /><figcaption>Creating a new Cosmos DB Account</figcaption></figure><p>Switch to your newly created Cosmos DB account and navigate to the <strong>Data Explorer</strong>. The data explorer gives you an overview of existing databases and graphs. It also can be used to visualize graphs and even manipulate vertices, edges and their properties.</p><p>Add a new graph as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xPIv0JqvodxsWUy7b3Yw_g.png" /><figcaption>Creating a new Cosmos DB Graph Database</figcaption></figure><p>After creating the database, you can switch to the “Keys” section and grab both the URI and primary key values. We will need both properties in the next section to establish a connection via C#.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PSYh83P8WZWS_rVWVC1rDw.png" /><figcaption>Getting the Cosmos DB Read-Write Key needed by the C# SDK</figcaption></figure><h4>First Steps with Gremlin</h4><p>Gremlin is a query language specifically designed for handling graph data structures. We can get familiar with some basic queries using the Data Cosmos DB Data Explorer. Go to the Data Explorer, select your previously created Graph Database and open the associated Graph. You will see an input box that allows you to run Gremlin queries.</p><p>Run the query g.addV(&#39;server&#39;).property(&#39;id&#39;, &#39;API Gateway&#39;) , as shown in the screenshot below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-75W7RAz9fJGde-0spba5g.png" /><figcaption>Running Gremlin queries in the Azure Cosmos DB Data Explorer</figcaption></figure><p>The query will add a new vertex to the graph. The parameter of addV specifies the label of the vertex, which denotes its type. Let’s add some more vertices by executing the following queries:</p><pre>g.addV(&#39;server&#39;).property(&#39;id&#39;, &#39;Storage Service&#39;)<br>g.addV(&#39;client&#39;).property(&#39;id&#39;, &#39;Mobile Device&#39;)<br>g.addV(&#39;client&#39;).property(&#39;id&#39;, &#39;Web App&#39;)</pre><p>By executing g.V() , we can retrieve all the vertices of the graph. The Data Explorer allows us to explore the query result either as JSON or as a graph visualization. We can see the following JSON result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jnDTduGpO7-9Nd7iUYFUDQ.png" /><figcaption>A query result shown by the Data Explorer</figcaption></figure><p>Of course, we can query vertices by label and property values. The following query retrieves all server vertices:</p><pre>g.V().hasLabel(&#39;server&#39;)</pre><p>And the following two queries both retrieve the vertex that has the ID “Web App”:</p><pre>g.V().has(&#39;id&#39;, &#39;Web App&#39;)<br>g.V(&#39;Web App&#39;)</pre><p>By adding some edges, we can make our graph more interesting:</p><pre>g.V(&#39;Web App&#39;).addE(&#39;connects&#39;).to(g.V(&#39;API Gateway&#39;))<br>g.V(&#39;Mobile Device&#39;).addE(&#39;connects&#39;).to(g.V(&#39;API Gateway&#39;))<br>g.V(&#39;API Gateway&#39;).addE(&#39;connects&#39;).to(g.V(&#39;Storage Service&#39;))</pre><p>This will connect both client vertices to the API Gateway and the API Gateway to the Storage Service. By executing g.V(&#39;API Gateway&#39;) , we can examine the result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CMY3FuBCjIj8L0ta4zAKvg.png" /><figcaption>Resulting Graph</figcaption></figure><p>Just like the vertices, edges also have a label, denoting their type. In our case, all edges are labeled “connects”. We can retrieve these edges by executing g.E().hasLabel(&#39;connects&#39;) .</p><p><strong>Exercise 1</strong>:<br>Experiment with the following queries using the Data Explorer and describe the result:</p><pre>A) g.V(&#39;API Gateway&#39;).inE()</pre><pre>B) g.V(&#39;API Gateway&#39;).inE().outV()</pre><pre>C) g.V(&#39;API Gateway&#39;).bothE()</pre><pre>D) g.V(&#39;API Gateway&#39;).inE().hasLabel(&#39;connects&#39;).count()</pre><pre>E) g.V(&#39;Web App&#39;).outE().inV().id()</pre><p>Answer:</p><pre>A) Retrieves incoming edges of the API Gateway</pre><pre>B) Retrieves the source vertices of the incoming edges of the API Gateway</pre><pre>C) Retrieves both incoming and outgoing edges of the API Gateway</pre><pre>D) Counts the incoming ‘connects’-edges of the API Gateway</pre><pre>E) Traverses from the Web App to the target vertices and get their ID</pre><p>After the next section, we will take a look at some more advanced Gremlin queries. But before, we might need some more complex data to play with.</p><h3>Graph Construction using Gremlin.Net</h3><p>Gremlin.Net is an implementation of the Gremlin language based on .NET Standard. It offers both an API for executing Gremlin query strings, as well as a fluent API that allows a typesafe approach towards graph queries. However, at the time of writing, Cosmos DB does not yet support the fluent API due to a missing bytecode stream implementation. There is a related discussion on Github: <a href="https://github.com/Azure/azure-cosmosdb-dotnet/issues/439">https://github.com/Azure/azure-cosmosdb-dotnet/issues/439</a></p><p>There is an open source project, Gremlin.Net.CosmosDb, that aims to provide fluent API queries by serializing them into a Gremlin string representation. You can get more information on the Github project page: <a href="https://github.com/evo-terren/Gremlin.Net.CosmosDb">https://github.com/evo-terren/Gremlin.Net.CosmosDb</a></p><p>For starters, we will send some plain Gremlin query strings to build and upload a graph. In the following example, we will construct a sociogram and upload it to Azure Cosmos DB. The resulting graph will look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pbCf8yMBopPOvY9Jn0-upA.png" /><figcaption>Vertices and Edges visualized by the Cosmos DB Data Explorer</figcaption></figure><p>You can grab the example on Github: <a href="https://github.com/marco-bue/azure-cosmos-gremlin">https://github.com/marco-bue/azure-cosmos-gremlin</a></p><p>Create a new .NET Core 2.0 console application and add the Gremlin.net package as a dependency.</p><p>We start by adding a simple class that allows us to store a Gremlin query alongside a query description:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d145ff95cf15b5ce7bdf1343e902036/href">https://medium.com/media/6d145ff95cf15b5ce7bdf1343e902036/href</a></iframe><p>We introduce two methods to create the necessary Gremlin statements:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a56ee8ca0df34599c98994097caf7aa9/href">https://medium.com/media/a56ee8ca0df34599c98994097caf7aa9/href</a></iframe><p>The following method takes an array of names and returns a list of Gremlin statements constructing the sociogram:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e6b083fa5d3a21d3bcd5661214c49759/href">https://medium.com/media/e6b083fa5d3a21d3bcd5661214c49759/href</a></iframe><p>Now that we got a list of queries that represents our graph, we need a way to run these queries. Thus, we add an async method to execute a list of Gremlin queries on a given Gremlin server connection. The connection is represented by a GremlinServer instance.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/df6f2242111c8ab7c436e075dc98b859/href">https://medium.com/media/df6f2242111c8ab7c436e075dc98b859/href</a></iframe><p>The dynamic returned by SubmitAsync corresponds to the JSON responses that we have already seen in the previous chapter. Thus, SubmitAsync can also be used to read query results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/1*HcugOIdjvHXcTWt7MJAMjw.png" /><figcaption>Observing a Gremlin response while debugging</figcaption></figure><p>Within our Main method, we finally wire everything together:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d397c44a61197b5a79c731cf33dc2d6a/href">https://medium.com/media/d397c44a61197b5a79c731cf33dc2d6a/href</a></iframe><p>Please note that async Main methods require C# 7.1 or above. After adjusting above constants from the keys and credentials of your Cosmos DB account, you can run the program and examine the resulting graph within the Data Explorer. As the edge construction is based on a random permutation of vertex indices, the results may vary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/516/1*Qn3YCN4RY91qJzOJkxnKxQ.png" /><figcaption>A small subset of the resulting Sociogram</figcaption></figure><h3>Gremlin revisited</h3><p>Now that we have created a bigger graph, we can take a look at some more advanced Gremlin queries. Again, we will do so by executing them within the Cosmos DB Data Explorer.</p><h4>Traversing to adjacent Vertices</h4><p>First, let’s see how many people Celia knows:</p><p>g.V(&#39;Celia&#39;).out().count()</p><p>out steps from a source vertex to all adjacent target vertices. Two vertices are called<strong> adjacent</strong> if they are connected by an edge. In the previous Gremlin chapter, we already queried an adjacent vertex by explicitly stepping over the edge, using outE().inV() . Of course we can also query adjacent source vertices:</p><p>g.V(&#39;Celia&#39;).in()</p><p>or all adjacent vertices:</p><p>g.V(&#39;Celia&#39;).both() .</p><p>Let’s traverse one step further and list the names of all the people that are known to all the people known to Celia:</p><p>g.V(&#39;Celia&#39;).out().out().id()</p><p>It is very likely that the result contains some duplicates. If Celia knows two people <strong>a</strong>, <strong>b</strong> and both <strong>a</strong>, <strong>b</strong> know person <strong>c</strong>, then <strong>c</strong> will be listed twice. We can use dedup to de-duplicate them:</p><p>g.V(&#39;Celia&#39;).out().out().dedup().id()</p><h4>Using Paths to query a Traversal</h4><p>Until now, all our queries returned a certain subset of our graph, be it a set of vertices, an edge or the property of a vertex. Using gremlin, we can also retrieve information about the walk that we have taken through a graph.</p><p>A <strong>path</strong> denotes a walk through the graph by returning all objects that were visited during a particular traversal. Let’s take a look at an example:</p><p>g.V(&#39;Celia&#39;).out().path().limit(1)</p><p>This will give us the following result:</p><pre>[<br>  {<br>    &quot;labels&quot;: [<br>      [],<br>      []<br>    ],<br>    &quot;objects&quot;: [<br>      {<br>        &quot;id&quot;: “Celia”,<br>        &quot;label&quot;: “person”,<br>        &quot;type&quot;: “vertex”<br>      },<br>      {<br>        &quot;id&quot;: “Benjamin”,<br>        &quot;label&quot;: “person”,<br>        &quot;type&quot;: “vertex”<br>      }<br>    ]<br>  }<br>]</pre><p>The path object consists of two properties:</p><ul><li>A list of labels of the steps we have made. The first step was the retrieval of Celia using g.V(&#39;Celia&#39;) and as a second step, we moved to an adjacent vertex using out() . Until now, we did not assign any labels to the steps, hence the list of empty labels.</li><li>A list of objects that has been visited on our traversal.</li></ul><p>Of course, using g.V(&#39;Celia&#39;).out() , we did not only traverse to Benjamin but to all other people Celia is connected to by an outgoing edge. We can see all these other paths when removing the limit(1) step from our query.</p><p><strong>Exercise 2</strong>:<br>Experiment with the following queries using the Data Explorer and describe the result:</p><pre>A) g.V(&#39;Benjamin&#39;).as(&#39;start&#39;).out().out().as(&#39;move&#39;).path()<br>    .limit(1)</pre><pre>B) g.V(&#39;Ron&#39;).in().outE().path()</pre><p>Answer:</p><pre>A) This time, the path consists of three steps. The path object contains the labels &#39;start&#39; and &#39;move&#39;, as we have assigned these labels to the first and third step of our traversal.</pre><pre>Using as(x), we can assign the label x to the previous step. </pre><pre>B) The path object contains a vertex as well as an edge.</pre><h4>Walk the Loop</h4><p>So far, we explicitly expressed every single step within a traversal. Using repeat , we can enrich queries with various loops. For example, we already executed the following query:</p><pre>g.V(&#39;Ron&#39;).out().out().id()</pre><p>We get the same result by repeating the out statement using a loop:</p><pre>g.V(&#39;Ron&#39;).repeat(out()).times(2).id()</pre><p>Basically, there are two kind of loops we can utilize:</p><ul><li>The <em>repeat…times</em> loop executes a sequence of steps a given number of times. The repeated steps are specified within the parameter of the repeat statement.</li><li>The <em>repeat…until</em> loop executes a sequence of steps until a certain condition has been met. The condition is specified within the parameter of the until statement.</li></ul><p>Let’s take a look at another example. Before we express some query, we assign a unique property to one of Celia’s friend-friends:</p><pre>g.V(&#39;Celia&#39;).out().out().limit(1).property(&#39;city&#39;, &#39;Speyer&#39;)</pre><p><em>(Speyer is one of Germany’s oldest cities and is definitely worth a visit! :) )</em><br>The next query searches a path from the Celia vertex to another vertex that has the property “city = Speyer”. The query above ensures that we find a match:</p><pre>g.V(&#39;Celia&#39;).repeat(out()).until(has(&#39;city&#39;, &#39;Speyer&#39;))<br> .path().limit(1)</pre><p>Of course, there are plenty of paths that lie between both vertices. By increasing the limit parameter, you can take a look at them. The first path will be the shortest.</p><p>This section has just shown some very basic query capabilities of Gremlin. By visiting the official Tinkerpop page, you can learn about more advanced query scenarios: <a href="http://tinkerpop.apache.org/docs/current/recipes/">http://tinkerpop.apache.org/docs/current/recipes/</a>.</p><h3>Using Graph Databases alongside other Databases</h3><p>In larger systems, we usually face a variety of different data storage problems. Some components might need good transaction support, while other components might need to efficiently store and query graphs, unstructured documents or blobs. In this case, we may want to take advantage of Polyglot Persistence.</p><p>Basically, <strong>Polyglot Persistence</strong> is the notion that a system can utilize multiple data storage technologies to solve different problems. It allows us to use the right tool for the the right use case. In the context of this article, it means that we can use a graph database like Azure Cosmos DB without using it for all our persistence concerns.</p><p>Of course, Polyglot Persistence comes at a price: Both the system architecture and the technology stack get more complex. Developers need to integrate multiple data access SDKs, there are multiple data models (which might also need to synchronize) and of course, there are multiple database systems that need to be deployed and managed. Still, the cost of accidental complexity, resulting from pushing everything into one database, might be significantly higher. At last, integrating a new database system is an architectural decision that must always be considered carefully.</p><p>Let’s assume we have decided to use a graph database alongside a relational database.</p><p>In a simple scenario, the graph data model and the relational data model are independent of each other. Each database manages write and read operations of its own set of entities. The only references between both data models are references to entity ids. The application, be it a monolith or a set of services, talks to one database depending on the use case:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q5pa2yU4z1gRaqsbssYjyQ.png" /><figcaption>A simple system that uses multiple databases</figcaption></figure><p>In a more complex scenario, both database models may contain the same entities and offer different representations in different use cases.</p><p>For example, think about a large enterprise organization model involving departments, groups and people, enriched by a complex set of roles and permissions at different levels. In some cases, you may want to filter and aggregate certain statistics based on all employees or groups. A relational database is well-suited for this task. In another use case, you may want to query certain rights of a single person, that belongs to certain groups, projects and a hierarchy of departments. To get this information, we basically need to traverse the organizational graph. In this case, a graph database model would be desirable.</p><p>In this case, we want to represent the enterprise organization both as a relational and graph-based data model.</p><p>Which software design could efficiently enable the above scenario? If both data models contain the same entities, they need to be in sync. However, applying every write statement to both databases is not an option that scales very well: To prevent inconsistencies, we would need to utilize heavyweight distributed transactions.</p><p>Instead, it is a good idea to declare one database (e.g. the relational db) as the <strong>source of truth</strong>: All write operations are exclusively applied to that database. The other database (e.g. the graph db) is used as a <strong>read store</strong>: The application logic may read from that store, but it never modifies its state. The read store is exclusively modified by a dedicated synchronization process, which runs asynchronously. The asynchronous update of the read store comes with some advantages: If the system is under heavy load, we can defer the update. If the update process fails, it can be retried.</p><p>By using the above asynchronous data synchronization, we can increase the throughput of write operations in our system. However, we also ease the consistency between both data models: The synchronization of the read store will be delayed by a finite amount of time. That architectural tradeoff between throughput and consistency has to be carefully aligned with the non-functional requirements of our system.</p><p>Implementing a read store synchronization would go far beyond the scope of this article and there are plenty of options. For example, the <strong>Event Sourcing</strong> pattern enables an efficient implementation that scales very well. However, that exercise will be left to another article.</p><p>That’s all for now. If you have any questions or feedback regarding Cosmos DB or graph databases in general, I would be happy to read your comments or messages.</p><p>— Marco</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f8b613d071f7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Some thoughts on using CQRS without Event Sourcing]]></title>
            <link>https://medium.com/@mbue/some-thoughts-on-using-cqrs-without-event-sourcing-938b878166a2?source=rss-20611e19fdf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/938b878166a2</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[cqrs]]></category>
            <category><![CDATA[event-sourcing]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Marco Bürckel]]></dc:creator>
            <pubDate>Wed, 22 Aug 2018 07:19:03 GMT</pubDate>
            <atom:updated>2018-08-24T06:57:55.064Z</atom:updated>
            <content:encoded><![CDATA[<p>In the last couple of years, I had the opportunity to design one or the other CQRS-based system together with my clients. In these projects, I regularly came across one question: How to build a CQRS-based architecture without adapting the Event Sourcing pattern? Especially when a developer team is new to both CQRS and Event Sourcing, trying to adapt these patterns can be quite challenging.</p><p>Both CQRS and Event Sourcing are powerful building blocks in architectural design, but they also add complexity and might not fit every situation. Thus, if you want to build a CQRS-based architecture, it is beneficial to know the alternatives to a persistence based on Event Sourcing.</p><p>Some blog articles and posts on Stackoverflow refer to CQRS and Event Sourcing as “orthogonal concepts”, which can be applied independently of each other. However, most articles and examples present both concepts baked into one event-based architecture, without discussing any approach to decouple them. Decoupling them is exactly what this article is about.</p><p>Do you want to use CQRS, but hesitate about the implications of Event Sourcing? Do you wonder how omitting Event Sourcing might affect a CQRS-based architecture? If the answer to this question is “yes”, this article might be interesting for you.</p><h3>The Essence of CQRS</h3><p>Looking at the core concept of the pattern, CQRS states that an application architecture can be divided into two parts:</p><ul><li><em>The write-side</em>: A part that modifies the application state by executing commands.</li><li><em>The read-side</em>: A part that reads the application state by executing queries.</li></ul><p>Both concerns are implemented independently of each other. How far the segregation is taken, depends on the concrete architectural design. One may apply it to the logical layers of a system and let both sides share the same database. But often, the segregation is extended to the database level. In that case, write operations and read operations work on different databases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/476/1*ibqX91nn3eF3ceUcv3XQWg.png" /><figcaption>Write Operations and Read Operations affecting different Databases</figcaption></figure><p>Just like segregating the application in a write-side and a read-side, the persistence layer is then segregated in a <em>write-store</em> and a <em>read-store</em>:</p><ul><li>The <em>write-store</em> is the database that acts as the source of truth and is affected by executing commands. It either contains the current application state or contains all information needed for rebuilding it. Thus, the write store is optimized towards data integrity and maintaining a correct application state. Using the typical RDBMS way of doing things, this could be achieved by data normalization, integrity checks and transactions. Another way of keeping the application state clean is to make modifying it dead simple. This is the approach the Event Sourcing pattern takes: The state is represented as an append-only sequence of events. Existing data is never modified or removed. This makes it hard to mess things up.</li><li>The <em>read-store</em> is the database used by queries and is tailored towards the needs of the data consumer. This could include denormalized, aggregated or hierarchical data. The same data could also be stored in a redundant way to serve different representation needs.</li></ul><p>Of course, the data stores need to synchronize. Each modification of the write-store should be reflected within the read-store. This step is called the <strong>projection</strong> of data. The projection can be implemented in various ways and greatly affects the application architecture. Especially, the choice over projection is closely related to the decision on using Event Sourcing or not. Thus, the next sections will take a closer look at different implementation strategies.</p><h3><strong>Event-based Projection</strong></h3><p>Most CQRS implementations utilize event streams as a trigger for data projection. This is an elegant approach, as events naturally describe application state changes. And most (distributed) systems somehow already process them. Processing an application state change usually involves the following steps:</p><ul><li>A command is instantiated, which represents the action that leads to a certain application state change.</li><li>The command is dispatched to a command handler which validates the command and triggers some domain logic to run.</li><li>The domain logic modifies the application state and emits one or more immutable event objects describing the state change that just happened.</li><li>The events are dispatched to appropriate event handlers which update the read database for each event. Events are usually processed asynchronously, in an eventual consistent manner.</li></ul><p>Event-based projection does not necessarily imply an event sourced persistence approach, but there are quite some facts which make it a rather obvious choice:</p><ul><li><strong>Event-based projection requires persisted events.</strong><br>Projections will not always go well. There might be some database connectivity issue or a bug inside an event handler implementation. Eventually, some projections will fail. Thus, the system needs a way to replay event projection. This requires a persistent event store.</li><li><strong>Having a write-database other than the event store requires a (distributed) transaction.<br></strong>What if we persist events inside an event store, but additionally store the current application state within a database? For example, could we put the current state of all entities in a relational database while storing the history of events within an additional event store? We could easily load entity states without touching any events and would still be able to replay projections whenever we need. <br>The problem is, that both write operations have to be handled as an atomic operation: To ensure data integrity, writing to the database and the event store must either both succeed or both fail. Unless both write stores use the same database technology and reside in the same database, a distributed transaction is needed to guarantee data integrity. While there are solutions to this problem, handling write operations significantly gets more complex. In contrast, when using Event Sourcing, this problem does not exist at all.</li><li><strong>Multiple write-stores are harder to manage.</strong><br>Using an additional write store next to an event store also increases database management complexity. As the source of truth is formed by both databases, they always have to be synchronized. In this case, this means that both databases must reflect the state where the exact same sequence of commands has been written to each respective database. This gets interesting when defining backup/restore strategies or when it comes to manual data correction.</li></ul><p>As you can see, when using event-based projection, it makes a lot of sense to make the persisted sequence of events the only source of truth. This is exactly what Event Sourcing does. It is the choice of event-based projection that make CQRS and Event Sourcing such a great match. And in such a system, CQRS and Event Sourcing can hardly be called “orthogonal concepts”, because without Event Sourcing, implementing a reliable event-based projection will become significantly harder.</p><p>So in case you want to use CQRS without Event Sourcing, what are the alternatives? <br>From a more abstract point of view, event-based projection is a projection strategy which is based on small deltas. As any projection needs to be replayable, any such strategy requires persisting these deltas as a source of truth. Thus, an alternative projection strategy would not be based on information about deltas at all, but rather on the current application state.</p><h3>State-based Projection</h3><p>I came across the term of “state-based projection” in a blog article written by Vladik Khononov [<a href="https://vladikk.com/2017/03/20/tackling-complexity-in-cqrs/">Tackling Complexity in CQRS</a>] and I think it perfectly describes the alternative type of projection strategies that exist besides a delta-based approach:</p><p>Instead of projecting small deltas, a state-based projection works on entities as a whole. There are different implementation strategies:</p><h4><strong>Database Views</strong></h4><p>When working with a relational database, the read-side of the CQRS-based system can operate on database views. The projection is then entirely handled by the database itself. Given SQL as a query language, this approach offers a good deal of flexibility regarding data aggregation and transformation. Using a Micro-ORM framework like Dapper, it is easy to implement a thin and lightweight read-side while taking full advantage of the CQRS pattern.<br>However, using database views, the denormalization is limited when it comes to hierarchical data structures.</p><h4><strong>Entity-based Projection Handlers</strong></h4><p>In event-based projection, the projection logic is implemented using a set of event handlers. In a similar way, a state-based projection logic can be implemented using entity-based projection handlers:</p><pre>interface IEntityProjection&lt;TEntity&gt;<br>{<br>    void Project(TEntity entity);<br>}</pre><p>Whenever an entity of type T is changed on the write-side, the projection handlers for type T need to be triggered. In larger systems, this can be done asynchronously in an eventual consistent manner. As Vladik suggests in his aforementioned article, the projection could be triggered by setting either a row-based dirty flag or a row version number that is observed by the projection engine.</p><p>Implementing state-based projection does not require events or event handlers. Thus, this approach works well if you want to build a CQRS-based system which does not utilize Event Sourcing. Of course, that does not necessarily mean that there are no events or event handlers in your system design. For example, you still might want to utilize them in scenarios related to communication between system components.</p><h3>Final Words</h3><p>In many CQRS-related articles, an event-based projection strategy is assumed without really naming implications or alternatives. However, in my opinion, choosing the right projection strategy is the most important question when it comes to CQRS-based system design. I hope this article helped you to gain some more generic idea about the pattern.</p><p>If you have any questions or feedback, just leave me a comment. 😊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=938b878166a2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Practical Tips on Software Architecture Design, Part One]]></title>
            <link>https://medium.com/@mbue/practical-tips-on-software-architecture-design-part-one-1c6bb0167157?source=rss-20611e19fdf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c6bb0167157</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[software-architecture]]></category>
            <dc:creator><![CDATA[Marco Bürckel]]></dc:creator>
            <pubDate>Sun, 29 Jul 2018 09:11:32 GMT</pubDate>
            <atom:updated>2020-04-13T06:17:19.302Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r94bmUrOkJp20RPERmDCSg.jpeg" /></figure><p>Let’s suppose you have just joined a developer team which is about to build an awesome new product. As the system is going to be fairly complex, you have decided to take some time upfront to think about the right, architectural design.</p><p>So you stand infront of an empty whiteboard. Quite some blank space to fill! Where do you even start? What is the best way to define a software architecture from scratch?</p><p>The truth is that it is hard or even impossible to define a strict procedure when it comes to constructing a software architecture. Just like it is impossible to give a step by step guide on how to win every argument. Or on writing a great book people care about.</p><p>However, there are quite some practical tools that can help you to get along. In this article, I want to present some practical tips and good practices on designing a software architecture. The tips are less formal and comprehensive than existing methodologies like ATAM. Instead, they rather aim on quick results. Further, they are highly opinionated and are based on my experience from several large consulting and engineering projects.</p><p>Do you have some favorite design methods which should be on the list? Leave me a comment, I’d love to read about them.</p><p>Alright, let’s get started with part one.</p><p><strong>#1 Start with thinking about abstract components, not deployment diagrams</strong><br>When it comes to software architecture, many developers immediately start to think in technical building blocks. Databases, VMs, Web Servers, Message Brokers, Cloud Computing Platforms and such. While it is important to think about a system’s deployment strategy, it highly depends on a lot of structural and behavioral decisions which are yet to be made. Besides that, 3rd party components add a lot of overall complexity to the software system. Don’t distract your focus by thinking about them too early.</p><p>At the beginning, it is better to use abstract components and abstract concepts as architectural building blocks, much like UML suggests with its component diagram. For example, a system might need a payment processor. You might decide that it should work asynchronously. You might decide that it needs some kind of persistent state. Characterizing these aspects in a rather abstract way is a good starting point to later think about concrete solutions. These solutions will highly depend on functional and non-functional requirements as well as organizational and technical constraints which are yet to be discovered. <br>A design discussion beginning like “Let’s take a MongoDB database and an Azure app service. I think C# might be cool choice.” does not pay enough attention to these things.</p><p><strong>#2 Don’t start by choosing patterns.</strong><br>Patterns are a great tool when it comes to structural component design. But using them as a starting point often leads to over-engineered systems or hype-driven-development. MVC, Pipes and Filters, DDD implementation patterns, CQRS, Ports and Adapters, Event Sourcing, … all of them can be highly valuable building blocks when creating the design of a certain component. But neither of them should be prematurely considered in being a top level approach. Before you utilize a certain approach or method, try to get an overall view on the top level components. The big picture, if you will. Every approach comes with its own advantages and disadvantages. Changes are that one single design approach will not fit every part of your software system.</p><p><strong>#3 Don’t expect too much from the first design iteration.</strong><br>Software architecture and project plans have one thing in common: The first shot is always wrong. The beginning of the project is where you have the least knowledge about the technical and non-technical challenges you are going to face throughout development. Defining a final architecture (or project plan) in that development phase is a rather bold venture. Thus, work in iterations. Allow the architectural design to grow with the knowledge you gather about the system.</p><p><strong>#4 Create a top level view on functional requirements<br></strong>Functional requirements define the functions a system must provide. Ideally, the product owner and other domain experts capture them in a set of user stories, which deliver detailed information about actors, preconditions, possible flows of a certain feature, and so on. However, in an early design phase, you do not have to know every business rule, user story and every aspect of the system’s domain. Instead, it is a good idea to start by getting a top level view on functional requirements. A good starting point is to create a mind map containing the most important nouns of the application domain. Cluster them by functional topics and find the most important actions/verbs around these nouns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/540/1*0EYARY497qu6_wVVaO42-g.png" /></figure><p>The mind map gives you a first idea about functional parts of the system and its complexity. This can also help in getting a feeling about the suitability of possible design approaches. For example, if you got a complex domain which dominates the overall complexity of the software, a domain driven design approach could be beneficial. However, if domain logic narrows down to some trivial aggregations and mappings, DDD and its typical implementation patterns might end up resulting in an over-engineered design phase combined with lots of shallow wrapper types and poor abstractions.</p><p><strong>#5 Identify non-functional requirements carefully</strong><br>Non-functional requirements naturally have major influence on architectural design. They describe the quality attributes of a system. Some examples are:</p><ul><li><strong>Extensibility</strong><br>The effort that is needed to extend a certain feature.</li><li><strong>Performance</strong><br>Short response time / high throughput.</li><li><strong>Scalability</strong><br>The ability of a system to handle an increasing amount of work.</li><li><strong>Portability</strong><br>The usability of a system in different environments.</li><li><strong>Compliance</strong><br>The conformability of a system to policies and standards.</li></ul><p>Looking at such a list of quality attributes, one could say: Fine, I want all of them! However, it is a good idea to choose the set of non-functional requirements carefully. First, each of them can significantly increase design and implementation complexity. Besides that, non functional requirements may conflict with each other. Let us take a look at some examples:</p><ul><li>A very performant system may be less portable because it might demand certain environment/hardware features.</li><li>A highly extensible system may be less performant because the extensibility introduces certain abstraction layers, lowering thoughput.</li><li>The CAP theorem states a proven, fundamental tradeoff between consistency, availability and partition tolerance in distributed shared-data systems.</li></ul><p>Such trade-off situations force us to carefully identify the important quality attributes, which must be satisfied by the software architecture. It is a good idea to document and discuss them with any stakeholders.</p><p><strong>#6 Watch the scope of non functional requirements</strong><br>Not every non functional requirement should be considered as a top level quality attribute of the system. Some quality attributes can be limited to a local scope. This is nice, because the implementation complexity resulting from the non-functional requirement can then also be limited to that local scope.</p><p>At some point, you are going to discuss possible non-functional requirements with product experts and other stakeholders. When you hear statements like “our system must have attribute X because of reason Y”, carefully listen whether reason Y applies to the system as a whole or to a limited part of it. In many cases, reason Y only applies to a very specific situation or use case.</p><p>For example, one stakeholder might say:<br>“Our system must allow a very high throughput because there is an unbelievable high number of data points that need to be aggregated for report generation.”<br>Instead of declaring “Performance” as a top level architecture goal, limit the scope of the proposed quality attribute. The above quality attribute proposal translates to “Our system has a component aggregating certain data. That component must have a very high throughput.”</p><p>In solution space, this may allow us to isolate the aggregating component as some kind of asynchronous worker. Any extra implementation or integration effort that supports high throughput (e.g. special data storage or patterns like CQRS or Event Sourcing) can be limited to that worker.</p><p>When discussing non-functional requirements, it is a good idea to bring some top level visualization of the system’s functional requirements (tipp #4). Ask the stakeholders to think about possible quality attributes for each of the functional clusters you identified there. This helps to encourage a scope-aware discussion of non-functional requirements without talking about technical components.</p><p>That’s all for now. In the next part, we will take a more in-depth look at component design and architectural documentation challenges. If you have any questions or feedback regarding the tipps above, just leave me a comment, I’d love to read your opinion. 😊</p><p>— Marco</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c6bb0167157" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[.NET Microservices entkoppeln mit MassTransit und RabbitMQ]]></title>
            <link>https://medium.com/@mbue/mehr-stabilit%C3%A4t-und-performance-net-microservices-entkoppeln-mit-masstransit-und-rabbitmq-18ea58e4722d?source=rss-20611e19fdf7------2</link>
            <guid isPermaLink="false">https://medium.com/p/18ea58e4722d</guid>
            <category><![CDATA[csharp]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Marco Bürckel]]></dc:creator>
            <pubDate>Mon, 26 Mar 2018 16:08:27 GMT</pubDate>
            <atom:updated>2018-08-29T07:26:37.554Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zkKlsoXgwGOTJvVFzDZjBg.jpeg" /></figure><h3>Motivation</h3><p>In Microservice-basierten Architekturen möchte man einzelne Dienste nach Möglichkeit isoliert und unabhängig voneinander gestalten. Bei der Implementierung von Anwendungsfällen kommt man dann jedoch relativ schnell an den Punkt, an dem zwei oder mehr Dienste miteinander kollaborieren müssen, um eine Anfrage zu verarbeiten. Mögliche Szenarien sind dabei beispielsweise:</p><ul><li><strong>Ein Service benötigt die Daten, welche von einem anderen Service bereitgestellt werden.</strong> <br>Dies könnte für eine Validierung der Anfrage notwendig sein, oder weil die Response eine Aggregation der Daten beider Dienste beinhaltet.<br><strong>Beispiel</strong>: Ein Dienst zur Verwaltung von Bestellungen (“Order-Service”) liefert über REST die Bestellhistorie eines Kunden aus, soll jedoch jede Bestellposition mit produktspezifischen Informationen anreichern, welche von einem zweiten Service (“Product-Service”) verwaltet werden.</li><li><strong>Ein Service triggert eine Aktion, welche durch einen anderen Service implementiert wird.</strong><br><strong>Beispiel</strong>: Der Order-Service möchte eine Bestellbestätigung an die Kunden-E-Mail-Adresse anstoßen. Für den Nachrichtenversand ist jedoch ein eigens für Benachrichtigungen entwickelter Dienst (“Notification-Service”) zuständig.</li></ul><p>Wie sprechen die Dienste im Zuge einer solchen Kollaboration nun miteinander?</p><p>In beiden Fällen sollte während der Request-Verarbeitung keine synchrone Kommunikation zwischen den Diensten stattfinden. Verdeutlicht am obigen Beispiel bedeutet dies, dass der Order-Service während der Anfrageverarbeitung weder auf Daten des Product-Services, noch auf eine Antwort des Notification-Services warten sollte. Andernfalls wäre der Erfolg einer Anfrage an die Verfügbarkeit aller beteiligten Services gekoppelt und die Ausfalltoleranz und Performance des gesamten Systems würde leiden.</p><h3>Asynchrone Kommunikation mit Commands und Events</h3><p>Damit keine derartige Kopplung zwischen Diensten entsteht macht es Sinn, die Kollaboration stattdessen auf Basis asynchroner Nachrichten zu gestalten:</p><ul><li><strong>Events</strong>: Statt Produktdaten in letzter Sekunde beim Product-Service anzufragen, könnte der Order-Service diese aus einer eigenen Datenbank auslesen. Diese Datenbank fungiert dabei als eine Art Cache oder Lesespeicher: Sie enthält Kopien der Daten aus der Produktdatenbank des Product-Service. Durch asynchrone Ereignisnachrichten können wir den Lesespeicher aktuell halten: Bei jeder Modifikation der Produktdaten versendet der Product-Service ein passendes Ereignis im System. Der Order-Service kann alle Änderungsereignisse zu Produktdaten abonnieren und die eigene Datenbank entsprechend nachziehen. Dabei können die Daten optional auch in einer anderen Struktur gespeichert werden, welche auf die Bedürfnisse des Order-Service zugeschnitten sind.</li><li><strong>Commands</strong>: Beim Versand der Bestellbestätigung kann die Beauftragung des Notification-Services ebenfalls asynchron erfolgen. Hierzu erhält der Notification-Service eine Warteschlange, in die Benachrichtigungsaufträge abgelegt werden können. Der Notification-Service arbeitet diese schrittweise ab. Sobald eine Bestellbestätigung versendet werden soll, wird der Order-Service einen entsprechenden Auftrag in die Warteschlage des Notification-Services stellen und dann direkt mit der Verarbeitung der Bestellung fortfahren. Da der Order-Service nicht direkt auf Rückmeldung des Notification Service wartet, und somit keine dringliche Antwort über Erfolg oder Misserfolg des Versands notwendig ist, wird auch die Flexibilität im Fehler-Handling erhöht: Falls eine Nachricht nicht versendet werden kann, könnte nach einiger Zeit beispielsweise ein Retry erfolgen.</li></ul><p>In beiden Szenarien wird die Verfügbarkeit des Order-Services durch Entkopplung verbessert. Nicht verschweigen sollte man dabei, dass dies zu Lasten der Konsistenz erreicht wird: Im ersten Beispiel findet die Datensynchronisation auf Basis asynchroner Events verzögert statt, im zweiten Beispiel bleibt eine verteilte Operation für eine gewisse Zeit unvollendet— ein typisches Beispiel für den unvermeidbaren Balanceakt zwischen Konsistenz und Verfügbarkeit in einem verteilten System.</p><p>Im Folgenden möchte ich zeigen, wie derartige Kollaborationsszenarien auf Basis von RabbitMQ und MassTransit unkompliziert implementiert werden können.</p><h3>Der Message Broker als Kommunikationszentrale</h3><p>Um asynchrone Kommunikationsszenarien zu realisieren, lohnt sich der Einsatz einer nachrichtenorientierten Middleware, welche grundlegende Aufgaben wie das Verteilen und Queueing von Nachrichten auf Basis logischer Kanäle bereits umsetzt. Ein zentraler Prozess, welcher Nachrichten zwischen verschiedenen Systemkomponenten vermittelt, wird im Allgemeinen als <strong>Message Broker</strong> bezeichnet.</p><p>Wichtig ist dabei in Bezug auf Microservices, dass die zentrale Nachrichtenverteilung möglichst leichtgewichtig sein sollte. In Microservice-basierten Architekturen gilt die Empfehlung “Smart Endpoints, dumb pipes” — der Kommunikationskanal selbst sollte sich demnach ausschließlich auf die Weitergabe von Nachrichten konzentrieren. Auf eine zentrale Aufbereitung, Filterung oder anderweitige Datenlogik sollte verzichtet werden. Andernfalls entstehen leicht Abhängigkeiten zwischen Systemkomponenten und Bottlenecks in der Verarbeitung.</p><p>Es gilt also, einen möglichst leichtgewichtigen Message Broker verwenden. Ein ausgereifter, erprobter und auch leichtgewichtiger Message Broker ist <strong>RabbitMQ</strong>. Die Open-Source Software implementiert verschiedene Kommunikationsprotokolle wie AMQP oder MQTT und ist für unterschiedliche Plattformen und Cloud-Umgebungen verfügbar. Client-Bibliotheken werden für eine Vielzahl unterschiedlicher Programmiersprachen angeboten.</p><p><strong>MassTransit</strong> ist ein .NET-basierter Nachrichtenbus, welcher eine einfach zu nutzende Abstraktion über verschiedenen Message Brokern wie RabbitMQ darstellt. Dabei kümmert sich MassTransit um Verbindungs- und Fehlermanagement, Serialisierung und die Verwaltung logischer Nachrichtenempfänger. Mit Sagas bietet MassTransit außerdem ein robustes Rahmenwerk für die Implementierung zustandsbehafteter, asynchroner Nachrichtenempfänger an, wie man sie für die Umsetzung von Workflows oder leichtgewichtige, verteilte Transaktionen benötigt. Ferner ist MassTransit kompatibel zu .NET Core.</p><p>Mit der Kombination aus RabbitMQ und MassTransit lassen sich asynchrone Kommunikationsszenarien in verteilten Architekturen unkompliziert und robust umsetzen. Der nächste Abschnitt geht auf einige Implementierungsdetails ein.</p><h3>Integration von MassTransit in ASP.NET Core</h3><p>Um MassTransit auf Basis von RabbitMQ verwenden zu können, muss letzteres für das jeweilige Zielsystem zunächst installiert werden. Dies ist glücklicherweise ein unkompliziertes Unterfangen, entsprechende Installer und Anleitungen findet man unter <a href="http://www.rabbitmq.com/download.html">http://www.rabbitmq.com/download.html</a></p><p>Die grundlegende Integration von MassTransit in einen ASP.NET Core Service umfasst nun folgende Schritte:</p><ul><li>Initialisierung und Definition der logischen Empfangskanäle (<strong>Queues</strong>) und Registrierung im IoC Container</li><li>Implementierung von Empfängerklassen (<strong>Consumer</strong>) von Nachrichten. Diese verarbeiten einen oder mehrere Nachrichtentypen und werden von MassTransit beim Empfang einer passenden Nachricht instanziiert.</li><li>Implementierung des Versands von Commands und Events</li></ul><h4>Schritt 1: Initialisierung und IoC</h4><p>Zur IoC-Integration von MassTransit gehört zum einen, dass eine konfigurierte Bus-Instanz in andere Klassen injiziert werden kann. Zum anderen soll es aber auch den Consumer-Klassen, welche von MassTransit beim Empfang einer entsprechenden Nachricht instanziiert werden, das Auflösen ihrer eigenen Abhängigkeiten ermöglicht werden.</p><p>MassTransit bietet Erweiterungen für verschiedene IoC-Container an. So kann MassTransit beispielsweise Assemblies nach Consumer-Klassendefinitionen durchsuchen und diese automatisch im jeweiligen Container registrieren. Im Folgenden betrachten wir die Integration am Beispiel von <strong>Autofac</strong>. Autofac ist ein erprobter und feature-starker IoC Container, welche dank der Erweiterung Autofac.Extensions.DependencyInjection sehr leicht in ASP.NET Core integriert werden kann. In Summe benötigt man folgende Nuget-Pakete:</p><pre>MassTransit<br>MassTransit.RabbitMQ<br>MassTransit.Autofac<br>Autofac<br>Autofac.Extensions.DependencyInjection</pre><p>Innerhalb der Startup-Klasse gestaltet sich die Registrierung des Nachrichtenbus-Singletons dann recht unkompliziert. In ConfigureServices wird der Autofac Container instanziiert und als IServiceProvideran ASP.NET Core zurückgegeben. Der Nachrichtenbus wird im IoC Container auf zweierlei Weise registriert:</p><ul><li>Dank der Erweiterung MassTransit.Autofac können Consumer-Klassen automatisch gesucht und im Container registriert werden. Dies geschieht mit der Extension-Methode RegisterConsumers(Assembly assembly)</li><li>Die Methode ConfigureBusübernimmt die Registrierung einer MassTransit-Singleton Instanz.</li></ul><p>Es ergibt sich folgender Quellcode:</p><pre>public IServiceProvider ConfigureServices(IServiceCollection services)<br>{<br>   services.AddMvc();<br> <br>   var builder = new ContainerBuilder();<br>   builder.RegisterConsumers(Assembly.GetExecutingAssembly());<br>   ConfigureBus(builder);<br>   builder.Populate(services);<br>   var container = builder.Build();<br> <br>   return new AutofacServiceProvider(container);<br> }</pre><pre>private void ConfigureBus(ContainerBuilder builder)<br>{<br>  builder.Register(context =&gt;<br>  {<br>    return Bus.Factory.CreateUsingRabbitMq(config =&gt;<br>    {<br>      var host = config.Host(new Uri(&quot;rabbitmq://localhost&quot;), h =&gt;<br>      {<br>        h.Username(&quot;guest&quot;);<br>        h.Password(&quot;guest&quot;);<br>      });<br> <br>      config.ReceiveEndpoint(host, &quot;my_queue&quot;, ep =&gt; <br>      {<br>        ep.LoadFrom(context);<br>      });<br>    });<br>  }).As&lt;IBus, IBusControl&gt;().SingleInstance();<br>}</pre><p>Die Methode ConfigureBus nimmt zwei wesentliche Konfigurationen vor:</p><ul><li>Als Zugang zu RabbitMQ wird in diesem Beispiel der Gastzugang verwendet, welcher bei einer Neuinstallation des Brokers automatisch eingerichtet wird.</li><li>Es wird ein logischer Empfangsendpunkt registriert, welcher Nachrichten entgegennimmt. Die hinter dem Empfangspunkt lauschenden Consumer werden mit LoadFrom(context) automatisch aus dem IoC Container geladen.</li></ul><h4>Schritt 2: Definition von Consumer-Klassen</h4><p><strong>Consumer</strong> sind Klasseninstanzen, welche eine asynchrone Nachricht verarbeiten. Sie werden von MassTransit selbstständig beim Erhalt einer passenden Nachricht instanziiert. Consumer implementieren das Interface IConsumer&lt;T&gt; , wobei der Typparameter den zu verarbeitenden Nachrichtentyp darstellt. Die Verarbeitungslogik wird in der Methode Consume erwartet:</p><pre>public class ProductRenamedConsumer : IConsumer&lt;ProductRenamed&gt;<br>{     </pre><pre>  public Task Consume(ConsumeContext&lt;ProductRenamed&gt; context)<br>  {<br>    var message = context.Message;</pre><pre>    // Hier z.B. Update Lesespeicher ...<br>    <br>    return TaskUtil.Completed;</pre><pre>  }<br>}</pre><p>Die Methode Consume gibt den Typ Task zurück, sodass sie auch als async deklariert werden kann. Für eine synchrone Verarbeitung kann — wie in obigem Beispiel — TaskUtil.Completed zurückgegeben werden.</p><p>Dank der IoC-Registrierung der Consumer aus Schritt 1 können Consumer auch Abhängigkeiten über Konstruktorparameter injiziert bekommen.</p><h4>Schritt 3: Versenden von Commands und Events</h4><p>Der Nachrichtenversand erfolgt im Kontext von MassTransit entweder gezielt an einen bestimmten, logischen Empfänger oder ungerichtet nach dem Publish/Subscribe-Muster.</p><p>Beim Versenden eines Commands hat man typischerweise einen bekannten Empfänger, von dem man weiß, dass der die Nachricht bearbeiten wird. Es ist üblicherweise auch nicht gewünscht, dass neben diesem Empfänger noch weitere Empfänger das Kommando verarbeiten. Daher nutzen wir für den Versand von Commands den gezielten Versand, welcher in MassTransit mit der Send Methode realisiert wird:</p><pre>var endpoint = await busControl.GetSendEndpoint(<br>  new Uri(“rabbitmq://localhost/notification-queue”)); <br>await endpoint.Send(new SendOrderNotification(...));</pre><p>Lösen wir ein Ereignis aus, kann es hingegen viele Interessenten geben, die auf das Ereignis reagieren möchten. In diesem Fall bietet sich der Versand mit der Publish Methode an:</p><pre>await busControl.Publish(new ProductRenamed(...));</pre><h3>Das Github-Beispiel</h3><p>Die Demo zum oben genannten Beispielszenario rund um den Bestell-Service ist auf Github verfügbar:<br><a href="https://github.com/marco-bue/masstransit-microservices">https://github.com/marco-bue/masstransit-microservices</a></p><p>Folgender Tech-Stack wurde dabei verwendet:</p><ul><li>.NET Core 2.0</li><li>ASP.NET Core 2.0</li><li>Autofac</li><li>Masstransit</li><li>RabbitMQ</li></ul><p>Der Order-Service wurde als ASP.NET Core 2.0 Anwendung realisiert, während der Notification Service und Product Service lediglich mit einer Konsolenanwendung simuliert wird, welche die passende Kommunikation mit dem Nachrichtenbus implementiert.</p><h4>Ausführen der Demo</h4><ul><li>Nach Klonen des Repositories und Installation von RabbitMQ können beide ausführbaren Projekte ( OrderService und OtherServicesSimulation ) gleichzeitig ausgeführt werden.</li><li>Der Produkt-Service wird für die Produkte P-123 , P-234 , P-345 wechselnde Produktnamen vergeben und entsprechende Ereignisnachrichten vom Typ ProductRenamed im System versenden.</li><li>Der Order-Service empfängt im ProductRenamedConsumer die Nachrichten und aktualisiert den eigenen Lesespeicher. Die Arbeit des Consumers kann auf der Konsole verfolgt werden, welche zyklisch Änderungsmeldungen ausspuckt:</li></ul><pre>Product P-345 is now called Jolly Rasta<br>Product P-345 is now called Death Starfish<br>Product P-123 is now called Screaming Narwhal</pre><ul><li>Der Order-Service bietet eine einfache REST-Api an, mit der Bestellungen getätigt und die Bestellhistorie gelesen werden kann. Während Produkte lediglich anhand ihrer ID bestellt werden, enthält die Historie auch die (sich häufig wechselnden) Namen der bestellten Produkte. Diese Namen werden aus dem lokalen Lesespeicher des Order-Service entnommen.</li></ul><h4>REST-API: Bestellung tätigen</h4><p>Die API kann mit einem REST-Testwerkzeug wie Postman oder Paw am leichtesten getestet werden.</p><p>Die Bestellung erfolgt mit einem POST an /api/orders und einem passenden JSON-Body:</p><pre>{<br>  “id”: “O-555”,<br>  “address”: {<br>    “street”: “2311 North Los Robles Avenue”,<br>    “zip”: “91004”,<br>    “city”: “Pasadena”,<br>    “countryIso”: “US”<br>   },<br>  “orderLines”: [<br>    {<br>      “productId”: “P-123”,<br>      “amount”: 10<br>    },<br>    {<br>      “productId”: “P-234”,<br>      “amount”: 5<br>    }<br>  ]<br>}</pre><p>Eine erfolgreiche Bestellung wird mit 200 OK quittiert.</p><h4>REST-API: Bestellhistorie abrufen</h4><p>Die Historie kann mit einem GET an /api/orders abgerufen werden. Beispielausgabe:</p><pre>[<br>  {<br>    “id”: “ORD-555”,<br>    “address”: {<br>       “street”: “2311 North Los Robles Avenue”,<br>       “zip”: “91004”,<br>       “city”: “Pasadena”,<br>       “countryIso”: “US”<br>  },<br>  “orderLines”: [<br>    {<br>      “productId”: “P-123”,<br>      “productName”: “Jolly Rasta”,<br>      “amount”: 10<br>    },<br>    {<br>      “productId”: “P-234”,<br>      “productName”: “Mad Monkey”,<br>      “amount”: 5<br>    }<br>   ]<br>  }<br>]</pre><p>Bei mehrfachem Abschicken der Anfrage werden sich die Produktnamen in der Antwort über die Zeit ändern.</p><p>Kleine Bemerkung am Rande: Die “Produktnamen” stammen im Übrigen aus dem Point&amp;Click-Klassiker Monkey Island und stellen dort die Namen verschiedener Piratenschiffe dar.</p><h3>Zusammenfassung</h3><p>Kommunikationsszenarien in microservice-basierten Architekturen müssen mit Bedacht implementiert werden. Asynchrone Nachrichten sind eine gute Möglichkeit, Abhängigkeiten zwischen einzelnen Diensten aufzulösen. Mit RabbitMQ und MassTransit existiert ein schlanker, lizenzfreier und erprobter Tech-Stack für .NET Core, um asynchrone Kommunikation schnell implementieren zu können.</p><p>Bei Fragen, Kritik und Anregungen stehe ich Ihnen gerne zur Verfügung, über Ihren Kommentar freue ich mich.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18ea58e4722d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>