<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anton Shyrokykh on Medium]]></title>
        <description><![CDATA[Stories by Anton Shyrokykh on Medium]]></description>
        <link>https://medium.com/@anton_shyrokykh?source=rss-f0dfebbb7a7f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*q-M-nuvFLsksiuSIkgW6dA.jpeg</url>
            <title>Stories by Anton Shyrokykh on Medium</title>
            <link>https://medium.com/@anton_shyrokykh?source=rss-f0dfebbb7a7f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 04:22:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@anton_shyrokykh/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Applying LRU cache in ASP NET Core applications]]></title>
            <link>https://medium.com/@anton_shyrokykh/applying-lru-cache-in-asp-net-core-applications-86dede5ce927?source=rss-f0dfebbb7a7f------2</link>
            <guid isPermaLink="false">https://medium.com/p/86dede5ce927</guid>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[asp-net-web-api]]></category>
            <category><![CDATA[lru-cache]]></category>
            <category><![CDATA[lru]]></category>
            <category><![CDATA[performance]]></category>
            <dc:creator><![CDATA[Anton Shyrokykh]]></dc:creator>
            <pubDate>Sat, 16 Jul 2022 15:48:20 GMT</pubDate>
            <atom:updated>2022-07-16T15:48:20.469Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*b81fabUrOMQfShiudHwIfA.jpeg" /></figure><p>In modern web application development, it is hard to overestimate the importance of cache. We store results for time-consuming, expensive or frequently repeating calculations in a kind of storage where calling storage is much faster and cheaper than calling origin or repeating calculations. We use RAM or optimized for fast key-value access databases such as Redis for this purpose. A cache is an ultimate tool for reducing response time and improving application scalability. But it has some limitations, which are primarily related to cache size. We will likely lack RAM or Redis storage to fully cache database table with millions of records. To mitigate cache size limitations while working with large datasets, we need to apply some kind of algorithm, which will help us to cache only data we need to cache — elements that are requested most frequently. In this article, we will consider in detail using one of such algorithms — LRU cache — in the context of ASP NET Core web applications.</p><p>This article could be useful for developers who are striving to continuously improve web application performance. Also, this article could be helpful for any web developers interested in expanding their professional toolkit.</p><h3>Description of LRU cache</h3><p>The solution to our problems should match two criteria. Firstly, cache size should be limited. Secondly, the algorithm should leave in cache only the most frequently called data. It seems that LRU (Least Recently Used) cache algorithm matches our criteria.</p><p>LRU is an algorithm that works using the principle of displacement of least recently used elements from the limited size cache. Implementation is often represented as a double-linked list and hash table. Hash-table provides fast access to the element by key (with the help of hash table data access usually takes constant time). As a value hash-table contains a reference to a double-linked list item, which includes references to prev, next and value itself. If we have found an element by key in the hash-table, we move the value to the head of the double-linked list (this operation is fast because we only need to reassign prev and next references), while other elements in the cache are automatically moved down. This way, at the tail of the list we always have the least recently used element. On adding a new element, if cache size is exceeded, the last item in the double-linked list is removed, maintaining maximum cache size.</p><p>It is usually better to understand how LRU works with illustrations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*EmW5QVejOz6EkDVXNEOK8Q.png" /><figcaption>LRU cache scheme</figcaption></figure><p>There are two main pitfalls in using LRU cache. Firstly, we need two data structures for LRU cache, which consumes more memory. Secondly, LRU cache while operating is continuously changing both data structures and the need for synchronization in a multithreaded environment arises. This is usually solved by wrapping all access code into a <strong><em>`lock`</em></strong> statement, which causes some performance overhead.</p><p>In standard C# libraries there is no default LRU cache implementation, but after some searching, we can discover <strong><em>`LRUCache`</em></strong> class in <a href="https://github.com/microsoft/botbuilder-dotnet/blob/402bc02b4cbbd2f4ec359134640e99211367e4a5/libraries/AdaptiveExpressions/LRUCache.cs">Microsoft Bot Framework SDK</a> repository. We can use this implementation in the article with some minor changes for adding expiration support.</p><h3>Attempting to use LRU cache directly</h3><p>Looking ahead, I note that the most efficient use of the LRU cache will require some preparatory work. But for demonstration purposes, let’s try to apply LRU cache directly and see what results we will receive.</p><p>For demo purposes, we will use ASP NET Core with Entity Framework application, the single option of which will be returning products from the database. We will have one million DB records, 10 000 (1%) of which will be requested more often <a href="https://github.com/MrPomidor/SmartCachingDemo/blob/master/src/Solution/Testing/LoadTest/Program.cs">according to the script</a>. As a load testing tool for checking cache efficiency we will use <a href="https://medium.com/@anton_shyrokykh/nbomber-as-an-alternative-to-jmeter-for-net-developer-432040b91763">NBomber</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c4be51f7efaf2471a70f2b6db4af5922/href">https://medium.com/media/c4be51f7efaf2471a70f2b6db4af5922/href</a></iframe><p>Let’s perform two tests: the first will be the control one without using the cache, second with <a href="https://github.com/MrPomidor/SmartCachingDemo/blob/master/src/Solution/APIs/Reusables/Caching/InMemory/LRUCache.cs">slightly modified LRU cache implementation</a> with capacity 5000. 5000 is a randomly taken number, which, as we know, is less than the often requested elements set size. Let’s see the results:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/38df038fd35148ce8746b6dfbb91b65f/href">https://medium.com/media/38df038fd35148ce8746b6dfbb91b65f/href</a></iframe><p>As we can see, applying LRU cache directly without preparation could have some positive effect on average response time, but this is far from its limit. Also worth noting that being not so lucky in choosing initial cache capacity and with a greater spread of often requested items we could get results worse than with no-cache. To make decisions consciously, we need to be more informed on how LRU cache works with different inputs. Let’s make an analysis.</p><h3>Impact of LRU cache size on its efficiency</h3><p>The metric by which we usually can judge cache efficiency is the ratio of “matches”, when requested elements were present in the cache, to “misses”, when requested elements were not found in the cache. By getting this ratio, we can make a conclusion on cache efficiency and also see if the most important elements “stay” in the cache for enough time. To collect this info let’s add several event-counters to <strong><em>`LRUCache`</em></strong> class and repeat a test to collect data with the help of <a href="https://github.com/MrPomidor/SmartCachingDemo/tree/master/src/Solution/Testing/CountersMonitor">the utility</a>. Event-counters is a tool for emitting real-time application metrics with minimal overhead, so developer could easily add and consume own metrics (to get more info on event-counters you can visit <a href="https://medium.com/criteo-engineering/net-core-counters-internals-how-to-integrate-counters-in-your-monitoring-pipeline-5354cd61b42e">this article</a>). Let’s add metrics for <strong><em>`TryGet`</em></strong> and <strong><em>`Set`</em></strong> methods and test again.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3aa9f7e28ba0318b4b3079694c9c0ead/href">https://medium.com/media/3aa9f7e28ba0318b4b3079694c9c0ead/href</a></iframe><p>By analyzing result data, we can conclude that for a cache with size of 5000 elements for each “match” we have approximately eight “misses”. Not quite impressive. Let’s try to illustrate how insufficient LRU cache size could influence its effectiveness with the next scheme:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*jJhql5JvLS3qsLiln4JZKw.png" /><figcaption>LRU cache with small size</figcaption></figure><p>When cache size is insufficient, the flow of rare requested elements displaces often requested elements from the cache. So to increase LRU cache efficiency, we need to choose the correct size to keep the required elements in cache for enough time. Also, keep in mind resource limitations. Let’s make a note regarding the correct cache size and continue our analysis.</p><h3>Elements popularity distribution as LRU cache efficiency factor</h3><p>As follows from LRU cache description, least recently used elements are displaced by often requested elements, meaning only needed items should stay in the cache. Several questions can arise. How often must one part of the elements be requested compared to the other part for the LRU cache to work effectively ? Will the LRU cache be efficient if all items are requested at the same frequency ? In other words, what elements popularity distribution should look like ?</p><p>By doing some search I’ve found a single source answering this question. As follows from <a href="https://redis.io/docs/manual/eviction/">Redis documentation</a> (we will consider Redis-based distributed version of LRU cache later in this article):</p><blockquote>Use the allkeys-lru policy when you expect a power-law distribution in the popularity of your requests. That is, you expect a subset of elements will be accessed far more often than the rest.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y6x8Y_UnAwPBY-2nZNHA9w.png" /><figcaption>Power-law distribution</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*txYTt0PC8BhESNhJXKtAnA.jpeg" /><figcaption>Uniform distribution</figcaption></figure><p>As a special case of power-law distribution we can consider Pareto distribution. Adapting Pareto distribution description to our case, we can get the best option for element popularity distribution: 20% of data should generate 80% of requests. As the worst option for using LRU cache, we can take a uniform distribution when elements are requested equally frequent.</p><p>Now we know that before applying LRU cache we should be sure that elements popularity distribution should approach Pareto distribution. We need to collect distribution data to determine if it is appropriate to use the LRU cache for our resources. We will collect this data using the next approach: firstly, we need <strong><em>`ConcurrentDictionary`</em></strong> where we will store and update resources call frequency data, <strong><em>`BackgroundService`</em></strong> which will aggregate values, calculate statistics and reset data, and event-counters to emit these statistics so we can consume it out-of-proc using <a href="https://github.com/MrPomidor/SmartCachingDemo/tree/master/src/Solution/Testing/CountersMonitor">our utility</a>. Let’s test our application without cache to collect elements popularity distribution data during several minutes.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/30c6218933850d774f9bd6c5cdc83d9f/href">https://medium.com/media/30c6218933850d774f9bd6c5cdc83d9f/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8bb012fff1640b701853104f02fe8bb2/href">https://medium.com/media/8bb012fff1640b701853104f02fe8bb2/href</a></iframe><p>According to the results, a small portion of data is requested significantly more often than others. Popularity distribution for our application is closer to Pareto distribution than to uniform one. That’s why applying LRU cache could be effective in our case. I think we get enough data to make some conclusions for our analysis.</p><h3>Choosing a mathematical model for LRU cache</h3><p>We have performed a shallow analysis of an attempt to apply LRU cache, and as a result, we have two recommendations. Firstly, for LRU cache being effective, cache size should be big enough. Secondly, elements popularity distribution should approach Pareto distribution. The first recommendation sounds good, but what does it mean “big enough” ?</p><p>To get a good answer, we should ask a good question, so let’s define it. We need cache size to be big enough so that often requested elements can be kept for long enough to be requested more than two times for the cache to be effective. Transforming the statement: cache size should provide a high enough <strong>probability</strong> of repeated “match” for often requested elements. We have the word <strong>probability</strong>, so let’s check if probability theory can help us.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/841/1*oYtJSc2ZTISOk7f_tafhfA.png" /></figure><p>Bernoulli formula by itself is not much useful for us, but if we can apply Bernoulli scheme, we also can use the formula for the most probable number of events. This sounds like exactly what we need and by slightly modifying the formula we can get a number of tests (cache size) for the desired number of events (we need more than two matches).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*-Kre6CFpWRTk7sKj6zYWQQ.png" /></figure><p>For applying the formula we only miss probability <em>p</em> of independent event <em>A </em>— the probability of the often requested element to “match” the cache. Luckily, we can easily get this probability from elements popularity distribution data we collected before using event-counters. Worth noting that we will make the calculation using real distribution data we get from our application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*AjwmdZLXcINFJRDU6U8ObQ.png" /></figure><p>After getting all inputs, let’s calculate LRU cache size, which will be effective for providing amount of “matches” more than two.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/37efa1bb62f3be43bcd274231faaeaec/href">https://medium.com/media/37efa1bb62f3be43bcd274231faaeaec/href</a></iframe><p>As we concluded before and based on formulas, the larger is LRU cache size, the more effective it is. For the test, let’s take the maximum value we’ve got from applying formulas and round it to 13 000. Also, let’s take a larger cache size of 15 000 as well to see the difference. Let’s run a test to collect “match”/”miss” and average response time stats.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/15f4c9ae38bde172ec9a533d974f4361/href">https://medium.com/media/15f4c9ae38bde172ec9a533d974f4361/href</a></iframe><p>From test results we can conclude that applying LRU cache with the correct size and after good preparations can result in significant performance growth. Also, good effectiveness is confirmed by “match”/”miss” rates. To demonstrate that our application became more scalable and load resistant, let’s repeat the test with higher RPS.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b450149e9b9eb0f4329c881e583b7747/href">https://medium.com/media/b450149e9b9eb0f4329c881e583b7747/href</a></iframe><h3>Distributed LRU cache</h3><p>We have considered using an in-memory LRU cache, which should match well for a system with one application instance. But what if we have a distributed system with many instances of the same application running ? In this case using in-memory LRU cache will be difficult due to load balancer requests distribution. We need elements tracking and displacement logic to be centralized. Luckily, Redis has build-in support for LRU eviction algorithm and you can configure it in two steps:</p><ol><li>Set memory limit with CONFIG command:</li></ol><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bc04a7185bf5b32c635187ca7adc66a7/href">https://medium.com/media/bc04a7185bf5b32c635187ca7adc66a7/href</a></iframe><p>2. Define LRU as key eviction policy:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9fd6480ef0533dab98e9a02f95039a5e/href">https://medium.com/media/9fd6480ef0533dab98e9a02f95039a5e/href</a></iframe><p>Let’s substitute our cache implementation with <a href="https://github.com/MrPomidor/SmartCachingDemo/blob/master/src/Solution/APIs/Reusables/Caching/Redis/ProductsRedisCache.cs">Redis-based</a> (we need just to substitute <strong><em>`IProductsCache`</em></strong> with another implementation) and run a load test.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ecd1d62570c18fe467622323e170fe67/href">https://medium.com/media/ecd1d62570c18fe467622323e170fe67/href</a></iframe><p>As we can see, distributed Redis-based version of LRU cache also improves our application performance.</p><h3>Considering memory limitations</h3><p>Remember that the main reason for using cache with displacement algorithm is limited system resources, especially memory. Suppose, as a result of request popularity distribution analysis, you have decided that applying LRU cache makes sense and you with the help of formulas have calculated minimum effective cache size. In that case, you also can calculate in advance how much memory it will take to support a cache of this size and if it matches your system resources limitations.</p><p>For calculating in-memory cache memory consumption we can write a simple console application:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/76397fb51a991e6a285fc3ceb8e332b1/href">https://medium.com/media/76397fb51a991e6a285fc3ceb8e332b1/href</a></iframe><p>For calculating Redis cache memory consumption we can use similar approach and write an integration test:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7d070801eb372c03dcd3a4e71f58d442/href">https://medium.com/media/7d070801eb372c03dcd3a4e71f58d442/href</a></iframe><p>This way you can receive more data which will help you to make an informed decision.</p><h3>Recommendations for optimal usage of LRU cache</h3><p>Let’s make a summary and form some action plan, following which you can make the right decision about applying or not applying LRU cache in your application.</p><p>0. Make initial system measurements with load testing to define if you have a performance bottleneck or problem to be solved and record starting point stats before starting optimization.</p><p>1. Gather request popularity distribution stats on real application to determine the appropriateness of using LRU cache. For LRU cache to be effective, the distribution must approach the Pareto distribution (20% of elements generate 80% of requests).</p><p>2. By using distribution data, calculate the minimum effective LRU cache size using the most probable number of events formula. Amount of matches should be greater than two (three, four or more).</p><p>3. Calculate the amount of memory consumed by LRU cache of such size and make sure it matches your limitations (server RAM or Redis subscription).</p><p>4. Implement LRU cache in your application</p><p>5. Perform load testing by emulating real request distribution. Analyse “match”/”miss” rate.</p><p>6. Test on a prod-like (or prod) environment.</p><h3>Bonus content: code that can be useful for you</h3><p>In this article, we have considered the nuances of using the LRU cache in web applications, and, as for me, we did it with enough level of detalization. Demo applications used in this article, docker-compose configuration, load test scenarios and other code are available on <a href="https://github.com/MrPomidor/SmartCachingDemo">GitHub</a>. However, I would consider this article incomplete without including the description of some code from the repository, which could potentially be useful for you in terms of LRU cache topic.</p><p><a href="https://github.com/MrPomidor/SmartCachingDemo/blob/master/src/Solution/Testing/UnitTests.Math/Probabilities.cs"><strong>Helper unit tests for formulas calculations</strong></a>. While working on this article, I have written some unit tests, for simplifying experimenting with different inputs: calculating cache size for known desired “matches” count, calculating the probability of often requested item appearance with known total items count, calculating most possible “matches” count with different inputs etc. By using these unit tests, you can easily test needed values and receive fast feedback.</p><p><a href="https://github.com/MrPomidor/SmartCachingDemo/tree/master/src/Solution/Testing/CountersMonitor"><strong>Utility for out-of-proc consuming of event-counters</strong></a>. Usually, for reading application metrics from event-counters you need a <strong><em>`dotnet-counters`</em></strong> console tool, but it is quite straightforward and it just displays real-time metrics values as they are emitted by an application. To be able to control emitted values, to aggregate and handle them, in terms of work on this article a utility tool was written. This utility reuses source code from <strong><em>`dotnet counters ps`</em></strong> command, to find the process we need by name, and from <strong><em>`dotnet counters monitor`</em></strong> command, to get access to counter values. The utility consists of three parts: <strong><em>`ProcessFetcher`</em></strong> to get Id of the required process, <strong><em>`StatsCollector`</em></strong> for collecting metrics and <strong><em>`StatsHandler`</em></strong> for handling collected information. The code could be useful if you need to read and process data from event-counters.</p><p><strong>Utility code measuring memory consumption</strong>. As we concluded before, to make the right decision on using LRU cache, you also need to know if the effective cache size matches your resource limitations. In the repository there is a code dedicated to finding the answer to this question for <a href="https://github.com/MrPomidor/SmartCachingDemo/blob/master/src/Solution/Testing/MemoryConsumptionTest/Program.cs">in-memory implementation</a> and <a href="https://github.com/MrPomidor/SmartCachingDemo/blob/master/src/Solution/Testing/UnitTests/RedisMemoryConsumptionTest.cs">Redis-based implementation</a> as well.</p><p>I hope this article and the materials attached to it were useful to you.</p><p>Thank you for your attention !</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86dede5ce927" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Entity Framework Core and high performance]]></title>
            <link>https://medium.com/@anton_shyrokykh/entity-framework-core-and-high-performance-b68e107972d4?source=rss-f0dfebbb7a7f------2</link>
            <guid isPermaLink="false">https://medium.com/p/b68e107972d4</guid>
            <category><![CDATA[entity-framework]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[dotnet-core]]></category>
            <category><![CDATA[entity-framework-core]]></category>
            <category><![CDATA[dapper]]></category>
            <dc:creator><![CDATA[Anton Shyrokykh]]></dc:creator>
            <pubDate>Wed, 13 Jul 2022 10:49:39 GMT</pubDate>
            <atom:updated>2022-07-13T10:49:39.159Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*QViub0GSY40-qrf85OIl6A.jpeg" /></figure><p>Entity Framework Core is recommended and the most popular tool for interacting with relational databases on ASP NET Core. It is powerful enough to cover most possible scenarios, but like any other tool, it has its limitations. Long time people said (not without reason) that Entity Frmaework does not match high load systems and for these scenarios it is better to use Dapper. But time goes by and Entity Framework is growing and getting better, especially from the performance perspective. In addition to internal performance improvements for standard APIs of Entity Framework and performance improvements of the platform itself, Entity Framework provides a special set of features and configuration options dedicated to significantly increase performance. In this article we will consider Entity Framework Core from the performance perspective and compare it with Dapper as of July 2022. Let’s check if recommendation “rewrite everything on Dapper” is still actual :)</p><p>This article could be useful for developers who use Entity Framework Core in everyday development. Also, this article could be helpful to high-load system developers to update their knowledge about Entity Framework latest performance-oriented features.</p><h3>Introduction to EF</h3><p>Before diving into performance topic, it will be useful to remember what EF is and describe some aspects of its work, so we can better understand optimization approaches which we will consider in this article. So EF is an object-relational mapper, or in other words tool that binds C# object model (classes, collections, properties) with relational database model (tables, columns, rows, foreign keys etc). The main EF object, which allows developers to interact with the database, is class inherited from <strong><em>`DbContext`</em></strong>. The context contains several collection objects <strong><em>`DbSet`</em></strong>, which usually represent a database table. To access data from table we call these collections using LINQ queries, which are behind the scenes translated into SQL by calling methods <strong><em>`ToArray`</em></strong>, <strong><em>`ToList`</em></strong>, <strong><em>`FirstOrDefault`</em></strong> etc., so we can work with data as we usually work with regular C# objects.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/872db062dbab03b4c025fec6e278e026/href">https://medium.com/media/872db062dbab03b4c025fec6e278e026/href</a></iframe><p>The way how EF works gives developers several advantages. Firstly EF takes responsibility for forming correct and SQL injection safe requests to database using strictly typed LINQ queries. The same C# code will work for MSSQL, Oracle and MySQL, because concrete database provider implementations are hidden from developers using abstractions. Developers in most cases shouldn’t deal with SQL syntax at all and can fully concentrate on application logic. Secondly, EF includes a mechanism that tracks changes in object properties (change-tracking), allowing to form Update and Delete database queries without interaction with any SQL code. For example:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/56d36fa84678fc544ecf197ce38d8f3b/href">https://medium.com/media/56d36fa84678fc544ecf197ce38d8f3b/href</a></iframe><p>EF has reach functionality, which makes development process significantly easier and faster. Still, it comes with a price and each stage of processing before sending SQL request to database and after receiving a response consumes additional resources. Let’s try to make a simplified scheme of how EF works from constructing LINQ requests to receiving data in client code.</p><ol><li>Getting an instance of <strong><em>`DbContext`</em></strong>. To begin the work we need to get an instance of<strong><em> `DbContext`</em></strong>, which contains everything we need.</li><li>Compiling LINQ query to SQL. <strong><em>`IQueryable`</em></strong> interface implementation, which we receive by calling LINQ extension methods on <strong><em>`DbSet`</em></strong>, is a request object which should be executed. The object is constructed in builder-like way: each extension method as <strong><em>`Where`</em></strong>, <strong><em>`OrderBy`</em></strong>, <strong><em>`Select`</em></strong> adds some additional information to request object, which should be translated to SQL. <strong><em>`IQueryable`</em></strong> inherits from <strong><em>`IEnumerable`</em></strong>, but until you explicitly call <strong><em>`IEnumerable`</em></strong> (or <strong><em>`IAsyncEnumerable`</em></strong>) methods — SQL query will not be constructed and executed. Translation process begins when method casting <strong><em>`IQueryable`</em></strong> to <strong><em>`IEnumerable`</em></strong> (or <strong><em>`IAsyncEnumerable`</em></strong>) such as <strong><em>`ToArray`</em></strong>, <strong><em>`ToList`</em></strong>, <strong><em>`FirstOrDefault`</em></strong> etc. was called. EF translates <strong><em>`IQueryable`</em></strong> to SQL while supporting an internal caching mechanism, which allows reusing calculation results for same LINQ queries.</li><li>Sending SQL to database and receiving a response. (server-side calculations)</li><li>Results materialization into C# objects.</li><li>Objects registration into the change-tracking system. After materialization happens, EF by default registers objects in internal change-tracking system, which tracks object property changes and on calling <strong><em>`SaveChanges`</em></strong> forms the corresponding Update database query. Supporting change-tracking creates an overhead and consumes resources.</li><li>Performing client-side part of LINQ query (client-side calculations). When receiving LINQ request EF tries to transform it to SQL so it can be executed on database side (server-side calculations). But in some cases requests cannot be fully translated to SQL so part of the expression should be calculated on client side after results from the database are received. In earlier versions of EF developers could know that request is executing on client side only from EF logs or by configuring throwing exceptions in such cases, but with the latest EF versions, when EF cannot fully convert LINQ query to SQL it will always throw an exception, requiring explicit call of <strong><em>`AsEnumerable`</em></strong>, <strong><em>`ToList`</em></strong> etc before LINQ expression part, which could be executed only on client code side (client-side calculations). You can get more detailed description of this mechanism by visiting this <a href="https://docs.microsoft.com/en-us/ef/core/querying/client-eval">Microsoft article</a>. It is for the benefit of the developer to write code in a way when most of the calculations, especially in <strong><em>`Where`</em></strong> block, is performed on database server side.</li><li>Receiving results by calling client code.</li></ol><p>As we can see, a lot of resource-consuming operations are happening between creating <strong><em>`DbContext`</em></strong> instance, calling ADO NET and receiving results in client code. These operations consume CPU resources, create a lot of objects, save references on them, create GC pressure, fill and flush internal caches etc. Dapper in its turn is a minimalistic proxy between ADO NET and client code, which lacks EF functionality, but has huge performance benefits over EF. Let’s demonstrate some code using Dapper:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/023e05346a41bf8850250fd38eac44de/href">https://medium.com/media/023e05346a41bf8850250fd38eac44de/href</a></iframe><p>Now when we better understand how EF works and what it does under the hood, we can move on to the system under test review, which we will try to optimize.</p><h3>System Under Test (SUT)</h3><p>For demonstration purposes we need some web API, which will interact with test MSSQL database AdventureWorks, implementing some of most common data access scenarios:</p><ul><li>GET request by Id with data from one table. <em>Get product by Id</em></li><li>GET request by Id with data from multiple related tables (JOIN-s). <em>Get product with model and product category by id</em></li><li>GET request of paged data from single table. <em>Get products page</em></li><li>GET request of paged data from multiple related tables (JOIN-s). <em>Get products page with model and product category datas</em></li><li>POST request for resourse creation. <em>Create product</em></li><li>PUT request for edit resourse. <em>Edit product name</em></li></ul><p>We need to implement several versions of API with different implementations of <strong><em>`IProductsRepository` </em></strong>using EF or Dapper for data access. For a complete load testing we will use NBomber for each of these scenarious one by one. To get more details on working with NBomber you can visit <a href="https://medium.com/@anton_shyrokykh/nbomber-as-an-alternative-to-jmeter-for-net-developer-432040b91763">this article</a>. For more fast running local test we in some cases will use some <a href="https://github.com/MrPomidor/EFCorePerformanceTipsDemo/tree/master/src/Solution/Tests/Benchmarks">BenchmarkDotNet scenarious</a>, which will repeat our API in miniature by calling different <strong><em>`IProductsRepository`</em></strong> implementations (Dapper, EF Default, EF with different improvement options) in DI scope:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/46044a7358e713a13764424e2e471dcb/href">https://medium.com/media/46044a7358e713a13764424e2e471dcb/href</a></iframe><p>After we get through all performance improvement recommendations <a href="https://docs.microsoft.com/en-us/ef/core/performance/advanced-performance-topics">in the list</a> we will perform the last NBomber test, where we combine all improvements to make some conclusions. All code used in this article, demos, load testing scenarios etc. is available on <a href="https://github.com/MrPomidor/EFCorePerformanceTipsDemo">GitHub repository</a>.</p><p>Before we start optimization let’s do some initial measurements for Dapper and “out of the box” EF version. We will run each version of application sequentially, testing each scenario using 30 parallel clients and sending requests non-stop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D22YJDNuUyaOXsZlG0t9dw.png" /><figcaption>EF Default and Dapper</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/50dab78869b7417f5bfa5f59833b796c/href">https://medium.com/media/50dab78869b7417f5bfa5f59833b796c/href</a></iframe><p>As we can see, in this configuration EF is <strong>19–30</strong> percent less performant than Dapper in most read scenarios, and significantly inferior in creation and editing scenarios. Now we have a starting point to compare with, so let’s start improving.</p><h3>DbContext pooling</h3><p>To raise EF Core performance we need to gradually reduce the influence of intermediate stages ,described earlier, by reducing allocations, repeated calculations and by doing some calculations in advance if possible. First of all Microsoft <a href="https://docs.microsoft.com/en-us/ef/core/performance/advanced-performance-topics#dbcontext-pooling">recomends</a> using pool for objects of type <strong><em>`DbContext`</em></strong>. Obvious advantage for this recomendation is reducing GC pressure by reusing “heavy” objects, which will positively influence application behavior under load. Easy configuration is another advantage — for enabling pooling you need to change single method call from <strong><em>`AddDbContext`</em></strong> to <strong><em>`AddDbContextPool`</em></strong> in Program.cs. Your data access code (in our case it is implementation of <strong><em>`IProductsRepository`</em></strong>) will not be affected. But worth noting that in this case your <strong><em>`DbContext`</em></strong> essentially becomes a singleton (instance of <strong><em>`DbContext`</em></strong> in the pool will leave during all application lifetime) and should not store any state between usings. However, if you need to use some scoped context data, EF developers created a good guide describing such possibility, which you can visit using <a href="https://docs.microsoft.com/en-us/ef/core/performance/advanced-performance-topics#managing-state-in-pooled-contexts">this link</a>. Also it is important to set correct pool size because when pool size exceeded, new instances of <strong><em>`DbContext`</em></strong> will be created.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2fa138371a05093fcf329fee6696b932/href">https://medium.com/media/2fa138371a05093fcf329fee6696b932/href</a></iframe><h3>Disable change tracking for read-only scenarios</h3><p>As we mentioned before, <a href="https://docs.microsoft.com/en-us/ef/core/querying/tracking">change-tracking system</a> allows us to update data by transforming changed properties of C# objects to SQL Update statements. This system is enabled by default but it makes sense only in scenarios when we need to edit or update something. In read-only scenario this system just creates additional overhead. Fortunately, this system can be easily disabled by calling <strong><em>`AsNoTracking`</em></strong> on your <strong><em>`IQueryable`</em></strong> expression.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e19124bed04d992903c3e9b01d9e7b1b/href">https://medium.com/media/e19124bed04d992903c3e9b01d9e7b1b/href</a></iframe><p>For a couple of years, I got into the habit of always writing requests with <strong><em>`AsNoTracking`</em></strong>, because read-only scenarios usually appear more often than edit. But if you have no such habit, you need to do some work to analyze your data access code, add <strong><em>`AsNoTracking`</em></strong> and do some testing to be sure any edit scenario was not broken.</p><p>Worth adding that default request behavior could be configured in EF in a way where all requests will by default copy <strong><em>`AsNoTracking`</em></strong> behavior without calling it explicitly. You can configure it in <strong><em>`AddDbContext`</em></strong> call. Then you should manually add <strong><em>`AsTracking`</em></strong> in scenarios where you need to edit something.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2abba74b4ec7f89dd0f2d0671e3ad09d/href">https://medium.com/media/2abba74b4ec7f89dd0f2d0671e3ad09d/href</a></iframe><h3>Using DbContext.Entry for editing</h3><p>Let’s highlight and investigate test results for editing scenario (Edit product), where Dapper is two times more performant than EF. This result can be easily explained by looking into editing code in EF version of <strong><em>`IProductsRepository`</em></strong>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7b0c61ee31faf0f4e316e4349da4000f/href">https://medium.com/media/7b0c61ee31faf0f4e316e4349da4000f/href</a></iframe><p>For performing an update with C# first you need to receive object, by requesting it from the database, then modify it in code and call <strong><em>`SaveChanges`</em></strong>, which sends another request to the database. Dapper is two times faster for editing because EF needs two times more database requests to perform an edit. But EF has one more way of editing entities, which allows us to perform single database request. For this we need to create <strong><em>`Product`</em></strong> instance manually, assign required values to properties, manually attach instance to <strong><em>`DbContext`</em></strong> change tracking, select only important properties for an update if needed, and then call <strong><em>`SaveChanges`</em></strong>. In our case, we only need to change product name, so the code will be:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c5113fd46a9f1307043e356c13d28c84/href">https://medium.com/media/c5113fd46a9f1307043e356c13d28c84/href</a></iframe><h3>LINQ to SQL pre-compilation</h3><p>An important part of EF work is the transformation of LINQ expressions, created by C# developer, to SQL queries, consumed by database. Requests compiling is often performed operation, so it should be considered as an optimization target. To avoid LINQ to SQL transformations during application handling workload, EF developers provide a mechanism for pre-compilation of LINQ code into a thread-safe delegate, for which all calculation was already performed and which could be saved in a static variable to be reused in the application. To create such a delegate, you need to pass your LINQ code into static method <strong><em>`EF.CompileQuery`</em></strong>/<strong><em>`EF.CompileAsyncQuery`</em></strong>, also passing all external variables used by your LINQ code as method parameters. As a result you will get a delegate of type <strong><em>`Func&lt;TDbContext, TParameter1, …, TResult&gt;`</em></strong> which you could call not wasting resources on LINQ to SQL translation while application handling workload.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/348c75363afbcca3d445e66c6f3f4928/href">https://medium.com/media/348c75363afbcca3d445e66c6f3f4928/href</a></iframe><p>Despite the expected benefits of using this approach, such as reducing allocations and CPU use, it also comes at a cost. Firstly, as you can notice from code sample, the code became a bit less readable. Secondly, to use this approach you should consider spending more time than adding <strong><em>`AsNoTracking`</em></strong>, especially for rewriting and testing existing code. Separately I would like to note that<strong><em> `EF.CompileAsyncQuery` </em></strong>method interface is, in my opinion, a bit confusing and its documentation is not very detailed. To get familiar with existing documentation, please follow <a href="https://docs.microsoft.com/en-us/ef/core/performance/advanced-performance-topics#compiled-queries">this link</a>.</p><h3>Disable internal concurrency checks</h3><p><strong><em>`DbContext`</em></strong> in Entity Framework Core, unlike the full framework version, is not supporting multithread scenarios. To support this limitation, EF contains internal checks, which detects accessing <strong><em>`DbContext`</em></strong> from several threads and through clear exception notify developers of improper context using. However, if your application was repeatedly tested in production for a long time, you are absolutely sure that you have no concurrency errors and you are using <strong><em>`DbContext`</em></strong> in a correct way, should we consider such checks as an overhead that we can optimize ? All recommendations above could create some discomfort while developing and have some limitations, but none of them compromise the correctness and operability of EF. Disabling concurrency checking code could have unpredictable effects and there is a direct warning about it in EF documentation:</p><blockquote><strong>WARNING</strong>: Only disable thread safety checks after thoroughly testing that your application doesn’t contain such concurrency bugs.</blockquote><p>But in the context of this article when listing ways to improve EF performance it is worth noting that we have the ability to disable such checks for <strong><em>`DbContext`</em></strong>. To do this you need to call correspondent configuration method in <strong><em>`AddDbContext`</em></strong>/<strong><em>`AddDbContextPool`</em></strong>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b663eadfbd9b9fd56de1f2e2d30a62df/href">https://medium.com/media/b663eadfbd9b9fd56de1f2e2d30a62df/href</a></iframe><p>This optimization option was tested with BenchmarkDotNet but compared to others shows minimal influence on application performance. Unfortunately, I could not reproduce <strong>5</strong> percent performance improvement described in <a href="https://github.com/dotnet/efcore/pull/24125#issuecomment-777780033">this GitHub issue</a>. It is up to you whether to apply this optimization or not.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6c2e1102bea2976efb946d0d110ce704/href">https://medium.com/media/6c2e1102bea2976efb946d0d110ce704/href</a></iframe><h3>Combining improvements</h3><p>We have considered Microsoft’s main recommendations for improving EF’s performance, explored how they work and pointed out possible pitfalls. Let’s summarize the list:</p><ul><li>use <strong><em>`DbContext`</em></strong> pooling</li><li>use <strong><em>`AsNoTracking`</em></strong> for read-only scenarios</li><li>use pre-compiled LINQ to SQL queries</li><li>disable EF internal concurrency checks (remembering the risks)</li></ul><p>It is time to combine them and run a load test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qSDVr83ZTffFg1jnNkxGtw.png" /><figcaption>EF Default, EF Improved and Dapper</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/40238f9a0e4b481fdf29f2cd3bc72d11/href">https://medium.com/media/40238f9a0e4b481fdf29f2cd3bc72d11/href</a></iframe><p>According to testing results of three API versions, we can see that improvements allow us to increase performance of EF compared to “out of the box” configuration by <strong>6–25</strong> percent. Also, the gap between Dapper and EF was significantly reduced and now Dapper is only <strong>1.5–4.2 </strong>percent better for read scenarios.</p><p>Unfortunately, we were not able to get performance similar to Dapper for edit and create scenarios, while isolating C# developer from writing SQL code. Dapper is <strong>76</strong> better in edit scenario and <strong>74</strong> better in create scenario. However, EF still allows developer to write SQL queries manually using <strong><em>`DbContext.Database.ExecuteSqlRaw`</em></strong>. This way you can optimize the bottleneck without using external libraries except for EF. Benchmark results show us that EF <strong><em>`ExecuteSqlRaw`</em></strong> performance is almost identical to code written with Dapper for both scenarios:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9ca059659c1b4091aa2c5e15d550fa23/href">https://medium.com/media/9ca059659c1b4091aa2c5e15d550fa23/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/737f12943a12c68fab708e1dccce1724/href">https://medium.com/media/737f12943a12c68fab708e1dccce1724/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d8606b1bfd2328cf686cf98184bf326a/href">https://medium.com/media/d8606b1bfd2328cf686cf98184bf326a/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a56dba4a30a7c980f428d40d411d10a9/href">https://medium.com/media/a56dba4a30a7c980f428d40d411d10a9/href</a></iframe><p>Worth noting that in <a href="https://docs.microsoft.com/en-us/ef/core/what-is-new/ef-core-7.0/plan#performance-of-database-inserts-and-updates">EF 7 roadmap</a> it is planned to perform optimization of change-tracking mechanism and improve performance for Insert and Update scenarious:</p><blockquote>For EF7, we plan to focus on performance related to database inserts and updates. This includes performance of change-tracking queries, performance of `DetectChanges`, and performance of the insert and update commands sent to the database.</blockquote><p>We can track the development process on <a href="https://github.com/dotnet/efcore/issues/26797">GitHub</a>, hoping that gap between EF and Dapper will be significantly reduced with the next release.</p><h3>Summary</h3><p>As we can see, EF Core as for July 2022 with correct configuration can show results comparable to Dapper for most read scenarios while saving advantage of the generation of correct and safe SQL code while using strictly-typed C# LINQ expressions. However, EF is still far behind Dapper in Insert and Update scenarios while using C# objects and change-tracking for editing. Still, developers can optimize this by using the raw-SQL approach if needed. We could expect reducing the gap between EF and Dapper in these scenarios in the next release. In my opinion, and as the practice has shown, the latest versions of EF Core are quite applicable for high-load systems. Given the rich functionality, support, powerful community and evolving performance of EF Core and NET platform with each release, you will make the right decision by choosing EF Core. Hope this article was useful for you.</p><p>Thank you for your attention !</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b68e107972d4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Simple steps towards boosting ASP NET Core application performance]]></title>
            <link>https://medium.com/@anton_shyrokykh/simple-steps-towards-boosting-asp-net-core-application-performance-82c564bdf092?source=rss-f0dfebbb7a7f------2</link>
            <guid isPermaLink="false">https://medium.com/p/82c564bdf092</guid>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[load-testing]]></category>
            <category><![CDATA[aspnetcoreapi]]></category>
            <category><![CDATA[dotnet-core]]></category>
            <category><![CDATA[asp-net-web-api]]></category>
            <dc:creator><![CDATA[Anton Shyrokykh]]></dc:creator>
            <pubDate>Tue, 12 Jul 2022 21:42:23 GMT</pubDate>
            <atom:updated>2022-07-12T21:46:06.301Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*V16fm_vrIuXAS27iW51lrA.jpeg" /></figure><p>Facing performance-related issues while developing applications is a matter of time. After searching using different data sources, you will find a list of performance advice for concrete situations and the entire application. In this article we will consider performance advice, which aims to boost entire application performance with maximum effect while minimizing development effort. We will test how this advice will affect application performance, calculate performance boost and analyze if there are any pitfalls to point out.</p><p>This article will be useful for developers and developer leads striving for high application performance. Also, it will be helpful for experienced developers to be used as a starting point for creating their own ASP NET Core performance checklist or extending an existing one.</p><h3><strong>When do we need to optimize ?</strong></h3><p>Performance is a large and complicated topic. To cover it, we need to address every application aspect: from source code, written by your team, to external libs in use, databases, infrastructure etc. Also, while talking about performance, it will be an omission not to talk about performance measurement. To define system performance level and its behavior under the load, we need performance testing using a variety of monitoring tools.</p><p>In any application, from simple CRUD to distributed microservice-based system, there will always be issues, a lot of real (or imaginary) performance bottlenecks and too many things which can go wrong. Talking about performance, what shall we begin with ? Let’s narrow a search field by modeling different situations which developers usually face.</p><p>According to my experience, the situation when users experience significant discomfort from slow interaction with application is the most common source of performance optimization tasks. Often solving this kind of tasks requires focus on concrete application part. Code changes are very local and rarely affect entire application. Each situation is unique and requires an individual approach. There is no single way to solve these issues, only a common approach — define bottleneck with performance testing, safe initial metrics and make improvements until results become satisfactory.</p><p>What are alternative sources of performance optimization tasks ? Let’s imagine the sales department reporting an amazing contract with a new partner was signed, and soon your application will get tens of thousands of new active users, or powerful marketing company was planned where expected effect is similar. In this case, we also need some kind of optimization, but it became far more complicated to define a concrete tasks and work focus areas. The tasks could be formed as <em>“We need to make our entire application more scalable and fast”</em>. What approach can we apply to solve tasks with such a wide focus area ? Does it mean that we need to optimize each endpoint in our application or at least a dozen of most often called, or do we have some other solution that requires less effort ? Are there any performance recommendations, which will take minimal effort but will generate a significant positive effect for the entire application performance ?</p><p>Doing short research on the internet, you will likely find a list of such recommendations for boosting performance for the entire ASP NET Core application. Still, before applying them, you need more knowledge regarding required developer efforts and the predicted effect on your application throughput (highly desirable to have some concrete numbers in percentages or RPS). This is the topic we will highlight in this article.</p><p>In this article we will go through next recommendations:</p><ul><li>use the latest version of the framework and dependent libs</li><li>use System.Text.Json for request serialization/deserialization</li><li>use server GC mode</li><li>use async/await</li><li>manually configure threads in thread pool</li></ul><h3>What do we need to remember before we start optimization ?</h3><p>Before performance optimization starts, you need to consider two points.</p><p>Firstly, your application should not lose functionality. To protect the application from regression bugs, you need to have integration and end-to-end tests scope in place, confirming that changes did not break anything and the application is working fine. This point is valid for any refactoring.</p><p>Secondly, any performance optimizations should rely on metrics. You should record initial results before making changes to be sure you make performance better but not worse. Even simple and, at first glance, predictable changes could lead to unexpected performance degradation, and we will consider case which demonstrates this in the article.</p><h3>System under test</h3><p>For testing different recommendations we will use some kind of cutted CMS system on ASP NET Core with Entity Framework, based on Microsoft test database Adventureworks. This CMS will return goods and orders, which makes it similar to most services that accept requests and return some data from the database. Database and other dependencies will be up and running with <strong><em>`docker compose up -d`</em></strong>. We will use <a href="https://medium.com/@anton_shyrokykh/nbomber-as-an-alternative-to-jmeter-for-net-developer-432040b91763">NBomber</a> as a load testing tool and <strong><em>`dotnet counters`</em></strong> and PerfView as monitoring tools. All demos and code samples for this application are available on <a href="https://github.com/MrPomidor/ASPNetPerfImprovementsDemo">GitHub</a>.</p><p>Worth noting that results which we will get are unique for OS and hardware configuration on which we will run performance testing. Its purpose is to demonstrate the effect and give some relative numbers for being used in estimates and forecasts. For your system, numbers could be different. You should always confirm positive (or negative) effects on a system that is as close to your prod configuration as possible.</p><h3>Recommendation 1: Use the latest versions of the framework and libraries</h3><p>By releasing new versions of .NET framework and related libraries and packages, such as ASP NET Core and Entity Framework Core, Microsoft not only extends existing APIs and introduces new features but also continuously works on performance improvements. Memory consumption optimizations, extending existing APIs with copy-free types support as <strong><em>`Span`</em></strong> or <strong><em>`Memory`</em></strong>, extending support for <strong><em>`ValueTasks`</em></strong> etc. To track optimization-related work, you can visit issues with related tags on GitHub (for example for <a href="https://github.com/dotnet/aspnetcore/labels/area-perf">ASP NET</a> and <a href="https://github.com/dotnet/runtime/labels/optimization">Runtime</a>). While reading release notes of each new version, we can point out that the framework becomes faster eventually.</p><p>Updating framework version on your project could be fast and safe (just change<strong><em> `TargetFramework`</em></strong> in project configuration and update related NuGet packages version), but it could also be a complicated and unpredictable process, which requires serious code changes, adapting the API of new library versions, fixing bugs which can appear etc. In order to make the decision for updating framework version team lead requires some understanding of what results it will bring and how beneficial it will be from application performance point of view. Let’s do some tests and try to answer these questions.</p><p>For this test, we will need two API projects with different LTS framework versions (Core 3.1 and NET 6), which will have identical code. We will test two methods:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/40593a7b1da7e5490f34192f1a7c2477/href">https://medium.com/media/40593a7b1da7e5490f34192f1a7c2477/href</a></iframe><p>In our <a href="https://github.com/MrPomidor/ASPNetPerfImprovementsDemo/tree/master/Solution/FrameworkVersionsComparison">load test scenario</a> we will have 14 clients (7 for each method) which will send maximum possible number of requests during 3 minutes test. That’s how we can see the difference in peak throughput for the current configuration. To become more familiar with NBomber you can visit <a href="https://nbomber.com/docs/loadtesting-basics">the project site</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/106c6cd3f6e7915212441e6b9df5466b/href">https://medium.com/media/106c6cd3f6e7915212441e6b9df5466b/href</a></iframe><p>Let’s run the test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/673/1*hFCgAKY3MQJEc_EG3m00yA.png" /><figcaption>Frameworks comparison</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e76192e56634b23d10f9a821b65f8333/href">https://medium.com/media/e76192e56634b23d10f9a821b65f8333/href</a></iframe><p>As a result, we can see significant performance growth — application could handle <strong>39</strong> percent requests more (with an average response time lower than 1 second). Worth noting that we have tested peak throughput. When testing these methods under normal load with stable RPS, the average response time will not be significantly lower.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2026b2ca8541bdf7de6597c738a88951/href">https://medium.com/media/2026b2ca8541bdf7de6597c738a88951/href</a></iframe><p>Upgrading framework version, as we can see, allows application to handle more requests and support higher throughput. Developer efforts for upgrading framework version could vary from one project to another. But you can make efforts more predictable by doing some preparations, learning libraries being used and reading Microsoft <a href="https://docs.microsoft.com/en-us/aspnet/core/migration/31-to-60?view=aspnetcore-6.0&amp;tabs=visual-studio">framework migration guides</a>. Recommendation for upgrading framework version, as for me, looks pretty viable and corresponds to criteria “minimum effort — significant impact on entire application”.</p><h3>Recommendation 2: Use System.Text.Json</h3><p>Serialization code participates in every response, which means it is definitely part of any hot path in the API application. For a long time, Newtonsoft.Json, with all its reach functionality, was used as the default serializer, but starting from ASP NET Core 3.0 it was substituted with System.Text.Json.</p><p>If a new, more performant serializer is used by default for all projects starting from ASP NET Core 3.0 (which was released more than three years ago), is it useful to recommend anyone to migrate to System.Text.Json ? Yes, it is. Many applications were written relying on Newtonsoft.Json API, such as hand-written JsonConverters. To save backward compatibility, many developers choose to leave Newtonsoft.Json as the default json serializer and don’t perform a migration. Backward compatibility is provided with <strong><em>`Microsoft.AspNetCore.Mvc.NewtonsoftJson`</em></strong> package, which is still popular even for NET 6.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/923/1*_1DzTmSFxVinUeLVJQD8EA.png" /><figcaption>Microsoft.AspNetCore.Mvc.NewtonsoftJson downloads stats</figcaption></figure><p>Worth noting that System.Text.Json differs from Netwonsoft.Json more than Core 3.1 differs from NET 6. System.Text.Json has more strict rules by default and also doesn’t support some scenarios (you can read more about it <a href="https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-migrate-from-newtonsoft-how-to">here</a>). Serialization/deserialization code is usually segregated from controllers and application logic and configured in Program.cs/Startup.cs, but if your code does not work with System.Text.Json “out of the box”, you could spend significant effort on migration. To make this decision, you need to know how it will affect your application performance and throughput.</p><p>For the test, we will need two identical projects on NET 6, which will only differ with default serializer. We will test two API endpoints. First will return the order entity by id, for testing small objects (~1 KB). The second will return a page with 100 order entities for testing performance on large objects (~190 KB). In our <a href="https://github.com/MrPomidor/ASPNetPerfImprovementsDemo/tree/master/Solution/SerializerComparison">test scenario</a> we will see peak load with 20 parallel clients (10 for each endpoint).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/852/1*fMRVfdTvrY2i8o2KEw_Ucg.png" /><figcaption>Serializers comparison</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7114eb196b9c423cd336a29d68c32f9f/href">https://medium.com/media/7114eb196b9c423cd336a29d68c32f9f/href</a></iframe><p>According to test results, application throughput has grown by <strong>22</strong> percent. As we can see, migration to System.Text.Json can significantly improve application behavior under load. Also worth noting lowered memory consumption and GC pressure (tested with stable RPS, same for both versions):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8b5e3494bbea7dfadde42fdb34ac4f5a/href">https://medium.com/media/8b5e3494bbea7dfadde42fdb34ac4f5a/href</a></iframe><p>But this is not the limit for System.Text.Json. With code generators feature appearing in NET 6, we are able to make auto-generated serializers for your models to make the performance even better because of the reflection-free approach. You need to create <strong><em>`partial`</em></strong> context class, inherited from <strong><em>`JsonSerializerContext`</em></strong>, point required classes via attributes, then register context in Program.cs/Startup.cs.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/af6e50e9009b9cb5bab084367fdc903b/href">https://medium.com/media/af6e50e9009b9cb5bab084367fdc903b/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6d2c3a2c80b55b3a67da41970d06069c/href">https://medium.com/media/6d2c3a2c80b55b3a67da41970d06069c/href</a></iframe><p>However, you should know that using generators has its costs. If for the majority of Newtonsoft.Json functions, you could find analog or workaround in System.Text.Json, for generators most of customization including custom value converters is unavailable (more detailed about limitations could be found <a href="https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-source-generation-modes?pivots=dotnet-6-0">here</a>). Functionality for handling cyclic references is also unavailable when using generators, which we can see even in test scenarios. When using Entity Framework models you could have parent entity that contains a reference to collection of children, when each child contains a reference to parent. In our test scenario we have <strong><em>`OrderHeaderHeader`</em></strong> and <strong><em>`SalesOrderDetail`</em></strong>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/293d32f10f5b219d0a3caf54fa35cabc/href">https://medium.com/media/293d32f10f5b219d0a3caf54fa35cabc/href</a></iframe><p>To solve this issue you have several options available. The first option is to limit using of navigation properties in models, which mean rewriting application and limiting development convenience. The second option is manually setting null to all references. Both of these options hardly match the “minimum efforts” principle, but let’s see if it is worth it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/842/1*6zE4dtf-G-cGAVNl6sIsJg.png" /><figcaption>Serializers with generators comparison</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/96439bb43891b0914391e1c3a574676c/href">https://medium.com/media/96439bb43891b0914391e1c3a574676c/href</a></iframe><p>Compared with default System.Text.Json we can see that application became <strong>6</strong> more performant. It is up to you to decide if using generators is worth it. But migration from Newtonsoft.Json to System.Text.Json, as for me, is worth considering as a good recommendation.</p><h3>Recommendation 3: Use server GC mode</h3><p>.NET framework has a garbage collection mechanism which, in most cases, frees developers from worrying about memory management in the application. In most cases, until application performance becomes an issue. This relates to how the garbage collection works in .NET. To tell a long story short, we can say that for garbage collection runtime stops application functioning (running threads) and continue only when garbage collection is completed. More actively application allocates objects in memory, more often GC is happening, and more time application is blocked. Time of GC work can be usually measured in percentages from the entire application worktime. What values should be considered normal ? It is hard to say, but most of the sources agree that if GC takes 20 percent of your application worktime, you definitely have some troubles :)</p><blockquote><strong><em>Allocation is cheap… until it is not <br></em></strong>Konrad Kokosa</blockquote><p>We can distinguish two approaches for optimizing GC work. First — rewrite application code with focus on low memory allocation using such techniques as object pooling, copy-free methods for working with arrays using <strong><em>`Span`</em></strong> or <strong><em>`Memory`</em></strong>, using structs instead of objects whenever it is possible, avoiding boxing/unboxing etc. This approach means a lot of developer effort in rewriting the entire application code, which definitely does not match the topic of this article.</p><p>The second approach — configure GC for more optimal work. We have two modes of GC work: workstation and server. I will try to tell the long story short, but if you want to get a full and correct picture, please visit next links: <a href="https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/workstation-server-gc">link 1</a>, <a href="https://devblogs.microsoft.com/premier-developer/understanding-different-gc-modes-with-concurrency-visualizer/">link 2</a>. In workstation mode there is one priority thread which is doing garbage collection in a single heap. In server mode for each logical CPU (more details on <a href="https://unix.stackexchange.com/questions/88283/so-what-are-logical-cpu-cores-as-opposed-to-physical-cpu-cores">logical CPU</a>) you have a heap and distinct priority thread doing garbage collection. This means that with the same entire heap size in server mode garbage collection should be performed faster. Also, in server mode, heap usually takes more memory, which makes collecting garbage less intensive than in workstation mode, so we sacrifice application low memory consumption for performance.</p><p>To enable server GC mode you need to add next value in the project configuration:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/53195c6a9f20e57bbfbe7278e56d3cd7/href">https://medium.com/media/53195c6a9f20e57bbfbe7278e56d3cd7/href</a></iframe><p>Worth noting that there is a peculiarity in GC mode configuration. <a href="https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/workstation-server-gc">Documentation</a> says that for standalone applications default GC mode is workstation, but for hosted applications (ASP NET Core is a hosted application) GC mode is defined by host settings. Unfortunately, I did not find more detailed info about when the host defines GC mode as workstation and when as server. While creating demo ASP NET Core applications for this article GC mode was automatically defined as server. You can see what mode is active by using <strong><em>`System.Runtime.GCSettings.IsServerGC`</em></strong> property in runtime, but I recommend setting GC mode explicitly in configuration.</p><p>For testing we will need <a href="https://github.com/MrPomidor/ASPNetPerfImprovementsDemo/tree/master/Solution/GCModeComparison">two identical API projects</a> on NET 6 which will differ only in <strong><em>`ServerGarbageCollection`</em></strong> setting value. Let’s try it out with peak load.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*RRXAE9u3mKHK9SXkkCZE6A.png" /><figcaption>GC modes comparison</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/68630353fa93592c618a8356c33f036a/href">https://medium.com/media/68630353fa93592c618a8356c33f036a/href</a></iframe><p>According to test results, server mode is <strong>51</strong> percent more performant than workstation, so we can state — if your web application for some reason is working in workstation mode, server mode will help dramatically increase your application performance with minimum efforts.</p><h3>Recommendation 4: Use async/await</h3><p>In ASP NET Core application one thread could handle one incoming request from the user. The amount of threads in the pool is limited. If the execution thread faces synchronous I/O bound operation, it will just wait, wasting CPU resources. Thread in this situation cannot do any useful work until the operation is completed. If while handling multiple requests most threads will be blocked the same way, application throughput and scalability will be limited. The recommendation in this case is simple — use asynchronous method overloads with async/await. The thread, instead of waiting, will return to the thread pool ready to do business useful job, and your application could handle more requests.</p><p>This recommendation, unlike previous ones, means multiple code changes in your application, from API controller method signature to the final point of calling DB using, for example, Entity Framework Core. Development efforts for this kind of refactoring could be significant, including integration testing for reducing regressions. Why are we considering this recommendation in terms of this article ?</p><p>Firstly, translating code to async/await is quite simple from my point of view. Translating the call chain to using async/await even could be performed automatically using Roslyn analyzers and code fixes (such analyzers are described in next articles: <a href="https://www.meziantou.net/enforcing-asynchronous-code-good-practices-using-a-roslyn-analyzer.htm">article 1</a> and <a href="https://cezarypiatek.github.io/post/async-analyzers-summary/">article 2</a>). To predict the required effort it is enough to run analyzers setting <strong><em>`severity = error`</em></strong> for async-related rules. This kind of refactoring with good preparations and good test coverage could be performed quite fast and safely.</p><p>Secondly, this recommendation is still actual. It is not a secret that there exists a lot of projects written a long time ago using the old framework and library versions. This kind of project is still supported. Support tasks for this kind of project also could include increasing responsiveness and performance of the application. During the last two years, I have faced three similar projects where we did such kind of refactoring, also for the purpose of improving application performance.</p><p>Before making this decision, it will be helpful for the team lead to know what effect it can potentially have on application throughput. To get this data let’s run a peak load test for <a href="https://github.com/MrPomidor/ASPNetPerfImprovementsDemo/tree/master/Solution/SyncAsyncComparison">two versions of API</a>, which consists of two methods: first for returning entity by id, second will be more heavy and returning page of one hundred entities. The first API version will be fully synchronous, and the second will be fully asynchronous.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/807/1*-9beo94Zf9QW2MraZzFpuw.png" /><figcaption>Sync vs Async comparison</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/aacef176356a6968106faef4ab8bb0f1/href">https://medium.com/media/aacef176356a6968106faef4ab8bb0f1/href</a></iframe><p>According to test results, we can see that the entire application throughput was increased by <strong>58</strong> percent, so the recommendation to rewrite application to async/await can be considered viable. But seeing the results, we must pay attention to the method which returns page results. According to NBomber report, its performance became <strong>11</strong> percent lower. By running another test for this method only, we can confirm performance degradation while switching to async/await:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1e7bad260c3bb5510ec4022ce750bcbd/href">https://medium.com/media/1e7bad260c3bb5510ec4022ce750bcbd/href</a></iframe><p>This can teach us that even changes, effect from which seems predictable could lead to unexpected results. It is important to point out that any optimization results should be confirmed by means of load tests and measuring.</p><h3>Recommendation 5: Manually configure threads in the thread pool</h3><p>While testing synchronous API from previous recommendation we also collected some threads-related stats using <strong><em>`dotnet counters`</em></strong>:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d7ad092cc4fa3e9fa8d526757e9acd29/href">https://medium.com/media/d7ad092cc4fa3e9fa8d526757e9acd29/href</a></iframe><p>As we can see, if all operations are synchronous and there are not enough threads in the pool, work starts piling up in the thread pool queue. Thread pool have the ability to extend under some circumstances, and it is highly possible that thread pool will adapt to this load, but it will take some time. Can we somehow affect this situation and set required threads count manually ? Yes, we can, with the help of <strong><em>`ThreadPool`</em></strong> class.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/de542161068c68c443c1e8b0aea4179c/href">https://medium.com/media/de542161068c68c443c1e8b0aea4179c/href</a></iframe><p>Worth noting that manual configuration of thread pool size can affect application performance either positively or negatively. By default thread pool automatically manages threads count using various metrics, such as CPU cores count, amount of work in the queue etc. Automatic management should find a balance when threads count is enough to manage application load but not greater to avoid context switching overhead (<a href="https://docs.microsoft.com/en-us/dotnet/api/system.threading.threadpool.setminthreads?view=net-6.0#remarks">documentation</a>).</p><p>As a starting point, we can take threads count involved in synchronous API testing (<strong>34</strong>) and try to increase this amount by adding thread pool queue length from the same test (<strong>50</strong>). Round result number to <strong>70</strong> and perform a test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/1*LHR06m4YBqEoyNXqk28lVw.png" /><figcaption>Sync vs Async with more threads comparison</figcaption></figure><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6208685d9530b0c78a6e187f96134111/href">https://medium.com/media/6208685d9530b0c78a6e187f96134111/href</a></iframe><p>As a result of manual threads count management, synchronous API performance closely approaches metrics for asynchronous API. We can conclude that in some cases, manual intervention in threads management can help increase application performance without any significant code changes. This approach could be useful when you have no ability to perform async/await refactoring. But you need to use it with great caution after testing in an environment which is as close to production as possible.</p><h3>Summary</h3><p>In this article we have considered several recommendations for improving performance of ASP NET Core applications. We have tested them and collected data on the impact of each recommendation on application performance. This data should help developers and team leads in making decisions about applying these recommendations to their projects. Source code for all demos used in this article, code for load test scenarios and more detailed testing reports are available on <a href="https://github.com/MrPomidor/ASPNetPerfImprovementsDemo">GitHub</a>.</p><p>Hope this article was useful for you. In the next article I will consider analyzing latest releases of Entity Framework Core from a performance perspective.</p><p>Thank you for your attention !</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=82c564bdf092" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Nbomber as an alternative to JMeter for .NET developer]]></title>
            <link>https://medium.com/@anton_shyrokykh/nbomber-as-an-alternative-to-jmeter-for-net-developer-432040b91763?source=rss-f0dfebbb7a7f------2</link>
            <guid isPermaLink="false">https://medium.com/p/432040b91763</guid>
            <category><![CDATA[dotnet-core]]></category>
            <category><![CDATA[nbomber]]></category>
            <category><![CDATA[load-testing]]></category>
            <category><![CDATA[asp-net-web-api]]></category>
            <category><![CDATA[jmeter]]></category>
            <dc:creator><![CDATA[Anton Shyrokykh]]></dc:creator>
            <pubDate>Thu, 07 Jul 2022 20:10:29 GMT</pubDate>
            <atom:updated>2022-07-07T20:10:29.597Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Nbomber as an alternative to JMeter for .NET developer</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*djAq3WIsxAExbek7zWSpiQ.jpeg" /></figure><p>Like many developers, which faced load testing first the time, I decided to start with JMeter. For familiarization purposes and small scenarios, JMeter completely satisfied me. But when tasks became more complicated, and I needed more control over scenario configuration, I started thinking about searching for a more handy alternatives. Especially wishing for a new tool to be easily adaptable to .NET ecosystem.</p><p>In this article, we will consider NBomber as an easy-to-learn and use alternative to JMeter and try to answer the question “Why should I consider NBomber for using as a load testing tool ?”.</p><p>This article could be useful for .NET developers willing to expand their toolkit with a handy tool for load testing and those who are considering alternatives to JMeter in .NET ecosystem.</p><h3>JMeter is the most popular load testing tool</h3><p>JMeter, according to Google statistics, is the most popular load testing tool. It has a graphical interface as well as a console one, rich functionality, and is able to carry out almost any task. During long years of active use of JMeter, a large community and a lot of related materials and guides have appeared, where you can find answers to almost any question. This is in theory, but what in practice ?</p><p>I had to apply JMeter for simple scenarios on both commercial and pet projects. To be able to perform simple scenarios, it is enough to get familiar with several video guides on JMeter’s user interface. After getting some practice, you can write simple scenarios and perform one-time load testing tasks for HTTP services easily. But as it usually happens, while tasks get more complicated, some issues can arise.</p><h3>What makes me think of an alternative ?</h3><p>I’m sure that JMeter is able to solve almost all possible tasks which can appear during load testing. The question is: how much time you can spend on learning and how usage comfort is important for you. I would like to immediately point out that all of the following regarding JMeter is extremely subjective and can easily be attributed to the laziness of the author :)</p><h4>User interface</h4><p>Wanna start with JMeter user interface, which from the very beginning could be considered a plus. If easy scenarios could be solved in several clicks, accessing more advanced functionality will likely require some time to search for learning materials. If JMeter is not the tool you use every day, it is highly possible that you will repeat this process in some time.</p><p>For solving some tasks, for example, authorization with JWT token, you need to do some sort of <a href="https://www.redline13.com/blog/2018/05/test-rest-apis-authentication-using-jmeter/">UI programming</a>. First, fill up the variable declaration form, then do the same with form for fetching value from “auth/token” response, then you need to fill-up the form for the next request etc.</p><h4>Dependency on other tools</h4><p>Sometimes complicated and unfamiliar UI can encourage you to find a simpler solution using more familiar tools. For example, JMeter has built-in functionality to fill up your requests with random data. More likely that you can satisfy your needs using this functionality. But for me, as .NET developer, it is more comfortable to solve such programming tasks using C#. That’s why I prefer to write data-generation code on .NET, then export data to CSV and then reuse it with JMeter data source functionality, which is much easier. This way JMeter sometimes is not a self-sufficient tool for me.</p><h4>Code reusing and working in a team</h4><p>After completing a task once, you will probably want to save JMeter scenario in a version control system for further reusing in the same project and for opening it to other developers. Also, version control system allows you to track what changes are being made to the scenario by your teammates. Sounds good, isn’t it ? But in practice JMeter uses XML for storing scenarios and hardly anyone wants to read it. It will be complicated to determine the contents of the scenario and what changes your colleague wants to make to it until you load this scenario locally and open it in JMeter. It limits the usefulness of saving JMeter scenarios in a version control system.</p><h4>Continuous integration and JMeter</h4><p>Working on my pet project I’ve created a load testing scenario and tried to add the possibility of running it in terms of nightly testing scope. Sounds possible because JMeter has a console interface ! But as for me, it is not much prettier than UI and the desire to continue studying was lost for a while:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/533/1*NVNHlqJtbVK2HRSRTfWj_Q.png" /><figcaption>JMeter console interface</figcaption></figure><h3>What is NBomber ?</h3><p>Before we start considering how NBomber will handle similar use-cases, let’s do a short introduction.</p><p>NBomber is an open-source F#/C# libriary, available via NuGet, which contains a set of abstractions and utilities for writing load test scenarios in C#. NBomber handles threads, load management, and collecting statistics, so you can concentrate on the test scenario itself. At its core, NBomber is somewhat similar to BenchmarkDotNet for full-fledged load testing.</p><p>After getting familiar with <a href="https://nbomber.com/docs/overview">documenration</a> let’s white some easy script to demonstrate an approach different from JMeter. Tasks will be similar to what we usually want using JMeter:</p><ul><li>configuring amount of users accessing HTTP endpoint</li><li>getting rich statistics in average response processing time, requests per second, errors percent etc.</li><li>getting reports in different formats, including graphs and diagrams</li></ul><p>Code:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d3e4dcbfb0eceecf9da01d759aecac68/href">https://medium.com/media/d3e4dcbfb0eceecf9da01d759aecac68/href</a></iframe><p>As for me, the code is quite brief and clear. You can store it in <a href="https://github.com/MrPomidor/NBomberDemo/blob/master/Solution/Cases.CallDateNow/Program.cs">version control system</a> and anyone can read it, which is an advantage itself. Let’s review the output format to see our execution results, graphs and statistics. Running application in console, NBomber will provide well-formatted execution statistics in the form of a table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*yldwxakWE1YTV7E7i1JTgQ.png" /><figcaption>Console stats</figcaption></figure><p>After choosing HTML report format, we will receive html page in “reports” folder, which contains well-looking and detailed stats for each scenario and different graphs, making information easier to understand:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hiiEzWyUhJR803vxZsGj0w.png" /><figcaption>Html report grahps</figcaption></figure><p>At this stage, NBomber can be considered as an alternative for JMeter for some easy tasks. But let’s consider trying something which is far more complicated to archive using JMeter.</p><h3>NBomber as an alternative for JMeter</h3><p>Let’s consider three tasks which are likely to appear while load testing HTTP API.</p><h4>Request with variable parameters count</h4><p><strong>Task</strong>: you need to test HTTP endpoint, which accepts array of values in query string in format <strong><em>“?myParameter=value1&amp;myParameter=value2”</em></strong>. Endpoint did not accept empty strings as values. Your CSV file consists of 3 columns, which means each data row could contain up to 3 values.</p><p>To solve this with JMeter you will first try to use <strong><em>“Parameters”</em></strong> tab in <strong><em>“HTTP Request”</em></strong> element. Define 3 parameters, each of them will read values from CSV column.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/707/1*YQE_7N4hYEQ8mTpX9NC0rw.png" /><figcaption>JMeter. Configure query parameters</figcaption></figure><p>Running test and …. get BadRequest.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2d6fe07fa6a6f7edfc4694039a817c33/href">https://medium.com/media/2d6fe07fa6a6f7edfc4694039a817c33/href</a></iframe><p>The problem is that<strong><em> “HTTP Request”</em></strong> element when receiving empty value for parameter preserves it and passes to our API:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/658/1*POOOC8MJDyQhjR_ima-Xag.png" /><figcaption>JMeter. Empty query parameter</figcaption></figure><p>Can we solve this somehow using JMeter ? Of course, we can ! But in some <a href="https://loadium.com/blog/how-to-add-a-variable-number-of-parameters-in-apache-jmeter">different way</a>, for finding which you should spend some time and this way will probably include writing some scripts on Java, Groovy or JavaScript.</p><p>When using NBomber you could solve the problem quite fast by rewriting query string construction logic using C# language, which is more familiar to you as for C# developer:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4dfdb358dab75e56b0c77b73f7e65a93/href">https://medium.com/media/4dfdb358dab75e56b0c77b73f7e65a93/href</a></iframe><h4>Request parameters generaion</h4><p><strong>Task</strong>: you need to test HTTP endpoint, which caches answers for some input values. To solve this issue, you need different parameters for each request.</p><p>JMeter has several <a href="https://jmeter.apache.org/usermanual/functions.html">built-in functions</a> as <strong><em>`Random`</em></strong>, <strong><em>`RandomDate`</em></strong>, <strong><em>`RandomString`</em></strong>, <strong><em>`UUID`</em></strong>. But it could hardly satisfy your needs if you need input data of special format, for example, valid name, email, telephone number, credit card etc. There exist at minimum two possible ways to solve this using JMeter: using JMeter scripting or using an external data source. First requires some knowledge and skills in JMeter, while the second requires using some external data provider service of self-written data generation application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/532/1*BHrxx4euA8W8rLGo0gqpkQ.png" /><figcaption>JMeter. Create random variable</figcaption></figure><p>Using NBomber you can gain several benefits. In addition to using C# language, you are also not limited in using popular C# libraries for test data generation as <a href="https://github.com/bchavez/Bogus">Bogus</a> or <a href="https://github.com/nickdodd79/AutoBogus">AutoBogus</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8d48d65de9164bb7a459cafb760aabeb/href">https://medium.com/media/8d48d65de9164bb7a459cafb760aabeb/href</a></iframe><p>Moreover, you can get benefits which are not obvious at first glance: if you store your NBomber load test in the same repository with your project under test, you can reuse your domain code or test data-generation code from your unit tests. Just add a reference to them to your NBomber load test project.</p><p>Can we solve this task with JMeter ? Yes. Is it convenient ? It’s up to you. Nbomber, in this case, allows you to lower the amount of used tools on your project, also allowing you to reuse familiar libraries and even your application code to solve the task effectively.</p><h4>CI quality gate</h4><p><strong>Task</strong>: you need to add load test scenario to your nightly running CI quality gate pipeline, which will check that deployed application is capable of handling expected RPS with an acceptable average response time. This scenario should work as fail/pass. Assume that on failing some more detailed tests will be running to investigate performance issues.</p><p>Let’s give the answer to the question “Can we solve this taks with JMeter ?”. Yes, we can ! There exist many ways to integrate JMeter into your CI pipeline, and you will probably find one which matches your comfort and available resources (for example <a href="https://schoeffm.github.io/posts/integration-of-loadtests-within-jenkins-pipelines/">here</a> or <a href="https://www.jenkins.io/doc/book/using/using-jmeter-with-jenkins/">here</a>). Let’s see if NBomber provides a more convenient way to solve this task.</p><p>Consider code from our earlier example with testing <strong><em>“/datenow”</em></strong> endpoint. Method <strong><em>`NBomber.Run`</em></strong> returns an object, which contains statistics of your load test run. This data is used to build reports in the console and in other formats. Having direct access to this data, you can easily use statistics of steps you are interested in (for example authorization steps can be skipped) to build a simple assert. If test conditions are met — console application will return 0, which means the test is passed. If conditions are not met — console application will return -1, which means the test failed.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0612afd196d5f26e6021bb299586521a/href">https://medium.com/media/0612afd196d5f26e6021bb299586521a/href</a></iframe><p>You just need to compile your console application and add it as a step to your CI pipeline. For this particular case, for me, it is easier to use NBomber.</p><h3>NBomber as a part of .NET developer toolkit</h3><h4>Run load testing scenarios from IDE</h4><p>The above examples of solving typical load testing tasks give us an idea of NBomber as a standalone load testing tool. But what if you want to integrate it more closely to your development process and run it not only as a standalone app but as part of your integration tests scope with Visual Studio or Rider ?</p><p>For writing tests you need arrange, act and assert parts. <strong><em>`NBomber.Run`</em></strong> is supposed to be used as act, and its return value is needed to write assert. But how should we write arrange to test our application without deployment ?</p><p>ASP NET gives us a convenient way of API integration testing using <strong><em>`WebApplicationFactory`</em></strong> class from <strong><em>`Microsoft.AspNetCore.Mvc.Testing` </em></strong>NuGet package. This class by using <strong><em>`Program.cs`</em></strong> creates an instance of our application in memory, so you can call it by creating <strong><em>`HttpClient`</em></strong> with <strong><em>`CreateClient()`</em></strong> method. To transform our standalone NBomber scenario with <strong><em>“/datenow”</em></strong> endpoint into xUnit integration test, we just need to substitute<strong><em> `HttpClient`</em></strong> used in the test by instance returned by `WebApplicationFactory`.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0c60667bcb4889ce6d696fcb2dae903c/href">https://medium.com/media/0c60667bcb4889ce6d696fcb2dae903c/href</a></iframe><p>Worth noting some important points regarding using this testing approach. First of all you should consider that the results of this test will fully depend on the machine on which it will run. From developer to developer you could have quite different configurations, and CI machines could be less powerful as well. Secondly, this test, of course, will not include possible network latency, as it calls your application in-memory, so results will differ from the standalone scenario. This limits using of this approach, but as for me it should have a bit different purpose.</p><p>Firstly this test will be useful as a performance estimation of some endpoint before its release. We don’t need to be very accurate when doing estimations, so it will be enough to say “this endpoint probably could handle 1000 requests per second with 200 milliseconds average response time”.</p><p>Secondly, when having this test you have the ability to receive fast feedback doing refactoring or some performance optimizations. Before doing any code changes run the performance test, record initial results, then do changes and run the test again, to confirm performance metrics were improved or at least not degraded before deploying code to the test environment.</p><h4>Run load testing scope via dotnet cli</h4><p>Can we adapt the approach described above for running full-fledged load tests with<strong><em> `dotnet test`</em></strong> command ? Yes, we can ! For this we need to substitute the call of an in-memory application with calling the real one using <strong><em>`HttpClient`</em></strong> and provide the required configuration via <strong><em>`appsetting.json`</em></strong>. Code changes in the test scenario are minimum:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1f80d21aea57eb8647ad9610bbe84d46/href">https://medium.com/media/1f80d21aea57eb8647ad9610bbe84d46/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GIeR5a7jKdyZtI2drxiJcw.png" /><figcaption>NBomber with dotnet test</figcaption></figure><h3>Summary</h3><p>In this article I wanted to represent NBomber for those who haven’t dealt with it yet, also showing it in action for common load testing scenarios in comparing with the most popular tool for now — JMeter. We have considered more close integration of NBomber in C# application development process and some scenarios, for example running tests from Visual Studio, for which JMeter will not match your needs. All code and examples is accessible on <a href="https://github.com/MrPomidor/NBomberDemo">GitHub by this link</a>. Also worth noting that code represented in the article is not 100% correct from NBomber documentation point of view. For example I intentionally have not used <strong><em>`DataFeed`</em></strong> or <strong><em>`ClientFactory`</em></strong> to make code as clear as possible. I recommend you to get familiar with <a href="https://nbomber.com/">NBomber documentation</a> to know about all its features.</p><p>As for me, NBomber has a chance to take place in your professional toolkit as easy to use and convenient load testing tool. It can be easily adapted in your C# development process, also opening possibilities of code and library reusing, can be integrated into CI and can for most common scenarios substitute JMeter. Hope this article was useful for you.</p><p>Thank you for your attention !</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=432040b91763" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>