<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vidhi Thakrar on Medium]]></title>
        <description><![CDATA[Stories by Vidhi Thakrar on Medium]]></description>
        <link>https://medium.com/@vidhi.thakrar?source=rss-c635f11ee4a5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*m3FHMOLGPwfzSqt_fEIgPQ.jpeg</url>
            <title>Stories by Vidhi Thakrar on Medium</title>
            <link>https://medium.com/@vidhi.thakrar?source=rss-c635f11ee4a5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 11:39:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vidhi.thakrar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Akka Http Client API: Request-level vs Host-level]]></title>
            <link>https://medium.com/@vidhi.thakrar/akka-http-client-api-request-level-vs-host-level-8bcf5da799e1?source=rss-c635f11ee4a5------2</link>
            <guid isPermaLink="false">https://medium.com/p/8bcf5da799e1</guid>
            <category><![CDATA[scala]]></category>
            <category><![CDATA[akka-http]]></category>
            <dc:creator><![CDATA[Vidhi Thakrar]]></dc:creator>
            <pubDate>Mon, 03 Jun 2019 02:25:00 GMT</pubDate>
            <atom:updated>2019-06-03T02:33:01.057Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://doc.akka.io/docs/akka-http/current/client-side/index.html">Akka Http Client</a> provides three different levels of APIs to consume HTTP services:</p><ol><li>Request-level API</li></ol><ul><li>High-level, easy to use abstraction</li><li>Manages all the connections itself</li><li>Builds upon Host-level API</li></ul><p>2. Host-level API</p><ul><li>Stream based API</li><li>Gives control to configure connection pool per Host(&lt;host:port&gt;)</li><li>Builds upon Connection-level API</li></ul><p>3. Connection-level API</p><ul><li>Low-level abstraction</li><li>Gives control to manage connections and how to schedule requests across them</li></ul><p>As Akka mentioned, Request-level API typically works for most of the use cases. Even for us, it was working fine in an aggregated HTTP service until recently when we had to explore Host-level API to make many parallel requests to underlying HTTP service.</p><p>Going through the Akka documentation, it seems Request-level and Host-level APIs behave the same with the default connection pool configuration. However, that’s not the case and it was an interesting learning to see the difference in certain situations.</p><p>In this article, Let’s try to mimic the situation we faced and compare Request-level and Host-level APIs to decide which one to use when.</p><p><em>Note: Connection level API is explicitly discarded in comparison as it’s pretty low-level abstraction to be used in a very specific use case where you need to manage connections on your own.</em></p><h3>Use case</h3><p>Let’s say we want to build a backend service for the financial web application that helps to monitor financial portfolio of a user. For simplicity, let’s focus on only implementing /transactions endpoint which will return all the transactions associated with the requested account ids. Let’s call it financial-portfolio-service.</p><p>Assume that there is an HTTP service called financial-data-service that manages financial data and has exposed /accounts/{id}/transactions endpoint to return all the transactions associated with the requested account id.</p><p>As /transactions endpoint accepts multiple account ids, we need to make parallel calls to /accounts/{id}/transactions endpoint for each account id as displayed below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/804/1*7s_wHjEo7mRAvU2RXKbbbw.png" /><figcaption>Component Interaction</figcaption></figure><p>To make the comparison, Let’s implement two aggregated services exposing /transactions endpoint, one which is internally using Request-level API and another using Host-level API.</p><p>Below are the code snippets to make parallel calls to financial-data-service:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fdc6ed85aa33fe17c1a8bb87f00e7cbd/href">https://medium.com/media/fdc6ed85aa33fe17c1a8bb87f00e7cbd/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/405817482f0e740df4f442216edc589b/href">https://medium.com/media/405817482f0e740df4f442216edc589b/href</a></iframe><h4>WebServer Implementation with Request-level client API:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fa72aba364ee389c1c91d948dd283715/href">https://medium.com/media/fa72aba364ee389c1c91d948dd283715/href</a></iframe><h4>WebServer Implementation with Host-level client API:</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7cd664e83ae431cefdfa344ef103507d/href">https://medium.com/media/7cd664e83ae431cefdfa344ef103507d/href</a></iframe><p>As you can see here, Request-level API is a high-level abstraction and easy to write compare to Host-level API.</p><p>Now, we will run multiple tests on these two services to make the comparison:</p><p><em>Note: Underlying </em><em>financial-data-service is simulated to perform slow operation and it is returning response in two seconds so that we can clearly see how many parallel requests are going at a time.</em></p><h4>#Test1: A user with a single account</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a95f9c8101b82df5a04fec326b5e178a/href">https://medium.com/media/a95f9c8101b82df5a04fec326b5e178a/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/abfeb99d5dccf17aa89ab1d5b2c03469/href">https://medium.com/media/abfeb99d5dccf17aa89ab1d5b2c03469/href</a></iframe><p>No difference as of now. Both are getting responses after two seconds.</p><h4>#Test2: A user with ten accounts</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/685f15440b7b53f12416aaebec550dff/href">https://medium.com/media/685f15440b7b53f12416aaebec550dff/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bc21c2c6850c2747789222654164c1ff/href">https://medium.com/media/bc21c2c6850c2747789222654164c1ff/href</a></iframe><p>Both are completing the request in almost same time around in 6 seconds.</p><p><strong>Request-level: </strong>It looks like all the ten requests are going at the same time, but if you notice, only 4 responses are arriving at a time.</p><p><strong>Host-level: </strong>It looks like only 4 requests are going at a time and as responses arrive more requests are going but at a time only 4 requests are being processed.</p><p>To understand this behaviour, Let’s see what Akka Http Client does underneath:</p><p>Akka Http Client has two main components which handle how many requests can go in parallel to a particular host:</p><ol><li><strong>Connection pool</strong></li><li><strong>Fixed size queue of requests</strong></li></ol><p>Akka http client puts the requests in a queue and then based on the availability of connections in a connection pool, it pulls the requests out from the queue and starts processing them.</p><p>There are two main <a href="https://doc.akka.io/docs/akka-http/current/configuration.html">configurations</a> on these components which play a critical part here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C4xffx6NAtCyzXYTjaVqHA.png" /></figure><p>As max-connections has default value as 4, only 4 requests are being processed in parallel at a time.</p><h4>#Test3: A user with fifty accounts</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/765f2a6cfe46376e16f9bd3a61ea7193/href">https://medium.com/media/765f2a6cfe46376e16f9bd3a61ea7193/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7052d771d3aa242386a55628672ab828/href">https://medium.com/media/7052d771d3aa242386a55628672ab828/href</a></iframe><p><strong>Request-level: </strong>It throws BufferOverflowException because it puts 50 requests in a fixed size queue having size of max-open-requests and default value for max-open-requests is 32.</p><p><strong>Host-level: </strong>It is able to process 50 requests though it is also using fixed size queue with max-open-requests size underneath. The difference is this stream based API is applying back-pressure and only max-connections number of requests are going in a queue at a time instead of all the 50 requests together as displayed below:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/63ed1434314cd4e2f9c6a75a5e8c3f6d/href">https://medium.com/media/63ed1434314cd4e2f9c6a75a5e8c3f6d/href</a></iframe><p><em>Note: Akka http client does not support pipelining. Default value of </em><em>pipelining-limit is 1 and hence in the above code, only </em>max-connections<em> number of requests are being processed at a time.</em></p><p>So far, Host-level API looks like satisfying all the test scenarios. But we have only run all the tests considering a single user. In reality, There will be multiple users using the service concurrently. Let’s see what happens when we run the the test for multiple users:</p><h4>#Test4: Fifty users with fifty accounts</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fd9d41165864bf13e2c4a51d72994488/href">https://medium.com/media/fd9d41165864bf13e2c4a51d72994488/href</a></iframe><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/92c829ef602b549e4271ae62bd170ccf/href">https://medium.com/media/92c829ef602b549e4271ae62bd170ccf/href</a></iframe><p><strong>Request-level: </strong>As expected, it is throwing BufferOverflowException for all the users because it puts 50 requests in a queue for each user.</p><p><strong>Host-level: </strong>Now, host-level is also throwing BufferOverflowException even though it is applying back-pressure. If you notice, it is applying back-pressure for each stream materialization and not across the materializations. In this test, there will be 50 materializations and each will put 4 requests to a queue, and queue will be overflown.</p><p>With the default configuration, Host-level can roughly handle only 8 (max-open-requests/max-connections) concurrent users at a time, given that for each user there are more than or equal to “max-connection” number of requests going in parallel.</p><h3><strong>Summary</strong></h3><p>As you can see from the tests’ output, There is no simple and straight forward answer to use a particular API-level that will work in all the scenarios. However, following guidelines can be useful to decide which API-level will work best in a given situation:</p><ul><li><strong>Connection pool configuration per Host</strong></li></ul><p>Only Host-level API can allow you to configure different connection pool configuration per host, So it’s an obvious choice.</p><ul><li><strong>No Parallel requests to fan out</strong></li></ul><p>Both the API-levels will behave the same in this situation. However, as Request-level is an easy to use, it is a better choice.</p><ul><li><strong>Parallel requests to fan out, SLA is not a big concern</strong></li></ul><p>If there is a huge number of parallel requests to fan out and you don’t want to bombard the underlying HTTP service with too many requests then go ahead with the Host-level API.</p><p><strong>You may wonder, What if we increase the </strong>max-open-requests<strong> and use Request-level API?</strong></p><p>Well, If you do that then you will not face BufferOverflowException. However, there is a major loophole in Request-level where users can be starved. Consider a case with the default connection pool configuration, where there are two users making a /transactions requests having a large number of account ids with a gap of few milliseconds. Requests level will put all the requests of the first user in a queue and then the second user has to wait till all the requests of first user is completed. For the same case, Host-level will only put max-connections number of requests in a queue for both the users and hence both the users will almost take a similar time.</p><ul><li><strong>Parallel requests to fan out, scaling and SLA is a concern</strong></li></ul><p>The only way to achieve this is to increase max-connections and if you do that, then Request-level and Host-level will behave the same.</p><h3>References</h3><ul><li><a href="https://www.gregbeech.com/2018/04/08/akka-http-client-pooling-and-parallelism/">https://www.gregbeech.com/2018/04/08/akka-http-client-pooling-and-parallelism/</a></li><li><a href="https://doc.akka.io/docs/akka-http/current/client-side/index.html">https://doc.akka.io/docs/akka-http/current/client-side/index.html</a></li><li>The whole codebase can be found at <a href="https://github.com/vidhithakrar/akka-http-client-api-comparison">https://github.com/vidhithakrar/akka-http-client-api-comparison</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8bcf5da799e1" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>