<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Philip Fulgham on Medium]]></title>
        <description><![CDATA[Stories by Philip Fulgham on Medium]]></description>
        <link>https://medium.com/@philip_fulgham?source=rss-f590e3d45f8b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*mCmW0cI8f4DRizh7UTb-ig.jpeg</url>
            <title>Stories by Philip Fulgham on Medium</title>
            <link>https://medium.com/@philip_fulgham?source=rss-f590e3d45f8b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 00:21:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@philip_fulgham/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Enumerate All the Things]]></title>
            <link>https://medium.com/carmax-engineering-blog/enumerate-all-the-things-cb49618542b5?source=rss-f590e3d45f8b------2</link>
            <guid isPermaLink="false">https://medium.com/p/cb49618542b5</guid>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[c-sharp-programming]]></category>
            <dc:creator><![CDATA[Philip Fulgham]]></dc:creator>
            <pubDate>Wed, 31 Jul 2024 14:07:14 GMT</pubDate>
            <atom:updated>2024-07-31T15:53:33.990Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2LB7h2L84d5cNoTMArH0zQ.jpeg" /></figure><p>I’ve been writing .NET applications in C# for well over a decade, during which the language has evolved and advanced considerably. Recently, I’ve had cause to deep-dive into a fundamental API that’s been around for even longer: the IEnumerable&lt;T&gt; interface, as well as its much-newer cousin, the IAsyncEnumerable&lt;T&gt; interface. These interfaces are commonly used as one-size-fits-all abstractions even in very simple code, but they have some impressive capabilities you can unlock when you understand the principles they follow. In this post, I’m going to walk through several different ways you can use these interfaces to keep your code flexible and lightweight.</p><h4>IEnumerable&lt;T&gt; for Collections</h4><p>I don’t have data on this, but if you asked me to guess what the most common use of an IEnumerable&lt;T&gt; is, it would be “an abstraction of a collection.” IEnumerable&lt;T&gt; lacks certain common APIs you’d often want in a collection, most notably a stored count of the items. But it’s <em>very</em> flexible when all you need is to iterate over a sequence of items.</p><p>Thus, it’s very useful for the API design practice of not requiring an input to be a more derived type than you actually need. An IEnumerable&lt;T&gt; parameter in a public method can accept a T[], a List&lt;T&gt;, a HashSet&lt;T&gt;, an ICollection&lt;T&gt;, or many other collection types in .NET (to say nothing of custom types), without being unnecessarily opinionated about the API consumer’s collection handling. It doesn’t even technically require the parameter to be a stored collection at all, as we’ll soon see.</p><h4>IEnumerable&lt;T&gt; for LINQ</h4><p>You’re probably aware that theSystem.Linq namespace includes a ton of extension methods for IEnumerable&lt;T&gt; that allow you to filter, project, aggregate, sort, and otherwise manipulate the enumerated results without changing the original enumerable. Sometimes the behavior of these methods is referred to as “deferred execution,” or “lazy,” because the operation isn’t actually performed until you enumerate the enumerable.</p><p>Terms like “deferred” or “lazy” sometimes mean that code won’t be executed until you need it, but then once it’s been executed, the result will be persisted in some fashion to optimize subsequent access. For example, a singleton may not be constructed until the first time it’s needed, but then it may remain available for the lifetime of the application without being constructed again. This is a key principle for setting up dependency injection in ASP.NET Core. However, it isn’t an accurate description of how the LINQ extension methods work.</p><p>When you use a LINQ extension method like IEnumerable&lt;T&gt;.Select(), the operation is performed on the items at the time they are enumerated. But, actually, it’s performed on the items <em>every time</em> they are enumerated. This can be proven very easily:</p><pre>IEnumerable&lt;string&gt; stuffs = [&quot;foo&quot;, &quot;bar&quot;, &quot;baz&quot;];<br><br>IEnumerable&lt;string&gt; uppercaseStuffs = stuffs.Select(s =&gt;<br>{<br>    Console.WriteLine(&quot;Stop shouting!&quot;);<br>    return s.ToUpper();<br>});<br><br>uppercaseStuffs.ToList();<br>uppercaseStuffs.ToList();</pre><p>Here, we’re using IEnumerable&lt;T&gt;.Select() (a projection method) to uppercase our strings, but we’ve also snuck in a side effect in the form of a console log (this isn’t production code, I’m just illustrating a point here). When we run this code, that console log prints not three times, but <em>six</em> times: three for the first call of IEnumerable&lt;T&gt;.ToList() (which enumerates the enumerable), and three more for the second call (which enumerates it again). So, the results of that projection aren’t actually being <em>stored</em> anywhere. Every time you enumerate, the projection will be performed again on each item that you enumerate.</p><p>If the operation is trivial performance-wise, this might not matter to you. But imagine that your LINQ lambda (or named method you’re passing in) is doing something expensive. Or imagine that you have a very large number of items with less-expensive but still non-trivial logic, and your code is enumerating over and over again as a result of multiple foreach loops, IEnumerable&lt;T&gt;.FirstOrDefault() calls, and/or other operations. In such situations, we should consider first enumerating into a concrete type like List&lt;T&gt; so that the items aren’t being recalculated every time we iterate over the collection. Those types generally implement IEnumerable&lt;T&gt;, so it’s still possible to work with the results using that interface; it just changes the underlying structure to a stored collection instead of on-demand calculations for each item.</p><h4>IEnumerable&lt;T&gt; for Methods</h4><p>The LINQ extension methods aren’t the only way to enumerate results in a “lazy” fashion; we can also write our own methods that do the same. If you’ve read enough C# code, you may have come across methods that assemble an IEnumerable&lt;T&gt; to return using logic similar to this:</p><pre>public IEnumerable&lt;string&gt; GetThingNames(<br>    IEnumerable&lt;string&gt; foodThings, <br>    IEnumerable&lt;string&gt; widgetThings,<br>    IEnumerable&lt;string&gt; thingamabobThings)<br>{<br>    List&lt;string&gt; thingNames = [];<br>    foreach (string thingName in foodThings)<br>    {<br>        things.Add($&quot;{thingName} Food Thing&quot;);<br>    }<br>    foreach (string thingName in widgetThings)<br>    {<br>        things.Add($&quot;{thingName} Widget Thing&quot;);<br>    }<br>    foreach (string thingName in thingamabobThings)<br>    {<br>        things.Add($&quot;{thingName} Thingamabob Thing&quot;);<br>    }<br>    return thingNames;<br>}</pre><p>Of course, there are ways to refactor this code using List&lt;T&gt;.AddRange(), IEnumerable&lt;T&gt;.Select(), and/or the .. spread operator. But they’ll all work essentially the same way, iterating over the three inputs, computing new results, adding those results to a List&lt;string&gt;, and then returning that list.</p><p>For some use-cases, this is fine. But consider a scenario where the main use-case for this method is to find, say, whether any item in the resulting set is more than 50 characters long. That doesn’t necessarily require enumerating <em>all</em> of the items; once you find one that meets the condition, you can stop enumerating. Imagine the set is very large, or the logic required for each item is more than the simple string interpolation in the example above. Enumerating the entire set of items isn’t a very efficient approach to this problem if the first or second item already meets your criteria. In fact, methods like IEnumerable&lt;T&gt;.Any() are optimized specifically to avoid that behavior when possible, so canceling that out by pre-enumerating the entire set isn’t ideal.</p><p>IEnumerable&lt;T&gt; provides a way to write a method that will enumerate results individually just like those LINQ extension methods. In this case, it also happens to avoid the allocation for a List&lt;string&gt; that we don’t really need. The key is the yield return statement:</p><pre>public IEnumerable&lt;string&gt; GetThingNames(<br>    IEnumerable&lt;string&gt; foodThings, <br>    IEnumerable&lt;string&gt; widgetThings,<br>    IEnumerable&lt;string&gt; thingamabobThings)<br>{<br>    foreach (string thingName in foodThings)<br>    {<br>        yield return $&quot;{thingName} Food Thing&quot;;<br>    }<br>    foreach (string thingName in widgetThings)<br>    {<br>        yield return $&quot;{thingName} Widget Thing&quot;;<br>    }<br>    foreach (string thingName in thingamabobThings)<br>    {<br>        yield return $&quot;{thingName} Thingamabob Thing&quot;;<br>    }<br>}</pre><p>This turns our method into an <a href="https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/iterators">iterator method</a>. It’s important to understand that yield return effectively pauses the execution of the method (in a procedural sense, not a threading sense), handing control back to the caller. It <em>does not</em> run the entire block of code to calculate all the items and then return them as a stored collection like the original method did. This means that if the caller is iterating over the enumerable returned from this method, and stops iterating for whatever reason, such as a break statement or an optimized LINQ extension method like IEnumerable&lt;T&gt;.Any() that doesn’t continue enumerating once it finds what it’s looking for, this method will not proceed to calculate the rest of the items.</p><p>This is obviously a highly simplified example, but it shouldn’t be too hard to imagine more complex scenarios where this could substantially improve performance versus the approach of eagerly calculating all possible items for an enumerable. Of course, as I suggested above, in some cases the eager approach may be better; the concepts in the previous section still apply, and which approach is better will vary based on how you intend to use the results of the method.</p><p>The “lazy” approach favors situations where being able to enumerate only some leading portion of the items will save effort. The “eager” approach favors situations where you want to iterate over the items multiple times. You can also use hybrid approaches, such as writing an iterator method to allow for an initial, efficient partial enumeration, but then calling a method like IEnumerable&lt;T&gt;.ToList() on those partially enumerated results to store them for further logic that will involve multiple passes over the collection.</p><p>There are also scenarios where the performance impact is negligible either way, but using yield return is simpler and cleaner than allocating a List&lt;T&gt; and calling List&lt;T&gt;.Add() a bunch of times.</p><p>One final note for this section: yield return also works in property get accessor bodies when the property type is IEnumerable&lt;T&gt;.</p><h4>IAsyncEnumerable&lt;T&gt; for Async Methods</h4><p>To be honest, when I first encountered IAsyncEnumerable&lt;T&gt; popping up in Microsoft SDKs, it annoyed me. I used the ToListAsync method from <a href="https://www.nuget.org/packages/System.Linq.Async">System.Linq.Async</a> a lot because I didn’t want to deal with this funky new type. But I was only thinking of it as an abstraction of a collection; I wasn’t yet thinking of it in terms of the other concepts we’ve covered. When you realize that IAsyncEnumerable&lt;T&gt; effectively lets you do all of those same things <em>but with async operations involved</em>, it suddenly becomes awesomely powerful, particularly for batch processing with async operations at both ends.</p><p>Let’s revisit the example from before, but this time assume that we need to call an async method to get our final strings, perhaps with some dependency call under the hood:</p><pre>public async IAsyncEnumerable&lt;string&gt; GetThingNamesAsync(<br>    IEnumerable&lt;string&gt; foodThings, <br>    IEnumerable&lt;string&gt; widgetThings,<br>    IEnumerable&lt;string&gt; thingamabobThings)<br>{<br>    foreach (string thingName in foodThings)<br>    {<br>        yield return await GetAsync(thingName, &quot;Food&quot;);<br>    }<br>    foreach (string thingName in widgetThings)<br>    {<br>        yield return await GetAsync(thingName, &quot;Widget&quot;);<br>    }<br>    foreach (string thingName in thingamabobThings)<br>    {<br>        yield return await GetAsync(thingName, &quot;Thingamabob&quot;);<br>    }<br>}</pre><p>(Note: You don’t necessarily have to yield return and await on the same line, we’re just doing so here for simplicity.)</p><p>Even without changing our assumptions about the use-cases for the method, it should now be clear how much we <em>don’t</em> want to process every single item if we don’t actually need them all. All sorts of expensive operations could be inside those GetAsync calls. If we were just using Task&lt;IEnumerable&lt;string&gt;&gt; as the return type for this method, we wouldn’t be able to use yield return, and we’d always have to build each and every possible value, no matter what.</p><p>But let’s consider a different use-case. Let’s say that the results of this method need to be fed into another dependency call, a remote API that requires an HTTP POST for each item we’re adding. With the Task&lt;IEnumerable&lt;string&gt;&gt; approach, we’d be building all of the items, accumulating them in memory, and then sending them all to the remote API either sequentially or in parallel. Depending on the specifics, that might be fine, or it might create a performance bottleneck: maybe making the remote API wait to receive the first item until after you’ve spent 30 seconds to build the rest of the items isn’t desirable for this application. It might prevent partial successes that you’d prefer to accept: for example, if the 24th GetAsync call throws an exception, <em>none</em> of the preceding 23 items will go to the remote API without additional code to handle the error. If the items could be really large and/or numerous, it might create an unnecessarily high memory requirement for the application.</p><p>But with the IAsyncEnumerable return type, we can call the method and send its results to our remote API like this:</p><pre>public async Task ProcessThingsAsync()<br>{<br>    await foreach (string thingName in GetThingNamesAsync(/* ... */))<br>    {<br>        await PostAsync(thingName);<br>    }<br>}</pre><p>This method will process each “thing” end-to-end in sequence, instead of having to do all the “get” operations in one batch followed by all the “post” operations in another batch. With this approach, you can create stacks of methods that stream data sequentially as fast as it can be processed, always keeping it moving, without having to stop and store it for each iteration along the way.</p><p>Do note that, once again, the same “deferred execution” rules apply. If you’re going to be enumerating multiple times, there’s a good chance you don’t want to be making all those async calls every time you do. You probably want to use some combination of stored collections, async enumerables, and regular enumerables to avoid repeating expensive operations.</p><h4>Parallelized Enumeration</h4><p>You may already have noticed that, in some situations, the last solution above won’t scale well. When working with a very large set of items, you might not want to load the whole thing into memory at once, but processing items only one at a time might be agonizingly slow. This is where one of my favorite modern .NET APIs comes in, introduced a few years ago in .NET 6: <a href="https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.parallel.foreachasync?view=net-8.0">Parallel.ForEachAsync()</a>.</p><p>This API makes it easy to parallelize an iteration over an enumerable. If you already have your set of thing names for your remote API as a regular IEnumerable&lt;T&gt;, you can handle them a few at a time:</p><pre>public async Task ProcessThingsAsync()<br>{<br>    IEnumerable&lt;string&gt; thingNames = GetThingNames(/* ... */);<br>    await Parallel.ForEachAsync(thingNames,<br>        new ParallelOptions() { MaxDegreeOfParallelism = 4 },<br>        async (thingName, cancellationToken) =&gt;<br>        {<br>            await PostAsync(thingName);<br>        });<br>}</pre><p>The API also has an overload that accepts IAsyncEnumerable&lt;T&gt; instead of IEnumerable&lt;T&gt;, and that’s where it gets even more useful. That overload will drain multiple items from the async enumerable at a time, up to the degree you specify. It effectively allows the async enumerable to proceed (to a point) with enumerating more items, even if some previously yielded items are still being processed by the caller. Of course, it works this way with regular enumerables too, but the advantage may be less noticeable unless the synchronous operations for each item are really expensive and have variable performance; it really shines with async enumerables.</p><p>Parallel.ForEachAsync() is not a “batch” or “chunk” API. It doesn’t split your set into groups of length N = MaxDegreeOfParallelism. It behaves more like a “throttling” API. If you allow a maximum of four operations at a time, and one of the first four operations takes way longer than the others, a chunking API would wait until that slowest operation has finished before starting the next chunk of four. Parallel.ForEachAsync() will start new operations to replace the three that have finished, even if the fourth is still in progress. It doesn’t “pause” processing new items every time one hangs for a few seconds the way a chunking API would.</p><h4>Conclusion</h4><p>IEnumerable&lt;T&gt; is a very widely used .NET API that is much more powerful than its most simple use-cases reveal. IAsyncEnumerable&lt;T&gt; is rising in popularity and provides a way to leverage the principles of enumerables with async operations, a valuable proposition in an ecosystem increasingly filled with async code. Built-in APIs like LINQ extension methods and Parallel.ForEachAsync(), along with libraries like System.Linq.Async, offer tools that will let you optimize how you use enumerables, provided you understand the rules behind how they work. A key factor in finding the right approach is determining whether your use-case’s performance and logic will benefit more from optimizing for partial iterations (lazy) or optimizing for multiple iterations (eager).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cb49618542b5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/carmax-engineering-blog/enumerate-all-the-things-cb49618542b5">Enumerate All the Things</a> was originally published in <a href="https://medium.com/carmax-engineering-blog">CarMax Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Very Helpful Service]]></title>
            <link>https://medium.com/carmax-engineering-blog/a-very-helpful-service-132c5e7b64da?source=rss-f590e3d45f8b------2</link>
            <guid isPermaLink="false">https://medium.com/p/132c5e7b64da</guid>
            <category><![CDATA[product-development]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[azure-logic-apps]]></category>
            <dc:creator><![CDATA[Philip Fulgham]]></dc:creator>
            <pubDate>Thu, 30 May 2024 13:16:50 GMT</pubDate>
            <atom:updated>2024-05-30T13:16:50.893Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5p7OG-hcUVp0ZxYOaaHtZg.jpeg" /></figure><p>This is the story of an innovative little product team and our quest for a new way of managing background tasks for a business-critical service. For six years, our service filled this need with Azure Functions, Microsoft’s serverless platform for .NET and other languages. That solution endured through our late-2019 upgrade from .NET Framework to what was then .NET Core, and our migration from in-process to isolated-process Azure Functions in early 2023.</p><p>Over the course of 2023, however, we noticed that our maintenance burden for our Azure Function apps was increasing substantially. As our product scaled, some of our functions reached runtimes several minutes long; some would even hit the platform’s 10-minute timeout occasionally, even if their average runtime was far shorter. Troubleshooting tasks on our Kanban board languished “in progress” for weeks with no resolution, and each new problem added to our general sense that we should consider a fresh approach to the impacted functionality.</p><p>Azure Functions is an elaborate platform that uses a myriad of bindings to enable all sorts of creative input and output scenarios. In our case, however, a close look revealed that we weren’t really leveraging this advanced functionality. All of our functions were timer triggers, HTTP triggers (including an Event Grid subscriber), or Event Hub or Service Bus listeners. Most of that is easy enough to do in a standard ASP.NET Core application. So, we explored what other approaches we could take to handle the one piece that is less straightforward: timer-based, or scheduled, tasks.</p><p>It turned out that Azure Functions wasn’t as necessary to deal with this requirement as we previously thought. We were able to develop a new architecture within Azure that met all of our needs, significantly reduced the amount of esoteric code in our solution, and substantially improved the reliability and performance of our background infrastructure. Let’s look at how it fits together, starting with the centerpiece of it all.</p><h4>App Service</h4><p>The core of the new system is about the most unassuming thing you can imagine (in Azure): an App Service. We call it Helper, and it’s a satellite service that lives alongside our main product in Azure, sharing Key Vault, App Configuration, and other resources. It’s a .NET 8 application in the same solution and GitHub repository as the main product, enabling a lot of code to be shared and reused between the two as well.</p><p>Helper has a range of endpoints that correspond to short-running (generally seconds long) recurring tasks that we use to keep our service happy and healthy. It also sports a reusable pattern for Event Hub listeners using the ASP.NET Core <a href="https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-8.0&amp;tabs=visual-studio">hosted services feature</a>. And it’s set up as an Event Grid subscriber as well; in fact, it’s now <em>the</em> Event Grid subscriber for our service, a role that used to be handled by satellite Azure Functions. The app runs as a pair of instances on App Service’s powerful and cost-effective Premium v3 tier. So far, even the smallest SKUs in this tier have been more than sufficient to handle everything we need, even with a very high volume of incoming events. Our Azure Functions were already running on a dedicated App Service plan due to some scaling challenges we experienced on the Consumption plan, so this wasn’t a significant cost increase for us.</p><p>Two other pieces of infrastructure interact with Helper to handle scheduled tasks and longer-running tasks.</p><h4>Logic Apps</h4><p>We explored a number of different approaches to handling scheduled tasks without Azure Functions, including a rather advanced product called Azure Batch that proved overly sophisticated for our needs, but the one that ultimately stood out to us was <a href="https://learn.microsoft.com/en-us/azure/logic-apps/logic-apps-overview">Azure Logic Apps</a>. With Logic Apps, it was relatively simple to use the Azure Portal to design a workflow that would run on a schedule, retrieve an API key from Key Vault, and then use it to make an HTTP call to Helper. Once crafted, this flow was reusable for any HTTP call we wanted to make, thanks to infrastructure-as-code.</p><p>The Bicep definition for a Logic App is somewhat arcane, but after we’d figured out the proper workflow once using the Azure Portal designer, we were able to export the template and create a Bicep module that abstracted away the details in favor of a simpler pattern we could repeat for each scheduled task. A reusable Key Vault connection and user-assigned managed identity for all our Logic Apps completed the package. Creating and using a Key Vault connection in the Portal designer gives you what you need for the workflows in the exported template. The Bicep for the Key Vault connection resource itself, however, is a little tricky to figure out, so I’ll share a reduced example here:</p><pre>resource keyVaultConnection &#39;Microsoft.Web/connections@2016–06–01&#39; = {<br>  name: &#39;logicapp-kv-connect&#39;<br>  location: keyVaultLocation // like &#39;eastus&#39;; must be the same region as your Key Vault<br>  properties: {<br>    api: {<br>        id: subscriptionResourceId(&#39;Microsoft.Web/locations/managedApis&#39;, &#39;${keyVaultLocation}&#39;, &#39;keyvault&#39;)<br>    }<br>    displayName: &#39;logicapp-kv-connect&#39;<br>    // these next ones aren&#39;t documented (hence you&#39;ll see warnings), but they work fine<br>    parameterValueType: &#39;Alternative&#39;<br>    alternativeParameterValues: {<br>        vaultName: &#39;your-keyvault&#39;<br>    }<br>  }<br>}</pre><p>Logic Apps have a robust interface in the Azure Portal, with a complete log of runs that can be explored, as well as the ability to trigger an off-schedule run manually at any time. Having access to this history without being affected by Application Insights sampling has proven very useful. What’s more, we’ve been using this approach in production for months, and the scheduled executions have been very dependable. We now have 55 Logic Apps and counting in our Azure subscription (including sandbox, QA, and production environments). Costs scale with the frequency of running the apps, ranging from less than a dollar per month for workflows that run five times an hour or less, to several dollars for tasks that run every minute, making it a particularly affordable solution for the hourly and daily tasks that comprise most of our needs.</p><h4>Container Instances</h4><p>Our service requires plenty of scheduled tasks that only take a matter of seconds to complete, but there are also several that take longer: anywhere from a couple minutes to a few hours. There are multiple reasons why it isn’t practical to run these directly in Helper, including the potential for deployments to interfere with them. For this purpose, we’ve substantially iterated upon and expanded the use of a system we call Batch Jobs, originally created for previous task architectures we employed, to manage longer-running scheduled and on-demand background tasks.</p><p>Batch Jobs are container-based workloads that run on the Azure Container Instances platform. Our Batch Jobs worker is a console app that lives in our main solution and is used to prepare and execute a single Batch Job command using ASP.NET Core style configuration and dependency injection. Each available Batch Job is a class that follows a uniform contract (defined by an abstract base class), providing a consistent API to the worker, but supporting virtually any operation we can do within our codebase. The worker and all class library code needed for the Batch Jobs are deployed as an image to an Azure Container Registry. Because containers can live as long as required to complete their tasks, there is no need to “fan out” large-batch operations using Service Bus or Storage queues; a single container can run an entire process from beginning to end without that extra overhead.</p><p>Helper has endpoints that are used to create Batch Job containers using the images from ACR, monitor their statuses, and clean them up after completion. We partnered with CarMax’s cloud engineers to create a custom Azure role, which gives Helper the ability to administrate these short-lived containers without requiring an unnecessary level of access to more durable or sensitive Azure resources. The most recent iteration of Helper can also programmatically determine and provide the names and options of available Batch Jobs, to power dynamic developer experiences such as admin portals, as well as alert developers to unexpected behavior in the system. Today this piece uses runtime reflection; we’re comfortable with this due to the limited volume and relaxed performance requirements of the feature, but we hope to eventually leverage a compile-time approach like source generation instead.</p><p>We think the Batch Jobs system works best with Helper and Logic Apps in the driver’s seat, but it can also be managed by other means. We’ve done some work to make key elements of our Batch Jobs pattern available internally as a NuGet package to be reused across several services of our own, and to be used by other CarMax product teams that expressed interest in using our pattern. Another team in our organization is now using our package to power Batch Jobs for their newest service, but with Azure Functions as the management layer.</p><h4>Caveats</h4><p>The Azure Functions product exists for a reason. As discussed in the intro, our use-cases for it were somewhat limited, making it relatively easy for us to move to a different solution.</p><p>Migration might not be as obvious a choice for your own team. You may be using complicated patterns of input and output bindings that would require a lot of code to replicate in a standard ASP.NET Core app. The amount of developer time our team could reclaim from Azure Functions maintenance was more than enough to offset any cost increase from going from a Consumption plan for Azure Functions to a Premium v3 App Service plan, but that may not be the case for every team.</p><p>We chose to create Helper as a separate App Service because our product is a high-volume, high-criticality service, and we wanted to maintain dedicated compute resources for that, without having to share them with background tasks that include also-high-volume event processing. Other teams may find it more cost-effective to have Logic Apps call endpoints that are built directly into their main service.</p><p>Our takeaway for you is not that <em>all</em> teams should immediately migrate from Azure Functions to an architecture like ours, but rather that <em>some</em> teams in a position similar to ours pre-migration might benefit from doing so. As always, consider pros and cons and decide what is best for <em>your</em> product and its specific needs and requirements.</p><h4>Conclusion</h4><p>The outcome of this architectural change has been very positive for us, in terms of both reliability and the reduced maintenance effort. Since migration, we’ve been able to continually add new functionality to Helper using the ASP.NET Core patterns we’re most familiar with, shifting a lot of time and effort from troubleshooting back to innovation. We’re excited to be sharing this story now in the hope that others may benefit in various ways from our findings and be inspired to develop your own creative solutions!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=132c5e7b64da" width="1" height="1" alt=""><hr><p><a href="https://medium.com/carmax-engineering-blog/a-very-helpful-service-132c5e7b64da">A Very Helpful Service</a> was originally published in <a href="https://medium.com/carmax-engineering-blog">CarMax Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>