<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Md hasanuzzzaman on Medium]]></title>
        <description><![CDATA[Stories by Md hasanuzzzaman on Medium]]></description>
        <link>https://medium.com/@md-hasanuzzaman?source=rss-38bdf5c88661------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*CdhwZ4DaIrAf-Gt_XD068g.png</url>
            <title>Stories by Md hasanuzzzaman on Medium</title>
            <link>https://medium.com/@md-hasanuzzaman?source=rss-38bdf5c88661------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 11:40:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@md-hasanuzzaman/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Outbox Pattern a reliable way of saving state to your database and publishing a message/event to a…]]></title>
            <link>https://md-hasanuzzaman.medium.com/outbox-pattern-a-reliable-way-of-saving-state-to-your-database-and-publishing-a-message-event-to-a-c57efb1030fe?source=rss-38bdf5c88661------2</link>
            <guid isPermaLink="false">https://medium.com/p/c57efb1030fe</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[csharp]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[design-patterns]]></category>
            <dc:creator><![CDATA[Md hasanuzzzaman]]></dc:creator>
            <pubDate>Mon, 09 Dec 2024 11:15:44 GMT</pubDate>
            <atom:updated>2024-12-09T11:15:44.403Z</atom:updated>
            <content:encoded><![CDATA[<h3>Outbox Pattern a reliable way of saving state to your database and publishing a message/event to a message broker</h3><h4>Achieve Data Consistency and Reliable Messaging Without Distributed Transactions Using the Outbox Pattern</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aJ4kUTFfSHmVrCQtxOdPFA.jpeg" /></figure><p>Data consistency in a <strong>distributed system</strong> where activities span several tasks is very challenging. A typical example is an activity that updates a database and sends a message to a <strong>message broker</strong>. Such an operation is challenging to implement <strong>atomically </strong>because the resources are different. Traditionally, such problems have been solved with <strong>distributed transactions</strong>, which also have their challenges. Another approach that keeps things reliable without the hassle of dealing with distributed transactions is the<strong> Outbox Pattern</strong>.</p><h4>Understanding Distributed Transactions</h4><p>With <strong>distributed transactions</strong>, we have the guarantee of all or nothing for a group of actions on multiple resources, keeping data consistency across a system. Usually this is done using some protocol like <strong>two-phase commit (2PC)</strong>. The tricky thing is a distributed transaction can really add lots of <strong>complexity </strong>and <strong>performance </strong>overhead. It can become the bottleneck very easily if you factor in that from my perspective <strong>scalability </strong>and low coupling / independence (which I think are highly related) are the most important things in<strong> microservices architecture</strong>.</p><h4>Introducing the Outbox Pattern</h4><p>The <strong>Outbox Pattern</strong> solves the dual-write problem by making database and message broker updates <strong>atomic</strong>, <strong>consistent</strong>, and <strong>durable</strong>. It does this by moving the responsibility of managing system events from the application to the <strong>database</strong>.</p><h4>Here’s how it works:</h4><ul><li><strong>Transactional Write:</strong> The application writes the business data and publishes the message to the message broker within a database transaction.</li><li><strong>Outbox Processing:</strong> A separate process/thread consumes events from the outbox table and publishes them to the message broker.</li><li><strong>Message Deletion:</strong> Once published successfully the message is deleted from outbox table or marked as sent.</li></ul><p>By separating database operation with message publishing, <strong>Outbox Pattern</strong> guarantees that messages won’t be lost and the system will not be in an <strong>inconsistent </strong>state without using <strong>distributed transactions</strong>.</p><h4>Implementing the Outbox Pattern: A Code Example</h4><p>Implementing the<strong> Outbox Pattern</strong> in <strong>C#</strong> with <strong>ASP.NET</strong> Core and <strong>Entity Framework Core</strong> examples saving an order to a database and publishing an event to a message broker.</p><h4>1. Setting Up the Database</h4><p>We will have two tables:</p><ul><li><strong>Orders</strong> table to store order data.</li><li><strong>OutboxEvents</strong> table to store events that need to be published.</li></ul><p><strong>Entity Classes:</strong></p><pre>public class Order<br>{<br>    public Guid Id { get; set; }<br>    public string CustomerName { get; set; }<br>    public decimal TotalAmount { get; set; }<br>    public DateTime CreatedAt { get; set; }<br>}<br><br>public class OutboxEvent<br>{<br>    public Guid Id { get; set; }<br>    public string EventType { get; set; }<br>    public string Payload { get; set; }<br>    public DateTime CreatedAt { get; set; }<br>    public bool Processed { get; set; }<br>}</pre><p><strong>DbContext:</strong></p><pre>public class AppDbContext : DbContext<br>{<br>    public DbSet&lt;Order&gt; Orders { get; set; }<br>    public DbSet&lt;OutboxEvent&gt; OutboxEvents { get; set; }<br><br>    public AppDbContext(DbContextOptions&lt;AppDbContext&gt; options) : base(options) { }<br><br>    protected override void OnModelCreating(ModelBuilder modelBuilder)<br>    {<br>        modelBuilder.Entity&lt;Order&gt;().ToTable(&quot;Orders&quot;);<br>        modelBuilder.Entity&lt;OutboxEvent&gt;().ToTable(&quot;OutboxEvents&quot;);<br>    }<br>}</pre><h4>2. Creating the Order and Saving the Outbox Event</h4><p>This ensures both operations are part of the same <strong>database transaction</strong>.</p><pre>public class OrderService<br>{<br>    private readonly AppDbContext _context;<br><br>    public OrderService(AppDbContext context)<br>    {<br>        _context = context;<br>    }<br><br>    public async Task&lt;Order&gt; CreateOrderAsync(Order order)<br>    {<br>        using var transaction = await _context.Database.BeginTransactionAsync();<br><br>        try<br>        {<br>            // Save the order<br>            order.Id = Guid.NewGuid();<br>            order.CreatedAt = DateTime.UtcNow;<br>            _context.Orders.Add(order);<br>            await _context.SaveChangesAsync();<br><br>            // Create an outbox event<br>            var outboxEvent = new OutboxEvent<br>            {<br>                Id = Guid.NewGuid(),<br>                EventType = &quot;OrderCreated&quot;,<br>                Payload = JsonSerializer.Serialize(order),<br>                CreatedAt = DateTime.UtcNow,<br>                Processed = false<br>            };<br><br>            _context.OutboxEvents.Add(outboxEvent);<br>            await _context.SaveChangesAsync();<br><br>            // Commit transaction<br>            await transaction.CommitAsync();<br><br>            return order;<br>        }<br>        catch<br>        {<br>            await transaction.RollbackAsync();<br>            throw;<br>        }<br>    }<br>}</pre><h4>3. Processing the Outbox Table</h4><p>A background service retrieves and processes unprocessed events from the <strong>OutboxEvents</strong> table.</p><pre>public class OutboxProcessor : BackgroundService<br>{<br>    private readonly IServiceProvider _serviceProvider;<br><br>    public OutboxProcessor(IServiceProvider serviceProvider)<br>    {<br>        _serviceProvider = serviceProvider;<br>    }<br><br>    protected override async Task ExecuteAsync(CancellationToken stoppingToken)<br>    {<br>        while (!stoppingToken.IsCancellationRequested)<br>        {<br>            using var scope = _serviceProvider.CreateScope();<br>            var context = scope.ServiceProvider.GetRequiredService&lt;AppDbContext&gt;();<br>            var messageBrokerClient = scope.ServiceProvider.GetRequiredService&lt;IMessageBrokerClient&gt;();<br><br>            var events = await context.OutboxEvents<br>                .Where(e =&gt; !e.Processed)<br>                .ToListAsync(stoppingToken);<br><br>            foreach (var outboxEvent in events)<br>            {<br>                try<br>                {<br>                    // Publish the event<br>                    await messageBrokerClient.PublishAsync(outboxEvent.EventType, outboxEvent.Payload);<br><br>                    // Mark as processed<br>                    outboxEvent.Processed = true;<br>                    context.OutboxEvents.Update(outboxEvent);<br>                    await context.SaveChangesAsync(stoppingToken);<br>                }<br>                catch (Exception ex)<br>                {<br>                    // Log the error and continue<br>                    Console.WriteLine($&quot;Failed to process event {outboxEvent.Id}: {ex.Message}&quot;);<br>                }<br>            }<br><br>            // Delay to avoid busy looping<br>            await Task.Delay(1000, stoppingToken);<br>        }<br>    }<br>}</pre><h4>4. IMessageBrokerClient Interface</h4><p>This is a simple interface for publishing messages to a broker (e.g., <strong>RabbitMQ, Kafka, Service Bus</strong>).</p><pre>public interface IMessageBrokerClient<br>{<br>    Task PublishAsync(string eventType, string payload);<br>}<br><br>public class FakeMessageBrokerClient : IMessageBrokerClient<br>{<br>    public Task PublishAsync(string eventType, string payload)<br>    {<br>        Console.WriteLine($&quot;Published event: {eventType}, Payload: {payload}&quot;);<br>        return Task.CompletedTask;<br>    }<br>}</pre><h4>5. Registering Dependencies and Hosted Service</h4><p>In <strong>Program.cs </strong>register the services:</p><pre>var builder = WebApplication.CreateBuilder(args);<br>builder.Services<br>       .AddDbContext&lt;AppDbContext&gt;(options =&gt;options.UseSqlServer(&quot;YourConnectionString&quot;));<br><br>builder.Services.AddScoped&lt;OrderService&gt;();<br>builder.Services.AddSingleton&lt;IMessageBrokerClient, FakeMessageBrokerClient&gt;();<br>builder.Services.AddHostedService&lt;OutboxProcessor&gt;();<br><br>var app = builder.Build();<br>app.Run();</pre><h4>Ensuring At-Least-Once Delivery</h4><p>The <strong>Outbox Pattern</strong> inherently provides <strong>at-least-once </strong>delivery guarantees. If the message publication fails or the system crashes before marking the event as processed, the <strong>OutboxProcessor </strong>will <strong>retry</strong>, potentially leading to duplicate messages. Therefore, it’s crucial to design idempotent consumers that can handle duplicate messages gracefully.</p><h4>Benefits of the Outbox Pattern</h4><ul><li><strong>Data Consistency</strong>: The state in the database and the messages sent to the broker are consistent.</li><li><strong>Decoupling:</strong> Decouples the data persistence concern from that of message publication, resulting in a more maintainable codebase.</li><li><strong>Reliability:</strong> Guarantees that messages will be eventually delivered despite failures.</li></ul><h4><strong>Conclusion</strong></h4><p>The <strong>Outbox Pattern</strong> is responsible for maintaining <strong>consistency </strong>between your <strong>database</strong> and <strong>message broker</strong> without utilizing <strong>distributed transactions</strong>. This <strong>C#</strong> example will show how to robustly implement the pattern in a modern.NET application.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c57efb1030fe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Comprehensive Guide to Logging and Monitoring with Grafana, Loki, and OpenTelemetry in Docker]]></title>
            <link>https://md-hasanuzzaman.medium.com/comprehensive-guide-to-logging-and-monitoring-with-grafana-loki-and-opentelemetry-in-docker-a7eeeb9efa2c?source=rss-38bdf5c88661------2</link>
            <guid isPermaLink="false">https://medium.com/p/a7eeeb9efa2c</guid>
            <category><![CDATA[logging]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[c-sharp-programming]]></category>
            <dc:creator><![CDATA[Md hasanuzzzaman]]></dc:creator>
            <pubDate>Sun, 01 Dec 2024 20:39:07 GMT</pubDate>
            <atom:updated>2024-12-01T20:39:07.261Z</atom:updated>
            <content:encoded><![CDATA[<h4>Setting Up Robust Logging and Monitoring with Grafana, Loki, and OpenTelemetry in Docker</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WJ-TaYWgcysNgK5jT4_l8Q.jpeg" /></figure><p>Modern applications demand robust solutions for logging and monitoring to ensure reliability and performance. In this guide, we’ll explore how to set up a complete local monitoring stack using <strong>Grafana</strong>, <strong>Loki</strong> and <strong>OpenTelemetry</strong>, all running in <strong>Docker</strong>. This includes advanced logging configurations in .NET with <strong>Serilog </strong>and <strong>NLog</strong> and <strong>distributed tracing</strong> and real-time metrics monitoring.</p><h4>Why Loki + Grafana for Logging?</h4><p><a href="https://grafana.com/docs/loki/latest/"><strong>Loki</strong></a><strong> </strong>is a log aggregation system designed to work seamlessly with Grafana. Unlike traditional logging solutions like <strong>Elasticsearch</strong>, <strong>Loki</strong> focuses on efficiently i<strong>ndexing metadata</strong>, making it lightweight and <strong>cost-effective</strong>.</p><h4>Prerequisites</h4><ol><li>Docker and <strong>Docker Compose</strong> installed on your system.</li><li>Basic understanding of <strong>Docker</strong>, <strong>.NET</strong>, and <strong>logging frameworks</strong>.</li></ol><h4>Setting Up the Monitoring Stack with Docker</h4><p>We’ll deploy <strong>Loki </strong>and <strong>Grafana </strong>using <strong>Docker Compose.</strong></p><p><strong>Step 1: Create a </strong><strong>docker-compose.yaml</strong><br>Create a file named docker-compose.yaml</p><pre>version: &#39;3.8&#39;<br><br>services:<br>  loki:<br>    image: grafana/loki:2.9.1<br>    container_name: loki<br>    ports:<br>      - &quot;3100:3100&quot;<br>    volumes:<br>      - ./loki-data:/loki<br>      - ./loki-config.yaml:/etc/loki/local-config.yaml<br>    command: -config.file=/etc/loki/local-config.yaml<br><br>  grafana:<br>    image: grafana/grafana:9.5.0<br>    container_name: grafana<br>    ports:<br>      - &quot;3000:3000&quot;<br>    depends_on:<br>      - loki<br>    environment:<br>      - GF_SECURITY_ADMIN_PASSWORD=admin</pre><p><strong>Note:</strong> Please create a directory named<strong><em>loki-data</em></strong> in the same directory where docker-compose.yaml will create.</p><p><strong>Step 2: Loki Configuration</strong><br>Create a <strong>loki-config.yaml</strong>:</p><pre>auth_enabled: false<br><br>server:<br>  http_listen_port: 3100<br>  <br>ingester:<br>  wal:<br>    enabled: true<br>    dir: /loki/wal<br>  lifecycler:<br>    ring:<br>      kvstore:<br>        store: inmemory<br>      replication_factor: 1<br>  chunk_idle_period: 3m<br>  chunk_retain_period: 1m<br>  max_transfer_retries: 0<br><br>storage_config:<br>  boltdb:<br>    directory: /loki/index<br>  filesystem:<br>    directory: /loki/chunks<br><br>limits_config:<br>  enforce_metric_name: false<br>  reject_old_samples: true<br>  reject_old_samples_max_age: 168h<br><br>schema_config:<br>  configs:<br>    - from: 2020-10-24<br>      store: boltdb<br>      object_store: filesystem<br>      schema: v11<br>      index:<br>        prefix: index_<br>        period: 24h</pre><p><strong>Step 3: Start the Stack<br></strong>Run the following command to start all services:</p><pre>docker-compose up -d</pre><p><strong>Note:</strong> please use docker-compose down to stop.</p><ul><li><strong>Grafana:</strong> Accessible at <a href="http://localhost:3000/">http://localhost:3000</a>, default login: <strong>admin/admin</strong></li><li><strong>Loki:</strong> Runs on port <strong>3100</strong></li></ul><h4>Logging in .NET with Serilog and Loki</h4><p>We will use <strong>Serilog </strong>for structured logging and send logs to <strong>Loki</strong>.</p><p><strong>Step 1: Install Dependencies</strong><br>Run the following commands in your <strong>.NET</strong> project:</p><pre>dotnet add package Serilog<br>dotnet add package Serilog.Sinks.Grafana.Loki</pre><p><strong>Step 2: Configure Serilog<br></strong>Update your <strong>Program.cs</strong>:</p><pre>using Serilog;<br>using Serilog.Sinks.Grafana.Loki;<br><br>var logger = new LoggerConfiguration()<br>    .WriteTo.GrafanaLoki(&quot;http://localhost:3100&quot;, new List&lt;LokiLabel&gt;<br>    {<br>        new() { Key = &quot;app&quot;, Value = &quot;web_app&quot; }<br>    }, [&quot;app&quot;])<br>    .CreateLogger();<br><br>Log.Logger = logger;<br><br>try<br>{<br>    for (var i = 0; i &lt; 100; i++)<br>    {<br>        Log.Information(&quot;Information log {i}&quot;, i);<br>        await Task.Delay(100);<br>        Log.Error(&quot;Error Log {i}&quot;, i);<br>        await Task.Delay(100);<br>        Log.Debug(&quot;Debug log {i}&quot;, i);<br>        await Task.Delay(100);<br>        Console.WriteLine($&quot;Log send {i}&quot;);<br>    }<br>}<br>catch (Exception ex)<br>{<br>    Log.Fatal(ex, &quot;Application terminated unexpectedly&quot;);<br>}<br>finally<br>{<br>    Log.CloseAndFlush();<br>}</pre><h4>Adding Grafana Loki Datasource</h4><ol><li>Open <strong>Grafana</strong> at <a href="http://localhost:3000/">http://localhost:3000</a>.</li><li>Login using <strong>admin/admin</strong>.</li><li>Go to <strong>Configuration &gt; Data Sources &gt; Add Data Source.</strong></li><li>Choose <strong>Loki </strong>and set the URL to <a href="http://loki:3100/">http://loki:3100</a>.</li><li>Save &amp; Test.</li></ol><h4>Visualizing Logs and Metrics in Grafana</h4><ol><li><strong>Logs:</strong><br><strong>*</strong> Create a dashboard and add a <strong>Loki </strong>panel.<br><strong><em>* </em></strong><em>Query</em>: {job=”varlogs”} to view logs.</li><li><strong>Metrics:<br>* </strong>Add a Prometheus data source pointing to <a href="http://localhost:9090/">http://localhost:9090</a><br><strong>*</strong> Create panels for custom metrics (e.g., <strong>HTTP </strong>requests).</li></ol><h4>Conclusion</h4><p>With <strong>Loki</strong>, <strong>Grafana </strong>and <strong>Serilog</strong>, you have a powerful stack for local logging and monitoring. Adding Prometheus and <strong>OpenTelemetry </strong>further enhances your <strong>observability</strong>. This setup not only simplifies debugging but also ensures your applications remain performant and reliable.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a7eeeb9efa2c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Async Streams and Data Processing Pipelines in CSharp]]></title>
            <link>https://md-hasanuzzaman.medium.com/async-streams-and-data-processing-pipelines-in-csharp-d100a125d0eb?source=rss-38bdf5c88661------2</link>
            <guid isPermaLink="false">https://medium.com/p/d100a125d0eb</guid>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[c-sharp-programming]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Md hasanuzzzaman]]></dc:creator>
            <pubDate>Sat, 23 Nov 2024 18:17:57 GMT</pubDate>
            <atom:updated>2024-11-23T18:17:57.632Z</atom:updated>
            <content:encoded><![CDATA[<h3>Async Streams and Data Processing Pipelines in C#</h3><h4>Demonstrating how to build efficient, responsive data pipelines using async streams and IAsyncEnumerable for real-time or large-scale data handling</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I9D3Mlug7ASlV0n7KIq4fg.jpeg" /></figure><p>In today’s world, applications often need to handle large amounts of data or real-time updates. Whether it’s streaming stock prices, log processing, or user-generated content, designing a responsive and efficient data pipeline is crucial. With C#’s async streams and <strong>IAsyncEnumerable</strong>, we can create a seamless flow of asynchronous data processing while maintaining excellent readability and performance.</p><p>In this post, we’ll explore how async streams simplify complex workflows, implement data processing pipelines, and handle real-world challenges.</p><h4>What Are Async Streams?</h4><p>Introduced in C# 8.0, async streams combine the power of asynchronous programming with enumerable collections. They use the <strong>IAsyncEnumerable&lt;T&gt;</strong> interface to allow iteration over a collection of asynchronous operations.</p><h4>Key Benefits:</h4><ol><li><strong>Asynchronous Execution</strong>: Reduces blocking in high-latency operations like I/O or network calls.</li><li><strong>Lazy Evaluation</strong>: Data is processed only when needed, reducing memory overhead.</li><li><strong>Simplified Code</strong>: Eliminates the need for callbacks or event-driven models.</li></ol><p>Here’s a quick example:</p><pre>async IAsyncEnumerable&lt;int&gt; GenerateNumbersAsync()<br>{<br>    for (int i = 1; i &lt;= 5; i++)<br>    {<br>        await Task.Delay(500); // Simulate async work<br>        yield return i;<br>    }<br>}<br><br>await foreach (var number in GenerateNumbersAsync())<br>{<br>    Console.WriteLine(number);<br>}</pre><p>Output:</p><pre>1<br>2<br>3<br>4<br>5</pre><h4>When Should You Use Async Streams?</h4><ul><li>Processing large datasets (e.g. reading logs or file streams).</li><li>Handling real-time data (e.g. IoT, live feeds).</li><li>Managing workflows with long-running operations.</li></ul><h4>Building a Data Processing Pipeline</h4><p>Let’s build a data pipeline for processing log files in real-time. Imagine a system generating log entries that need to be filtered, transformed, and saved to a database. Here’s how async streams simplify this workflow:</p><p><strong>Step 1: Generate Logs (Source)<br></strong>The first step is to create an asynchronous log generator.</p><pre>async IAsyncEnumerable&lt;string&gt; GenerateLogsAsync()<br>{<br>    string[] logLevels = { &quot;INFO&quot;, &quot;WARN&quot;, &quot;ERROR&quot; };<br>    for (int i = 1; i &lt;= 10; i++)<br>    {<br>        await Task.Delay(300); // Simulate log generation delay<br>        yield return $&quot;{DateTime.UtcNow}: {logLevels[i % 3]} Log entry {i}&quot;;<br>    }<br>}</pre><p><strong>Step 2: Filter Logs (Processing)<br></strong>Let’s filter out logs with the level “<em>INFO</em>”.</p><pre>async IAsyncEnumerable&lt;string&gt; FilterLogsAsync(IAsyncEnumerable&lt;string&gt; logs)<br>{<br>    await foreach (var log in logs)<br>    {<br>        if (!log.Contains(&quot;INFO&quot;))<br>        {<br>            yield return log;<br>        }<br>    }<br>}</pre><p><strong>Step 3: Transform Logs (Processing)<br></strong>Convert log entries into a structured format.</p><pre>record LogEntry(DateTime Timestamp, string Level, string Message);<br><br>async IAsyncEnumerable&lt;LogEntry&gt; TransformLogsAsync(IAsyncEnumerable&lt;string&gt; logs)<br>{<br>    await foreach (var log in logs)<br>    {<br>        var parts = log.Split(&#39; &#39;, 3);<br>        yield return new LogEntry(<br>            DateTime.Parse(parts[0]),<br>            parts[1].TrimEnd(&#39;:&#39;),<br>            parts[2]<br>        );<br>    }<br>}</pre><p><strong>Step 4: Save Logs (Sink)<br></strong>Finally, store the logs in a database.</p><pre>async Task SaveLogsAsync(IAsyncEnumerable&lt;LogEntry&gt; logs)<br>{<br>    await foreach (var log in logs)<br>    {<br>        Console.WriteLine($&quot;Saving log to DB: {log}&quot;);<br>        await Task.Delay(100); // Simulate DB save latency<br>    }<br>}</pre><p><strong>Step 5: Assemble the Pipeline<br></strong>Putting it all together:</p><pre>async Task ProcessLogsAsync()<br>{<br>    var logs = GenerateLogsAsync();<br>    var filteredLogs = FilterLogsAsync(logs);<br>    var structuredLogs = TransformLogsAsync(filteredLogs);<br>    await SaveLogsAsync(structuredLogs);<br>}<br><br>await ProcessLogsAsync();Putting it all together:</pre><p>Output:</p><pre>Saving log to DB: LogEntry { Timestamp = 2024-11-22T12:00:00Z, Level = WARN, Message = Log entry 2 }<br>Saving log to DB: LogEntry { Timestamp = 2024-11-22T12:00:01Z, Level = ERROR, Message = Log entry 3 }<br>...</pre><p><strong>Error Handling in Async Streams<br></strong>Errors can occur at any stage of the pipeline. Use <strong>try-catch</strong> blocks inside the <strong>await </strong>foreach loop or within the generator:</p><pre>async IAsyncEnumerable&lt;string&gt; SafeGenerateLogsAsync()<br>{<br>    try<br>    {<br>        for (int i = 1; i &lt;= 5; i++)<br>        {<br>            if (i == 3) throw new Exception(&quot;Simulated error&quot;);<br>            yield return $&quot;Log {i}&quot;;<br>        }<br>    }<br>    catch (Exception ex)<br>    {<br>        yield return $&quot;Error: {ex.Message}&quot;;<br>    }<br>}</pre><p>Handling errors in the pipeline:</p><pre>await foreach (var log in SafeGenerateLogsAsync())<br>{<br>    Console.WriteLine(log);<br>}</pre><h4>Best Practices for Async Streams</h4><ol><li><strong>Minimize Latency:</strong> Use asynchronous operations within each step to prevent bottlenecks.</li><li><strong>Limit Concurrency:</strong> Use Channel or Parallel.ForEachAsync for controlled parallelism.</li><li><strong>Resource Management</strong>: Dispose of resources like file handles or DB connections properly.</li><li><strong>Avoid Overloading:</strong> Don’t fetch or process more data than necessary.</li></ol><h4>Real-World Use Case: Streaming Stock Prices</h4><p>Here’s an example of streaming and processing stock prices:</p><pre>async IAsyncEnumerable&lt;(string Symbol, decimal Price)&gt; FetchStockPricesAsync(string[] symbols)<br>{<br>    var random = new Random();<br>    foreach (var symbol in symbols)<br>    {<br>        await Task.Delay(500); // Simulate data fetch<br>        yield return (symbol, random.Next(100, 200) + random.NextDecimal());<br>    }<br>}</pre><p>Transform and filter the data:</p><pre>async IAsyncEnumerable&lt;string&gt; AnalyzeStockPricesAsync(IAsyncEnumerable&lt;(string Symbol, decimal Price)&gt; prices)<br>{<br>    await foreach (var (symbol, price) in prices)<br>    {<br>        if (price &gt; 150)<br>        {<br>            yield return $&quot;{symbol}: {price} is above the threshold!&quot;;<br>        }<br>    }<br>}</pre><h4>Conclusion</h4><p>Async streams <strong>IAsyncEnumerable</strong> provide an elegant solution for building responsive and scalable data pipelines. From log processing to real-time data analysis, their potential applications are vast. By combining asynchronous programming with enumerable collections, they simplify complex workflows, reduce memory overhead, and enhance performance.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d100a125d0eb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Supercharge Your dotnet Integration Tests: Using Testcontainers, xUnit, and AssemblyFixture…]]></title>
            <link>https://md-hasanuzzaman.medium.com/supercharge-your-net-36924d78104e?source=rss-38bdf5c88661------2</link>
            <guid isPermaLink="false">https://medium.com/p/36924d78104e</guid>
            <category><![CDATA[software-testing]]></category>
            <category><![CDATA[csharp]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Md hasanuzzzaman]]></dc:creator>
            <pubDate>Thu, 14 Nov 2024 08:02:48 GMT</pubDate>
            <atom:updated>2024-11-14T16:05:41.780Z</atom:updated>
            <content:encoded><![CDATA[<h3>Supercharge Your dotnet Integration Tests: Using Testcontainers, xUnit, and AssemblyFixture for Real SQL Server Testing</h3><h4>Unlock Reliable SQL Server Integration Testing in .NET with Testcontainers, xUnit, and AssemblyFixture</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qNnLroxUXBCmiAg8yDPAqQ.jpeg" /></figure><p>Testing database functionality in .NET applications can feel like a juggling act. Between keeping test data isolated, resetting environments, and avoiding conflicts, it’s easy to see why developers are constantly looking for better ways to streamline the process. What if I told you that by using <strong>Testcontainers</strong>, <strong>xUnit</strong>, and <strong>AssemblyFixture</strong>, you could create a repeatable, reliable, and clean testing setup for SQL Server that runs automatically within your test suite?</p><p>In this post, I’ll walk you through how these tools work together to create a <strong>disposable</strong>, containerized SQL Server instance for <strong>testing</strong>, along with a flexible testing framework. We’ll discuss the setup process, write a few integration tests, and go over <strong>best practices</strong> to help make this part of your workflow.</p><h4>Why Use Containerized Databases for Testing?</h4><p>Testing against a live database can get messy for a few reasons. Here’s a quick breakdown of some common issues:</p><ol><li><strong>Data Contamination</strong>: Every test leaves some trace, and when it’s time to run them again, that leftover data can cause issues.</li><li><strong>Inconsistent Results</strong>: If someone else modifies the database, your tests can start failing unpredictably.</li><li><strong>Environment Discrepancies</strong>: Local databases might not match production environments.</li></ol><p>By using containers, you can avoid these headaches. Containerized databases allow you to set up fresh environments for each test, ensuring consistency and isolation without manual intervention.</p><h4>What Are Testcontainers and Testcontainers.MsSql?</h4><p><strong>Testcontainers </strong>is a powerful .NET library that helps you create containerized environments on-demand, right from your test code. Built on <strong>Docker</strong>, <strong>Testcontainers </strong>allows you to spin up disposable environments for testing, then tear them down <strong>automatically </strong>once your tests complete.</p><p>The <strong>Testcontainers.MsSql</strong> module is specifically for Microsoft SQL Server, meaning you can spin up a SQL Server container with just a few lines of code and test your database operations in an environment that mimics production — without actually touching your live database.</p><p><strong>Key Benefits</strong><br>Using <strong>Testcontainers.MsSql</strong> comes with several advantages:</p><ul><li><strong>Clean Start</strong>: Each test runs in a fresh environment, so no leftover data.</li><li><strong>Cross-Platform</strong>: Runs on any OS that supports Docker.</li><li><strong>Easy Teardown</strong>: The container is automatically destroyed when tests are done, leaving no traces.</li></ul><h4>Setting Up xUnit AssemblyFixture for Shared Resources</h4><p>If you’ve worked with <strong>xUnit</strong>, you probably already know that it’s a popular, open-source testing framework for .NET. One of the things that makes <strong>xUnit </strong>versatile is its support for <strong>fixtures</strong>, which allow you to share setup and teardown logic across tests.</p><p><strong>AssemblyFixture </strong>is an <strong>xUnit </strong>feature that lets you share a single setup across all test classes within a test assembly, which is incredibly useful when you’re working with resources like a SQL Server container that are slow or resource-intensive to initialize.</p><h4>Step-by-Step Setup with Testcontainers, xUnit, and AssemblyFixture</h4><p>Let’s jump into how to set up an integration testing suite with these tools in your .NET project.</p><p><strong>Step 1: Install Required NuGet Packages</strong><br>First, add the necessary NuGet packages. In your test project directory, run:</p><pre>dotnet add package Testcontainers.MsSql<br>dotnet add package xunit<br>dotnet add package xunit.runner.visualstudio<br>dotnet add package Microsoft.NET.Test.Sdk</pre><p><strong>Step 2: Set Up the AssemblyFixture for SQL Server Container</strong><br>Now, let’s create a fixture class to manage the SQL Server container’s lifecycle. The fixture class will:</p><ul><li>Start a SQL Server container before any tests run.</li><li>Expose the container’s connection string for use in test classes</li></ul><p>Here’s a sample implementation:</p><pre>using System;<br>using System.Threading.Tasks;<br>using Testcontainers.MsSql;<br>using Xunit;<br><br>public class MsSqlContainerFixture : IAsyncLifetime<br>{<br>    public MsSqlContainer MsSqlContainer { get; private set; }<br><br>    public string ConnectionString =&gt; MsSqlContainer.GetConnectionString();<br><br>    public async Task InitializeAsync()<br>    {<br>        MsSqlContainer = new MsSqlBuilder()<br>            .WithPassword(&quot;your_password&quot;) // Replace with a strong password<br>            .Build();<br><br>        await MsSqlContainer.StartAsync();<br>    }<br><br>    public async Task DisposeAsync()<br>    {<br>        await MsSqlContainer.DisposeAsync();<br>    }<br>}</pre><p>In this code, <strong>MsSqlContainerFixture</strong> is a class that sets up a containerized SQL Server instance and exposes its <strong>connection string</strong>. The fixture ensures the container starts up before tests and stops after they finish, which helps us avoid repeated database setup and teardown.</p><p><strong>Step 3: Define the Collection Fixture in xUnit</strong><br>Next, we need to configure xUnit to use the fixture. To do this, create a <strong>MsSqlCollection </strong>class and define it as a <strong>CollectionFixture</strong>:</p><pre>[CollectionDefinition(&quot;MsSql Server Collection&quot;)]<br>public class MsSqlCollection : ICollectionFixture&lt;MsSqlContainerFixture&gt; { }</pre><p>This collection fixture will be shared across all test classes in the assembly, meaning we can reuse the SQL Server container for <strong>multiple test classes</strong>.</p><p><strong>Step 4: Write Integration Tests Using the Containerized SQL Server</strong><br>With the container running and the connection string available, you’re ready to write tests that interact with the containerized database. Here’s a sample test class to demonstrate:</p><pre>using Xunit;<br><br>[Collection(&quot;MsSql Server Collection&quot;)]<br>public class MyDatabaseTests<br>{<br>    private readonly MsSqlContainerFixture _fixture;<br><br>    public MyDatabaseTests(MsSqlContainerFixture fixture)<br>    {<br>        _fixture = fixture;<br>    }<br><br>    [Fact]<br>    public async Task TestDatabaseInsert()<br>    {<br>        // Use the connection string from the fixture<br>        var connectionString = _fixture.ConnectionString;<br><br>        // Example: Insert a row, query it, and assert the results<br>    }<br><br>    [Fact]<br>    public async Task TestDatabaseQuery()<br>    {<br>        var connectionString = _fixture.ConnectionString;<br><br>        // Perform a query operation and verify the results<br>    }<br>}</pre><p>Each test can access the <strong>MsSqlContainerFixture </strong>and use its connection string to interact with the database. This setup ensures each test is isolated from the others, while avoiding the need to repeatedly start and stop the SQL Server container.</p><p><strong>Step 5: Running the Tests</strong><br>Run your tests as usual, either through Visual Studio Test Explorer or by using the <strong>dotnet test</strong> command. The containerized SQL Server instance will be available for the duration of the test run and will be automatically cleaned up afterward.</p><h4>Best Practices for Containerized Integration Testing</h4><p>Here are a few tips to keep your tests fast, consistent, and reliable:</p><ol><li><strong>Keep Tests Isolated</strong>: Ensure each test starts with a clean slate by either resetting tables or using transactions that roll back after each test.</li><li><strong>Avoid Hardcoded Secrets</strong>: Store sensitive data like passwords in environment variables or a secure store, not in your code.</li><li><strong>Parallel Testing</strong>: For parallel tests, each test should ideally have its own container instance to avoid conflicts.</li><li><strong>Optimize Container Lifecycle</strong>: Instead of spinning up a new container for every test, use <strong>xUnit’s AssemblyFixture </strong>to share a single container across tests, as demonstrated here.</li></ol><h4>Conclusion</h4><p>Combining <strong>Testcontainers.MsSql</strong> and <strong>xUnit AssemblyFixture</strong> makes it easy to set up robust integration tests for .NET applications, without worrying about database pollution or environment discrepancies. This setup is a great way to ensure your tests are consistent and isolated, mirroring real-world conditions as closely as possible.</p><p>Once you’ve set this up, you’ll wonder how you ever managed without it. Start using these tools today to simplify your .NET integration testing and build better, more resilient applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36924d78104e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementing the Request-Response Pattern with Azure Service Bus: A Practical Guide for Reliable…]]></title>
            <link>https://md-hasanuzzaman.medium.com/implementing-the-request-response-pattern-with-azure-service-bus-a-practical-guide-for-reliable-e6bb5ee5fd77?source=rss-38bdf5c88661------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6bb5ee5fd77</guid>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[microsoft]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Md hasanuzzzaman]]></dc:creator>
            <pubDate>Thu, 07 Nov 2024 13:47:20 GMT</pubDate>
            <atom:updated>2024-11-07T13:47:20.386Z</atom:updated>
            <content:encoded><![CDATA[<h3>Implementing the Request-Response Pattern with Azure Service Bus: A Practical Guide for Reliable Communication in Distributed Systems</h3><h4>Learn how to use Azure Service Bus for asynchronous request-response messaging, including topics and filters for even more efficient, scalable communication in your microservices architecture.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RRNtlhQq3bdwjrtDPwX4AA.jpeg" /><figcaption>Azure Service Bus with topic and filter</figcaption></figure><p>In modern distributed systems, <strong>Azure Service Bus</strong> provides a robust way to implement <strong>request-response messaging</strong> patterns between services. As an enterprise-grade message broker, Azure Service Bus offers <strong>queues</strong> for direct point-to-point messaging, as well as <strong>topics with subscriptions</strong> for publish-subscribe messaging. Using topics and message filters in a request-response pattern allows for <strong>multi-consumer support</strong> and more <strong>targeted message delivery</strong>.</p><p>This guide covers setting up <strong>Azure Service Bus queues and topics with filters</strong>, implementing request-response messaging, and best practices for reliability.</p><h4>What is a request-response pattern?</h4><p>In the <strong>request-response pattern</strong>, a service (the requester) sends a message expecting a response. With Azure Service Bus, this typically involves:</p><ul><li><strong>A request queue</strong> for sending requests and response queue for receiving responses in point-to-point communication.</li><li><strong>Topics and filters</strong> to direct messages to multiple subscribers based on criteria, useful when multiple consumers need specific responses or notifications.</li></ul><p>Using Azure Service Bus topics and filters, you could achieve flexible request-response messaging while maintaining efficiency in <strong>multi-consumer</strong> environments.</p><h4>Why Use Azure Service Bus Topics with Filters?</h4><p>Azure Service Bus topics provide a <strong>publish-subscribe model</strong>, allowing multiple consumers to listen to the same topic through individual <strong>subscriptions</strong>. The use of <strong>filters </strong>allows consumers to determine which messages interest them based on attributes, allowing them to send messages more selectively. This is especially useful for:</p><ul><li><strong>Multi-tenant applications</strong>, where different services or customers only want specific message types.</li><li><strong>Workflow orchestration</strong>, where various services process different parts of a request.</li></ul><p><strong>Topics and filters</strong> allow for more precise control of message delivery, improving performance and reducing implementation costs.</p><h4>Step 1: Setting Up Azure Service Bus Topics and Subscriptions</h4><p>To implement a request-response pattern with topics, set up:</p><ol><li><strong>An Azure Service Bus Topic</strong> for broadcasting messages to various subscribers.</li><li><strong>Subscriptions with filters</strong> to control which messages each subscriber receives.</li></ol><p>In the <strong>Azure portal</strong>:</p><ol><li>Navigate to your <strong>Service Bus namespace</strong> and create a topic (e.g., <strong><em>request-response-topic</em></strong>).</li><li>Create <strong>subscriptions</strong> (e.g., <strong><em>ResponseA</em></strong>, <strong><em>ResponseB</em></strong>) with different filters based on message properties, like <strong><em>CorrelationId</em></strong> or <strong><em>ResponseType</em></strong>.</li></ol><h4>Step 2: Sending Requests with Topics</h4><p>In this approach, each request message includes properties like <strong>CorrelationId </strong>to uniquely identify requests. Additionally, specify properties that will allow subscriptions to filter messages.</p><p>Here’s how to send a message to a topic in C#:</p><pre>public async Task SendRequestAsync(string requestData, string responseType)<br>{<br>    var client = new TopicClient(connectionString, topicName);<br>    var message = new Message(Encoding.UTF8.GetBytes(requestData))<br>    {<br>        CorrelationId = Guid.NewGuid().ToString(),<br>        Label = responseType<br>    };<br><br>    // Additional message properties can be set for filtering.<br>    message.UserProperties[&quot;ResponseType&quot;] = responseType;<br><br>    await client.SendAsync(message);<br>    await client.CloseAsync();<br>}</pre><ul><li><strong>Label</strong>: An optional property for quick identification of message types.</li><li><strong>UserProperties</strong>: Custom properties like <strong><em>ResponseType </em></strong>that can be used for filtering.</li></ul><h4>Step 3: Configuring Filters on Subscriptions</h4><p>Filters allow subscribers to selectively receive messages based on message properties. Azure Service Bus provides various types of filters, including SQL filters and Correlation filters.</p><p>For example, if <strong><em>ResponseA </em></strong>subscription should only receive messages with <strong><em>ResponseType = “A”</em></strong>:</p><pre>var responseSubscriptionClient = new SubscriptionClient(connectionString, topicName, &quot;ResponseA&quot;);<br>var sqlFilter = new SqlFilter(&quot;ResponseType = &#39;A&#39;&quot;);<br>await responseSubscriptionClient.AddRuleAsync(new RuleDescription(&quot;ResponseFilter&quot;, sqlFilter));</pre><p>This ensures only messages where <strong><em>ResponseType </em></strong>is “A” are delivered to the <strong><em>ResponseA </em></strong>subscription.</p><h4>Step 4: Processing the Request and Sending a Response Back to the Topic</h4><p>Each responder listens to its respective subscription, processes the message, and sends the response back to the response subscription on the original topic, using the <strong><em>ReplyTo</em></strong> property. This step completes the request-response cycle by ensuring the original requester receives the processed response.</p><p>Example code for listening on the response subscription:</p><pre>public async Task ListenForRequestsAndRespondAsync()<br>{<br>    var subscriptionClient = new SubscriptionClient(connectionString, topicName, &quot;ResponseA&quot;);<br><br>    subscriptionClient.RegisterMessageHandler(async (message, token) =&gt;<br>    {<br>        // Extract and process the request data<br>        var requestData = Encoding.UTF8.GetString(message.Body);<br>        var responseType = message.UserProperties[&quot;ResponseType&quot;] as string;<br><br>        // Process the request...<br><br>        // Prepare the response message<br>        var responseMessage = new Message(Encoding.UTF8.GetBytes(&quot;Response Data&quot;))<br>        {<br>            CorrelationId = message.CorrelationId,<br>            Label = responseType<br>        };<br><br>        // Send the response to the specified subscription<br>        var responseClient = new TopicClient(connectionString, topicName);<br>        responseMessage.UserProperties[&quot;TargetSubscription&quot;] = message.UserProperties[&quot;ReplyTo&quot;].ToString();<br>        await responseClient.SendAsync(responseMessage);<br>        await responseClient.CloseAsync();<br><br>        // Mark the message as complete<br>        await subscriptionClient.CompleteAsync(message.SystemProperties.LockToken);<br>    }, new MessageHandlerOptions(ExceptionReceivedHandler) { MaxConcurrentCalls = 1, AutoComplete = false });<br>}</pre><p>This pattern allows each subscriber to process messages as per the specified filter, enabling a multi-consumer setup where each service only processes the messages relevant to it.</p><h4>Step 5: Listening for Responses in the Response Subscription</h4><p>Finally, the requester service listens to the <strong>response subscription</strong> on the topic, matching responses by <strong>CorrelationId </strong>to ensure each response corresponds to the correct request.</p><pre>public async Task ListenForResponsesAsync()<br>{<br>    var subscriptionClient = new SubscriptionClient(connectionString, topicName, &quot;response-subscription&quot;);<br>    subscriptionClient.RegisterMessageHandler(async (message, token) =&gt;<br>    {<br>        if (message.CorrelationId == expectedCorrelationId)<br>        {<br>            var responseData = Encoding.UTF8.GetString(message.Body);<br>            // Process the response data...<br>        }<br><br>        await subscriptionClient.CompleteAsync(message.SystemProperties.LockToken);<br>    }, new MessageHandlerOptions(ExceptionReceivedHandler) { MaxConcurrentCalls = 1, AutoComplete = false });<br>}</pre><h4>Step 6: Error Handling and Retry Mechanisms</h4><p>As with queues, topics support <strong>dead-letter queues</strong> and <strong>retry mechanisms</strong> for handling errors gracefully. Configuring these can help manage message failures and ensure reliable processing:</p><ul><li><strong>Dead-letter Queue</strong>: Each subscription has its dead-letter queue, allowing you to track failed messages for each consumer.</li><li><strong>Retry Policies</strong>: Configure retry settings to handle transient issues.</li></ul><p>Example retry configuration:</p><pre>var options = new ServiceBusClientOptions<br>{<br>    RetryOptions = new ServiceBusRetryOptions<br>    {<br>        MaxRetries = 5,<br>        Delay = TimeSpan.FromSeconds(2)<br>    }<br>};</pre><h4>Best Practices for Azure Service Bus Topics and Filters</h4><ol><li><strong>Use Filters to Minimize Traffic</strong>: Design filters to ensure each subscriber only processes relevant messages.</li><li><strong>Monitor Subscription Metrics</strong>: Track metrics in the Azure portal to identify potential issues or imbalances.</li><li><strong>Use Dead-Letter Queues</strong>: Regularly monitor dead-letter queues for failed messages and reprocess them as needed.</li></ol><h4>Conclusion</h4><p>By implementing <strong>Azure Service Bus topics</strong> <strong>with filters</strong> for request-response messaging, you can enhance the flexibility and scalability of your distributed system. This pattern enables seamless, asynchronous communication across multiple services, allowing you to support more complex workflows and scale your application with ease.</p><p><strong>Azure Service Bus</strong> is a powerful choice for achieving resilient, distributed messaging in your microservices architecture. Explore topics, filters, and best practices to make the most out of this messaging solution!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6bb5ee5fd77" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Custom Background Task Queue in ASP.NET Core Without Hangfire]]></title>
            <link>https://md-hasanuzzaman.medium.com/building-a-custom-background-task-queue-in-asp-net-core-without-hangfire-21397f96323b?source=rss-38bdf5c88661------2</link>
            <guid isPermaLink="false">https://medium.com/p/21397f96323b</guid>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[aspnetcore]]></category>
            <category><![CDATA[csharp]]></category>
            <dc:creator><![CDATA[Md hasanuzzzaman]]></dc:creator>
            <pubDate>Mon, 21 Oct 2024 11:10:03 GMT</pubDate>
            <atom:updated>2024-10-21T11:10:03.182Z</atom:updated>
            <content:encoded><![CDATA[<h4>Improve Application Performance by Offloading Long-Running Tasks to Background Services Using QueueBackgroundWorkItem</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NiJOsicfuWiqvoRpJgSAzA.jpeg" /></figure><h4>Introduction</h4><p>When building scalable web applications in ASP.NET Core, it’s often necessary to perform time-consuming tasks, such as sending emails, data processing, or calling external APIs, without blocking the main request-response flow. Running those operations in the background can significantly enhance application performance.</p><p>Here, we will learn how to create a custom background task queue and processor without the usage of libraries like Hangfire. We’ll demonstrate how to pass jobs to a background service using a <em>QueueBackgroundWorkItem</em> approach and trigger the <em>background</em> task from an <em>API controller</em>, including sending emails as an example.</p><h4>Why Use Background Jobs?</h4><p><em>Background</em> jobs are essential for tasks that don’t need to block the user’s interaction with the application. For example:</p><ul><li><strong>Email notifications:</strong> Sending emails after a user action.</li><li><strong>Long-running processes:</strong> Executing data-heavy operations.</li><li><strong>Third-party API calls:</strong> Non-blocking interactions with external services.</li></ul><p>By queuing these tasks to run in the background, we free up the server to address different requests, enhancing the general efficiency of the application.</p><h4>Understanding Background Task Queue</h4><p>ASP.NET Core’s <em>BackgroundService </em>provides a way to implement long-running background tasks. To make this more adaptable, we can set up a background task queue that lets us add tasks to be processed later. The queued tasks will be processed asynchronously by a background worker.</p><h4>Setting Up the Task Queue</h4><p>We’ll start by defining an interface for our background task queue:</p><pre>public interface IBackgroundTaskQueue<br>{<br>    void QueueBackgroundWorkItem(Func&lt;IServiceProvider, CancellationToken, Task&gt; workItem);<br>    Task&lt;Func&lt;IServiceProvider, CancellationToken, Task&gt;&gt; DequeueAsync(CancellationToken cancellationToken);<br>}</pre><p>The <em>QueueBackgroundWorkItem</em> method allows queuing tasks, and the <em>DequeueAsync</em> method retrieves them for processing.</p><h4>Task Queue Implementation</h4><p>Next, we implement this interface using a <em>ConcurrentQueue </em>and a <em>SemaphoreSlim</em> to signal when a new task is available:</p><pre>public class BackgroundTaskQueue : IBackgroundTaskQueue<br>{<br>    private readonly SemaphoreSlim _signal = new(0);<br>    private readonly ConcurrentQueue&lt;Func&lt;IServiceProvider, CancellationToken, Task&gt;&gt; _workItems = new();<br><br>    public void QueueBackgroundWorkItem(Func&lt;IServiceProvider, CancellationToken, Task&gt; workItem)<br>    {<br>        if (workItem == null) throw new ArgumentNullException(nameof(workItem));<br><br>        _workItems.Enqueue(workItem);<br>        _signal.Release(); // Signal that a new item is available<br>    }<br><br>    public async Task&lt;Func&lt;IServiceProvider, CancellationToken, Task&gt;&gt; DequeueAsync(CancellationToken cancellationToken)<br>    {<br>        await _signal.WaitAsync(cancellationToken);<br>        _workItems.TryDequeue(out var workItem);<br><br>        return workItem!;<br>    }<br>}</pre><p>This class allows us to enqueue tasks in a thread-safe manner using <em>ConcurrentQueue </em>and signals the background service to start processing when a task is added.</p><h4>Creating the Background Processor</h4><p>With the queue in place, we need a background service to process the queued tasks. ASP.NET Core’s <em>BackgroundService </em>is the ideal candidate for this:</p><pre>public class QueuedProcessorBackgroundService : BackgroundService<br>{<br>    private readonly IBackgroundTaskQueue _taskQueue;<br>    private readonly IServiceProvider _serviceProvider;<br>    private readonly ILogger _logger;<br><br>    public QueuedProcessorBackgroundService(IBackgroundTaskQueue taskQueue,<br>        IServiceProvider serviceProvider,<br>        ILoggerFactory loggerFactory)<br>    {<br>        _taskQueue = taskQueue;<br>        _serviceProvider = serviceProvider;<br>        _logger = loggerFactory.CreateLogger&lt;QueuedProcessorBackgroundService&gt;();<br>    }<br><br>    protected override async Task ExecuteAsync(CancellationToken cancellationToken)<br>    {<br>        _logger.LogInformation(&quot;Queued Processor Background Service is starting.&quot;);<br><br>        while (!cancellationToken.IsCancellationRequested)<br>        {<br>            var workItem = await _taskQueue.DequeueAsync(cancellationToken);<br><br>            try<br>            {<br>                await workItem(_serviceProvider, cancellationToken);<br>            }<br>            catch (Exception ex)<br>            {<br>                _logger.LogError(ex, $&quot;Error occurred executing {nameof(workItem)}.&quot;);<br>            }<br>        }<br><br>        _logger.LogInformation(&quot;Queued Processor Background Service is stopping.&quot;);<br>    }<br>}</pre><p>This service continuously checks the task queue and processes tasks as they are dequeued. If the service is stopped, the cancellation token ensures a graceful shutdown of any ongoing tasks.</p><h4>Enqueueing Jobs from an API</h4><p>We can now create an API endpoint that enqueues a job for background processing. The job will resolve required services like <em>IEmailService </em>from the <em>IServiceProvider</em> and process them asynchronously.</p><pre>[ApiController]<br>[Route(&quot;api/[controller]&quot;)]<br>public class JobController : ControllerBase<br>{<br>    private readonly IBackgroundTaskQueue _taskQueue;<br><br>    public JobController(IBackgroundTaskQueue taskQueue)<br>    {<br>        _taskQueue = taskQueue;<br>    }<br><br>    [HttpPost(&quot;enqueue-email&quot;)]<br>    public IActionResult EnqueueEmailJob([FromBody] EmailRequest emailRequest)<br>    {<br>        _taskQueue.QueueBackgroundWorkItem(async (serviceProvider, token) =&gt;<br>        {<br>            var logger = serviceProvider.GetRequiredService&lt;ILogger&lt;JobController&gt;&gt;();<br>            var emailService = serviceProvider.GetRequiredService&lt;IEmailService&gt;();<br><br>            logger.LogInformation(&quot;Email job started&quot;);<br><br>            try<br>            {<br>                // Send the email<br>                await emailService.SendEmailAsync(emailRequest.To, emailRequest.Subject, emailRequest.Body, token);<br>                logger.LogInformation(&quot;Email job completed successfully.&quot;);<br>            }<br>            catch (Exception ex)<br>            {<br>                logger.LogError(ex, &quot;Error occurred while sending email.&quot;);<br>            }<br>        });<br><br>        return Ok(&quot;Email job has been queued.&quot;);<br>    }<br>}</pre><p>This API endpoint receives an email request and queues the email-sending job using the task queue.</p><h3>Complete Example: Sending an Email in Background</h3><p>To send an email in the background, we’ll define a model <em>EmailRequest</em> to handle the incoming email data and an email service to simulate sending emails.</p><p><strong>EmailRequest Model</strong></p><pre>public class EmailRequest<br>{<br>    public string To { get; set; }<br>    public string Subject { get; set; }<br>    public string Body { get; set; }<br>}</pre><p><strong>IEmailService and EmailService Implementation</strong></p><pre>public interface IEmailService<br>{<br>    Task SendEmailAsync(string to, string subject, string body, CancellationToken token);<br>}<br><br>public class EmailService : IEmailService<br>{<br>    private readonly ILogger&lt;EmailService&gt; _logger;<br><br>    public EmailService(ILogger&lt;EmailService&gt; logger)<br>    {<br>        _logger = logger;<br>    }<br><br>    public async Task SendEmailAsync(string to, string subject, string body, CancellationToken token)<br>    {<br>        _logger.LogInformation($&quot;Sending email to {to} with subject {subject}.&quot;);<br>        <br>        // Simulate email sending delay<br>        await Task.Delay(2000, token);<br><br>        _logger.LogInformation($&quot;Email to {to} sent successfully.&quot;);<br>    }<br>}</pre><p>This service simulates sending an email with a small delay. In a real-world scenario, this would involve integrating with an SMTP server or a third-party email provider like SendGrid.</p><p><strong>Registering Services in Startup.cs</strong></p><pre>public class Startup<br>{<br>    public void ConfigureServices(IServiceCollection services)<br>    {<br>        services.AddSingleton&lt;IBackgroundTaskQueue, BackgroundTaskQueue&gt;();<br>        services.AddHostedService&lt;QueuedProcessorBackgroundService&gt;();<br>        services.AddSingleton&lt;IEmailService, EmailService&gt;();<br>        services.AddControllers();<br>        services.AddLogging();<br>    }<br><br>    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)<br>    {<br>        if (env.IsDevelopment())<br>        {<br>            app.UseDeveloperExceptionPage();<br>        }<br><br>        app.UseRouting();<br>        app.UseEndpoints(endpoints =&gt; { endpoints.MapControllers(); });<br>    }<br>}</pre><p><strong>Testing the API<br></strong>You can now trigger the email job by sending a POST request to the <em>/api/job/enqueue-email</em> endpoint:</p><pre>POST /api/job/enqueue-email<br>Content-Type: application/json<br><br>{<br>    &quot;to&quot;: &quot;example@example.com&quot;,<br>    &quot;subject&quot;: &quot;Test Email&quot;,<br>    &quot;body&quot;: &quot;This is a test email body.&quot;<br>}</pre><p>This will enqueue the email job, and the background service will process it without blocking the API response.</p><h4><strong>Best Practices</strong></h4><p><strong>Respect Cancellation Tokens:</strong> Always ensure your background tasks respect the <em>CancellationToken</em> to allow graceful shutdown of tasks.</p><p><strong>Error Handling:</strong> Implement proper error handling in background jobs to handle any failures and provide appropriate logging.</p><p><strong>Dependency Resolution:</strong> Use <em>IServiceProvider</em> correctly within <em>QueueBackgroundWorkItem</em> to ensure proper service lifetimes (e.g., scoped services).</p><p><strong>Monitoring:</strong> Consider logging or monitoring tools to keep track of queued and processed tasks.</p><h4>Conclusion</h4><p>Here we built a lightweight solution for running background jobs in ASP.NET Core without relying on external libraries like Hangfire. We created a background service to process tasks, and showed a way to enqueue a task from the queue and send an email. This method helps you to manipulate time-consuming tasks well at the same time as maintaining your app responsive.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=21397f96323b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Table-Valued Parameters: The Secret Weapon for High-Performance Data Transfer in SQL Server and C#]]></title>
            <link>https://md-hasanuzzaman.medium.com/table-valued-parameters-the-secret-weapon-for-high-performance-data-transfer-in-sql-server-and-c-83dc954fad08?source=rss-38bdf5c88661------2</link>
            <guid isPermaLink="false">https://medium.com/p/83dc954fad08</guid>
            <category><![CDATA[csharp]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[sql-server]]></category>
            <dc:creator><![CDATA[Md hasanuzzzaman]]></dc:creator>
            <pubDate>Sat, 12 Oct 2024 18:24:21 GMT</pubDate>
            <atom:updated>2024-10-12T18:24:21.352Z</atom:updated>
            <content:encoded><![CDATA[<h4>Leveraging IEnumerable&lt;SqlDataRecord&gt; for Efficient Bulk Data Inserts and Updates Using TVPs in .NET</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RH250kVxSOVI_ODSyxRxBQ.jpeg" /></figure><h3>Introduction</h3><p>When handling large data transfers between your C# application and SQL Server, performance becomes a critical concern. A common solution involves passing individual records one at a time, which can lead to slow performance due to repeated database calls. Instead, you can leverage Table-Valued Parameters (TVP) to pass multiple records in a single query.</p><p>In this blog, we’ll reveal how to use <strong>Table-Valued Parameters (TVP)</strong> in SQL Server with C#, focusing on the use of <strong><em>IEnumerable&lt;SqlDataRecord&gt;</em></strong>, which is extra green than <strong><em>DataTable</em></strong> for high-overall performance facts transfer.</p><h4>1. What is a Table-Valued Parameter (TVP)?</h4><p>Table-Valued Parameters (TVPs), which were added in SQL Server 2008. It’s allows you to pass multiple rows of data to a stored procedure or query in a single database call. This significantly cuts down the number of times the database is accessed, which can boost the speed of handling large amounts of data.</p><h4>2. Creating a Table-Valued Parameter in SQL Server</h4><p>To use TVP, you first need to define a user-defined table type and a stored procedure in SQL Server.</p><p><strong>Step 1: Define a User-Defined Table Type</strong></p><pre>CREATE TYPE dbo.MyTableType AS TABLE<br>(<br>    Id INT,<br>    Name NVARCHAR(50),<br>    Quantity INT<br>);</pre><p>This type will allow us to pass structured data (i.e., rows and columns) as a parameter to our stored procedures.</p><p><strong>Step 2: Create a Stored Procedure to Use the TVP</strong></p><pre>CREATE PROCEDURE dbo.InsertMyTable<br>    @MyTable MyTableType READONLY<br>AS<br>BEGIN<br>    INSERT INTO MyTable (Id, Name, Quantity)<br>    SELECT Id, Name, Quantity FROM @MyTable;<br>END</pre><p>This stored procedure takes the TVP as an input parameter (@MyTable) and inserts the records into the actual table within the database.</p><h4>3. Using IEnumerable&lt;SqlDataRecord&gt; in C#</h4><p>Now, let’s enforce the C# code on the way to send statistics to the saved system the use of <strong><em>IEnumerable&lt;SqlDataRecord&gt;</em></strong>, that is a greater reminiscence-efficient way to handle massive data units than <strong><em>DataTable</em></strong>.</p><p><strong>Step 1: Creating the Data Source as IEnumerable&lt;SqlDataRecord&gt;<br></strong>We can represent every row of the TVP as a <strong><em>SqlDataRecord</em></strong>. Below is an instance of how to create a set of <em>SqlDataRecord</em> gadgets that represent the statistics to be inserted.</p><pre>public static IEnumerable&lt;SqlDataRecord&gt; GetSqlDataRecords(List&lt;Product&gt; products)<br>{<br>    // Define the schema for the SqlDataRecord (same as MyTableType)<br>    SqlMetaData[] metaData = new SqlMetaData[]<br>    {<br>        new SqlMetaData(&quot;Id&quot;, SqlDbType.Int),<br>        new SqlMetaData(&quot;Name&quot;, SqlDbType.NVarChar, 50),<br>        new SqlMetaData(&quot;Quantity&quot;, SqlDbType.Int)<br>    };<br><br>    // Populate the records<br>    foreach (var product in products)<br>    {<br>        SqlDataRecord record = new SqlDataRecord(metaData);<br>        record.SetInt32(0, product.Id);   // Set Id<br>        record.SetString(1, product.Name); // Set Name<br>        record.SetInt32(2, product.Quantity); // Set Quantity<br><br>        yield return record;<br>    }<br>}</pre><p>Here, <strong>Product</strong> is a simple class:</p><pre>public class Product<br>{<br>    public int Id { get; set; }<br>    public string Name { get; set; }<br>    public int Quantity { get; set; }<br>}</pre><p>This approach creates an <strong><em>IEnumerable&lt;SqlDataRecord&gt;</em></strong> object, which may be used to send data to SQL Server.</p><p><strong>Step 2: Passing IEnumerable&lt;SqlDataRecord&gt; to SQL Server<br></strong>Once we’ve got the records within the shape of <strong><em>IEnumerable&lt;SqlDataRecord&gt;</em></strong>, we are able to bypass it to the saved technique using a <strong><em>SqlParameter </em></strong>of type <strong><em>SqlDbType.Structured</em></strong>.</p><pre>public void InsertDataUsingTVP(IEnumerable&lt;SqlDataRecord&gt; dataRecords)<br>{<br>    string connectionString = &quot;Your SQL Server connection string here&quot;;<br><br>    using (SqlConnection connection = new SqlConnection(connectionString))<br>    {<br>        using (SqlCommand command = new SqlCommand(&quot;dbo.InsertMyTable&quot;, connection))<br>        {<br>            command.CommandType = CommandType.StoredProcedure;<br><br>            // Create the TVP parameter<br>            SqlParameter tvpParam = new SqlParameter();<br>            tvpParam.ParameterName = &quot;@MyTable&quot;;<br>            tvpParam.SqlDbType = SqlDbType.Structured;<br>            tvpParam.TypeName = &quot;dbo.MyTableType&quot;; // Specify the table type<br>            tvpParam.Value = dataRecords; // Pass the IEnumerable&lt;SqlDataRecord&gt;<br><br>            // Add parameter to the command<br>            command.Parameters.Add(tvpParam);<br><br>            // Execute the command<br>            connection.Open();<br>            command.ExecuteNonQuery();<br>        }<br>    }<br>}</pre><p>In this case, we create a <strong><em>SqlParameter </em></strong>with a based type (SqlDbType.Structured), assign it to the TVP kind (dbo.MyTableType), and skip the <strong><em>IEnumerable&lt;SqlDataRecord&gt;</em></strong> as the cost.</p><h4>4. Handling Bulk Updates with IEnumerable&lt;SqlDataRecord&gt;</h4><p>TVP may be used not handiest for bulk inserts however also for bulk updates. Below is an instance of a stored system for updating statistics.</p><p><strong>SQL Server Stored Procedure for Update:</strong></p><pre>CREATE PROCEDURE dbo.UpdateMyTable<br>    @MyTable MyTableType READONLY<br>AS<br>BEGIN<br>    UPDATE MyTable<br>    SET Name = src.Name, Quantity = src.Quantity<br>    FROM MyTable tgt<br>    INNER JOIN @MyTable src ON tgt.Id = src.Id;<br>END</pre><p>This stored procedure updates current records based totally on the Id column.</p><p><strong>C# Code for Bulk Update:<br></strong>The C# code to handle updates is sort of equal to the insert instance, however you will call the UpdateMyTable system alternatively.</p><pre>public void UpdateDataUsingTVP(IEnumerable&lt;SqlDataRecord&gt; dataRecords)<br>{<br>    string connectionString = &quot;Your SQL Server connection string here&quot;;<br><br>    using (SqlConnection connection = new SqlConnection(connectionString))<br>    {<br>        using (SqlCommand command = new SqlCommand(&quot;dbo.UpdateMyTable&quot;, connection))<br>        {<br>            command.CommandType = CommandType.StoredProcedure;<br><br>            SqlParameter tvpParam = new SqlParameter();<br>            tvpParam.ParameterName = &quot;@MyTable&quot;;<br>            tvpParam.SqlDbType = SqlDbType.Structured;<br>            tvpParam.TypeName = &quot;dbo.MyTableType&quot;;<br>            tvpParam.Value = dataRecords;<br><br>            command.Parameters.Add(tvpParam);<br><br>            connection.Open();<br>            command.ExecuteNonQuery();<br>        }<br>    }<br>}</pre><p>This process effectively updates more than one statistics in a single name, making it a great answer for bulk operations.</p><h4><strong>5. Performance Comparison</strong></h4><p>Here’s a short look at how this technique compares to standard row-by using-row insertion.</p><p><strong>Traditional Row-through-Row Insertion:</strong></p><pre>public void InsertDataRowByRow(List&lt;Product&gt; products)<br>{<br>    string connectionString = &quot;Your SQL Server connection string here&quot;;<br><br>    using (SqlConnection connection = new SqlConnection(connectionString))<br>    {<br>        connection.Open();<br><br>        foreach (var product in products)<br>        {<br>            using (SqlCommand command = new SqlCommand(&quot;INSERT INTO MyTable (Id, Name, Quantity) VALUES (@Id, @Name, @Quantity)&quot;, connection))<br>            {<br>                command.Parameters.AddWithValue(&quot;@Id&quot;, product.Id);<br>                command.Parameters.AddWithValue(&quot;@Name&quot;, product.Name);<br>                command.Parameters.AddWithValue(&quot;@Quantity&quot;, product.Quantity);<br><br>                command.ExecuteNonQuery();<br>            }<br>        }<br>    }<br>}</pre><p><strong>TVP Insertion with IEnumerable&lt;SqlDataRecord&gt;:</strong></p><pre>public void InsertDataUsingTVP(IEnumerable&lt;SqlDataRecord&gt; dataRecords)<br>{<br>    // Code as shown above for TVP insertion<br>}</pre><p>For massive datasets, using TVP with <strong><em>IEnumerable&lt;SqlDataRecord&gt;</em></strong> drastically reduces the quantity of database calls, making it some distance more performant than row-by-row insertion.</p><h4>Conclusion</h4><p>Table-Valued Parameters (TVP), combined with <strong><em>IEnumerable&lt;SqlDataRecord&gt;</em></strong>, are a notably green way to carry out bulk facts operations among a C# utility and SQL Server. This approach minimizes database round journeys and may substantially decorate performance in information-intensive applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=83dc954fad08" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Dependency Injection in ASP.NET Core: Going Beyond the Basics]]></title>
            <link>https://md-hasanuzzaman.medium.com/mastering-dependency-injection-in-asp-net-core-going-beyond-the-basics-98d39cbbdca6?source=rss-38bdf5c88661------2</link>
            <guid isPermaLink="false">https://medium.com/p/98d39cbbdca6</guid>
            <category><![CDATA[dependency-injection]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[aspnetcore]]></category>
            <category><![CDATA[dotnet-core]]></category>
            <category><![CDATA[csharp]]></category>
            <dc:creator><![CDATA[Md hasanuzzzaman]]></dc:creator>
            <pubDate>Mon, 23 Sep 2024 12:25:28 GMT</pubDate>
            <atom:updated>2024-09-23T12:25:28.771Z</atom:updated>
            <content:encoded><![CDATA[<h4>Improving Dependency Injection in ASP.NET Core Using Advanced Methods</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kAiY2s6nYw5WtjD6eNqnwg.jpeg" /></figure><h4><strong>Introduction</strong></h4><p>Dependency Injection (DI) is an important feature in ASP.NET Core, enabling you to manage the parts of your utility that rely upon every different in a bendy manner. While many developers understand the basics, superior DI strategies can extensively enhance the capability of huge or complex initiatives. In this article, We will go through few advance techniques.</p><h4>Section 1: Deep Dive into Service Lifetimes</h4><p>ASP.NET Core supports three kinds of provider lifetimes: Singleton, Scoped, and Transient. Each has its very own use case and implications for utility performance.</p><ul><li><strong><em>Singleton</em></strong>: A single instance is used during the application’s lifetime. Suitable for stateless services or caching.</li><li><strong><em>Scoped</em></strong>: A new instance is created according to request. Best for services that want to manipulate state for a request however no longer globally.</li><li><strong><em>Transient</em></strong>: A new instance is created every time the provider is asked. Ideal for lightweight, stateless offerings.</li></ul><p><strong><em>Advanced Tip</em></strong>: Be careful with injecting scoped offerings into singleton offerings. ASP.NET Core will throw exceptions in case you try to clear up a scoped provider from a singleton. A common answer is to use <strong><em>IServiceProvider </em></strong>or a<strong><em> factory sample</em></strong> to manipulate the scope manually.</p><pre>public class MySingletonService<br>{<br>    private readonly IServiceProvider _serviceProvider;<br><br>    public MySingletonService(IServiceProvider serviceProvider)<br>    {<br>        _serviceProvider = serviceProvider;<br>    }<br><br>    public void DoWork()<br>    {<br>        using (var scope = _serviceProvider.CreateScope())<br>        {<br>            var scopedService = scope.ServiceProvider.GetRequiredService&lt;IMyScopedService&gt;();<br>            scopedService.PerformTask();<br>        }<br>    }<br>}</pre><h4>Section 2: Using IServiceProvider and IServiceScopeFactory</h4><p>In some eventualities, you can want greater management over service lifetimes. The <strong><em>IServiceProvider</em></strong> interface lets in for programmatic introduction of provider scopes, which is specifically useful in lengthy-walking responsibilities like history services or in programs with non-standard request lifetimes.</p><p>For instance, if you have a hosted service that desires to remedy scoped services, you can use <strong><em>IServiceScopeFactory</em></strong>:</p><pre>public class MyBackgroundService : BackgroundService<br>{<br>    private readonly IServiceScopeFactory _scopeFactory;<br><br>    public MyBackgroundService(IServiceScopeFactory scopeFactory)<br>    {<br>        _scopeFactory = scopeFactory;<br>    }<br><br>    protected override async Task ExecuteAsync(CancellationToken stoppingToken)<br>    {<br>        while (!stoppingToken.IsCancellationRequested)<br>        {<br>            using (var scope = _scopeFactory.CreateScope())<br>            {<br>                var scopedService = scope.ServiceProvider.GetRequiredService&lt;IMyScopedService&gt;();<br>                await scopedService.DoWorkAsync();<br>            }<br>            await Task.Delay(1000, stoppingToken);<br>        }<br>    }<br>}</pre><p>This sample ensures that every iteration of the background undertaking has its very own sparkling instance of the scoped provider.</p><h4>Section 3: Conditional DI with IServiceProvider</h4><p>In advanced eventualities, you may want to register offerings conditionally based on runtime configurations or different factors. Using the <strong><em>IServiceProvider</em></strong> for runtime DI can give you this pliability:</p><pre>public void ConfigureServices(IServiceCollection services)<br>{<br>    if (someCondition)<br>    {<br>        services.AddTransient&lt;IMyService, MyServiceA&gt;();<br>    }<br>    else<br>    {<br>        services.AddTransient&lt;IMyService, MyServiceB&gt;();<br>    }<br>}</pre><p>ASP.NET Core’s <strong><em>IServiceProvider </em></strong>permits runtime good judgment to decide which implementation to inject based totally on the environment or specific situations.</p><h4>Section 4: Handling Multiple Implementations with IEnumerable&lt;T&gt;</h4><p>Handling Multiple Implementations with <strong><em>IEnumerable&lt;T&gt;</em></strong></p><p>Sometimes, you may want to inject a couple of implementations of an interface and work with them dynamically. ASP.NET Core DI supports this through <strong><em>IEnumerable&lt;T&gt;</em></strong>:</p><pre>public class MyConsumerService<br>{<br>    private readonly IEnumerable&lt;IMyService&gt; _services;<br><br>    public MyConsumerService(IEnumerable&lt;IMyService&gt; services)<br>    {<br>        _services = services;<br>    }<br><br>    public void ExecuteAll()<br>    {<br>        foreach (var service in _services)<br>        {<br>            service.PerformOperation();<br>        }<br>    }<br>}</pre><p>This permits you to register multiple offerings and determine at runtime which one to invoke.</p><pre>services.AddTransient&lt;IMyService, MyServiceA&gt;();<br>services.AddTransient&lt;IMyService, MyServiceB&gt;();</pre><h4>Section 5: Managing Circular Dependencies</h4><p>Circular dependencies may be problematic while the usage of DI. For example, if two offerings rely upon each other, it may cause runtime exceptions. ASP.NET Core lets in for breaking such cycles using constructor injection or by applying strategies like technique or assets injection, or by the usage of factories.</p><p>To clear up circular dependencies:</p><ol><li>Use a manufacturing unit sample to break the cycle.</li><li>Inject <strong><em>IServiceProvider</em></strong> and resolve one of the offerings at runtime.</li></ol><pre>public class ServiceA<br>{<br>    private readonly IServiceProvider _serviceProvider;<br><br>    public ServiceA(IServiceProvider serviceProvider)<br>    {<br>        _serviceProvider = serviceProvider;<br>    }<br><br>    public void UseServiceB()<br>    {<br>        var serviceB = _serviceProvider.GetService&lt;ServiceB&gt;();<br>        serviceB.PerformOperation();<br>    }<br>}</pre><h4>Section 6: Implementing a Class with Multiple Interfaces Sharing the Same Instance</h4><p>When a category implements more than one interface, it’s viable to sign in and inject the equal example for special interfaces, ensuring that the same item is shared throughout the application for each interface.</p><p>Here’s an example wherein a single instance of a class implements multiple interfaces:</p><pre>public interface IEmailSender<br>{<br>    void SendEmail(string to, string subject, string body);<br>}<br><br>public interface ISmsSender<br>{<br>    void SendSms(string number, string message);<br>}<br><br>public class NotificationService : IEmailSender, ISmsSender<br>{<br>    public void SendEmail(string to, string subject, string body)<br>    {<br>        // Email sending logic<br>    }<br><br>    public void SendSms(string number, string message)<br>    {<br>        // SMS sending logic<br>    }<br>}</pre><p>To make sure the same instance is shared throughout each interfaces, sign in the elegance as soon as and map both interfaces to that equal instance:</p><pre>services.AddSingleton&lt;NotificationService&gt;(); // Register the class<br>services.AddSingleton&lt;IEmailSender&gt;(provider =&gt; provider.GetService&lt;NotificationService&gt;()); // Use the same instance<br>services.AddSingleton&lt;ISmsSender&gt;(provider =&gt; provider.GetService&lt;NotificationService&gt;()); // Use the same instance</pre><p>Here, <strong><em>NotificationService </em></strong>is registered as a singleton, and each <strong><em>IEmailSender </em></strong>and <strong><em>ISmsSender </em></strong>resolve to the equal example of <strong><em>NotificationService </em></strong>using <strong><em>issuer.GetService&lt;NotificationService&gt;()</em></strong>.</p><p>When you inject both interfaces into a category, the same <strong><em>NotificationService </em></strong>example can be shared:</p><pre>public class CommunicationController<br>{<br>    private readonly IEmailSender _emailSender;<br>    private readonly ISmsSender _smsSender;<br><br>    public CommunicationController(IEmailSender emailSender, ISmsSender smsSender)<br>    {<br>        _emailSender = emailSender;<br>        _smsSender = smsSender;<br>    }<br><br>    public void NotifyUser(string email, string phoneNumber)<br>    {<br>        _emailSender.SendEmail(email, &quot;Subject&quot;, &quot;Body&quot;);<br>        _smsSender.SendSms(phoneNumber, &quot;Message&quot;);<br>    }<br>}</pre><p><strong>Why This Works</strong></p><p><strong><em>NotificationService </em></strong>is best instantiated once because of its singleton registration.</p><p>Both <strong><em>IEmailSender</em></strong> and <strong><em>ISmsSender</em></strong> are resolved because of the same instance of <strong><em>NotificationService</em></strong>, ensuring steady conduct and nation.</p><h4>Section 7: Using AddKeyedTransient for Multiple Implementations</h4><p>In .NET 8, ASP.NET Core introduces <strong><em>AddKeyed </em></strong>techniques, which includes <strong><em>AddKeyedTransient</em></strong>, <strong><em>AddKeyedScoped</em></strong>, and <strong><em>AddKeyedSingleton</em></strong>, which permit you to register a couple of implementations of the same carrier and retrieve the precise one based on a key at runtime. This is mainly beneficial if you have a couple of versions of the equal provider interface but need to choose one dynamically relying on the context.</p><p>Let’s bear in mind an example in which you’ve got distinct implementations of a <strong><em>IPaymentProcessor </em></strong>interface based on the price type, which includes <strong><em>CreditCardProcessor </em></strong>and <strong><em>PayPalProcessor</em></strong>:</p><pre>public interface IPaymentProcessor<br>{<br>    void ProcessPayment(decimal amount);<br>}<br><br>public class CreditCardProcessor : IPaymentProcessor<br>{<br>    public void ProcessPayment(decimal amount)<br>    {<br>        // Logic for credit card payment<br>        Console.WriteLine($&quot;Processing credit card payment of {amount}&quot;);<br>    }<br>}<br><br>public class PayPalProcessor : IPaymentProcessor<br>{<br>    public void ProcessPayment(decimal amount)<br>    {<br>        // Logic for PayPal payment<br>        Console.WriteLine($&quot;Processing PayPal payment of {amount}&quot;);<br>    }<br>}</pre><p>Using <strong><em>AddKeyedTransient</em></strong>, you may sign in both implementations with exclusive keys:</p><pre>services.AddKeyedTransient&lt;IPaymentProcessor, CreditCardProcessor&gt;(&quot;CreditCard&quot;);<br>services.AddKeyedTransient&lt;IPaymentProcessor, PayPalProcessor&gt;(&quot;PayPal&quot;);</pre><p>To resolve the proper implementation based totally on the key, you can inject <strong><em>IKeyedServiceProvider</em></strong>:</p><pre>public class PaymentService<br>{<br>    private readonly IKeyedServiceProvider _keyedServiceProvider;<br><br>    public PaymentService(IKeyedServiceProvider keyedServiceProvider)<br>    {<br>        _keyedServiceProvider = keyedServiceProvider;<br>    }<br><br>    public void MakePayment(string paymentType, decimal amount)<br>    {<br>        var processor = _keyedServiceProvider.GetRequiredService&lt;IPaymentProcessor&gt;(paymentType);<br>        processor.ProcessPayment(amount);<br>    }<br>}</pre><p>Now, depending on the important thing (“<strong><em>CreditCard</em></strong>” or “<strong><em>PayPal</em></strong>”), the best implementation of <strong><em>IPaymentProcessor</em></strong> may be resolved and used.</p><pre>public class PaymentService<br>{<br>    private readonly IKeyedServiceProvider _keyedServiceProvider;<br><br>    public PaymentService(IKeyedServiceProvider keyedServiceProvider)<br>    {<br>        _keyedServiceProvider = keyedServiceProvider;<br>    }<br><br>    public void MakePayment(string paymentType, decimal amount)<br>    {<br>        var processor = _keyedServiceProvider.GetRequiredService&lt;IPaymentProcessor&gt;(paymentType);<br>        processor.ProcessPayment(amount);<br>    }<br>}</pre><p>This approach is particularly useful when your software supports multiple strategies or behaviors (e.g. charge strategies, notification channels) and lets in for easy separation between one-of-a-kind implementations.</p><h4>Conclusion</h4><p>Advanced DI techniques in ASP.NET Core offer more control over utility conduct, making it viable to optimize for overall performance, manipulate service lifetimes, and implement complex architectures cleanly. By expertise in a way to handle scoped offerings, decorators, and provider lifetimes properly, you can take complete benefit of ASP.NET Core’s powerful DI framework.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=98d39cbbdca6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Custom Configuration Providers in .NET Core: A Comprehensive Guide]]></title>
            <link>https://md-hasanuzzaman.medium.com/mastering-custom-configuration-providers-in-net-core-a-comprehensive-guide-15e2e1615f9d?source=rss-38bdf5c88661------2</link>
            <guid isPermaLink="false">https://medium.com/p/15e2e1615f9d</guid>
            <category><![CDATA[aspnetcore]]></category>
            <category><![CDATA[csharp]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[dotnet-core]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Md hasanuzzzaman]]></dc:creator>
            <pubDate>Wed, 18 Sep 2024 19:43:35 GMT</pubDate>
            <atom:updated>2024-09-18T19:43:35.582Z</atom:updated>
            <content:encoded><![CDATA[<h3>Mastering Custom Configuration Providers in .NET Core: A Comprehensive Guide</h3><h4>Learn how to extend the .NET Core configuration system by building a custom provider for flexible and scalable app configurations.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EhLbwC3_xMpK3Yk3ad0yEQ.jpeg" /></figure><h4><strong>Introduction</strong></h4><p>To keep modern apps flexible and able to grow, it’s really important to have a good system for managing their settings and configurations. In .NET Core, you can load configuration from various sources like JSON, XML, environment variables and more. But sometimes, you might need to get configuration from less common places, such as a database, an API or a custom file format.</p><p>In this post, we’ll explore how to create a <strong>custom configuration provider in .NET Core </strong>to handle non-standard configuration sources.</p><h4><strong>Overview of .NET Core Configuration System</strong></h4><p>Before jumping into creating a custom configuration provider, let’s review how the configuration system works in .NET Core.</p><p>At the core of the configuration system is the <strong><em>IConfiguration </em></strong>interface, which is responsible for managing key-value pairs. You can choose where to get your app’s settings from, like regular text files (JSON), special environment variables and more.</p><p>Here’s an example of the standard configuration setup in .NET Core:</p><pre>var configuration = new ConfigurationBuilder()<br>    .AddJsonFile(&quot;appsettings.json&quot;)<br>    .AddEnvironmentVariables()<br>    .AddUserSecrets(typeof(Program).Assembly, true)<br>    .Build();<br><br>string mySetting = configuration[&quot;MySetting&quot;];</pre><h4><strong>Understanding Configuration Providers</strong></h4><p>At its core, a configuration provider is a class that adheres to the <strong><em>IConfigurationProvider</em></strong> interface. This system requires you to create a function called <strong><em>Load</em></strong> that will get the app’s settings from the chosen place. Once loaded, this data becomes accessible through the <strong><em>IConfiguration</em></strong> interface.</p><h4>Why Create a Custom Configuration Provider?</h4><p>You might want to create a custom provider for various reasons, such as:</p><ul><li><strong>Custom File Formats</strong>: When your application uses configuration files in non-standard formats (e.g., YAML, INI).</li><li><strong>Database Integration</strong>: You need to load configurations directly from a database.</li><li><strong>External APIs</strong>: You want to fetch configuration from an external API.</li><li><strong>Complex Logic</strong>: Your configuration source has special rules or transformations that can’t be handled by the built-in providers.</li></ul><p>In this guide, we’ll create a tool that gets app settings from a simple list in the computer’s memory. This can be easily changed to use more complicated sources later.</p><h4><strong>Creating a Custom Configuration Provider</strong></h4><p>Let’s break down the process of creating a custom configuration provider in .NET Core:</p><ol><li><strong><em>Create a custom configuration source.</em></strong></li><li><strong><em>Implement the custom configuration provider.</em></strong></li><li><strong><em>Creating an Extension Method for Configuration Loading.</em></strong></li><li><strong><em>Use the custom provider in your application.</em></strong></li></ol><h4><strong>Step 1: Create a Custom Configuration Source</strong></h4><p>A configuration source is responsible for creating the configuration provider. To create a custom source, we need to implement the <strong><em>IConfigurationSource</em></strong> interface.</p><p>Here’s the implementation of a custom configuration source:</p><pre>public class CustomConfigurationSource : IConfigurationSource<br>{<br>    private readonly IDictionary&lt;string, string&gt; _data;<br><br>    public CustomConfigurationSource(IDictionary&lt;string, string&gt; data)<br>    {<br>        _data = data;<br>    }<br><br>    public IConfigurationProvider Build(IConfigurationBuilder builder)<br>    {<br>        return new CustomConfigurationProvider(_data);<br>    }<br>}</pre><p>In this example, we’re using an in-memory dictionary (<strong><em>IDictionary&lt;string, string&gt;</em></strong>) to hold configuration values. However, you can replace this with logic that loads data from a database, API, or any other source.</p><h4>Step 2: Create a Custom Configuration Provider</h4><p>Next, we’ll create the actual configuration provider that loads configuration values from the custom source and provides them to the <strong><em>IConfiguration</em></strong> system.</p><p>Our custom provider will inherit from <strong><em>ConfigurationProvider </em></strong>and override the Load method:</p><pre>public class CustomConfigurationProvider : ConfigurationProvider<br>{<br>    private readonly IDictionary&lt;string, string&gt; _data;<br><br>    public CustomConfigurationProvider(IDictionary&lt;string, string&gt; data)<br>    {<br>        _data = data;<br>    }<br><br>    public override void Load()<br>    {<br>        foreach (var kvp in _data)<br>        {<br>            Data[kvp.Key] = kvp.Value;<br>        }<br>    }<br>}</pre><p>In the <strong><em>Load </em></strong>method, we’re populating the <strong><em>Data </em></strong>dictionary, which is used internally by the <strong><em>ConfigurationProvider</em></strong> class to manage configuration values.</p><h4>Step 3: Create an Extension Method</h4><p>To make it easier to integrate our custom configuration provider, let’s create an extension method that allows users to add it to the <strong><em>IConfigurationBuilder </em></strong>without directly interacting with the provider.</p><pre>public static class CustomConfigurationExtensions<br>{<br>    public static IConfigurationBuilder AddCustomConfiguration(this IConfigurationBuilder builder, IDictionary&lt;string, string&gt; data)<br>    {<br>        return builder.Add(new CustomConfigurationSource(data));<br>    }<br>}</pre><p>Now, adding the custom provider becomes as simple as calling <strong><em>AddCustomConfiguration </em></strong>on the <strong><em>ConfigurationBuilder</em></strong>.</p><h4>Step 4: Use the Custom Configuration Provider</h4><p>Finally, let’s integrate our custom provider into a .NET Core application:</p><pre>class Program<br>{<br>    static void Main(string[] args)<br>    {<br>        var customData = new Dictionary&lt;string, string&gt;<br>        {<br>            { &quot;MyCustomKey&quot;, &quot;CustomValue&quot; },<br>            { &quot;AnotherKey&quot;, &quot;AnotherValue&quot; }<br>        };<br><br>        var configuration = new ConfigurationBuilder()<br>            .AddCustomConfiguration(customData)<br>            .Build();<br><br>        Console.WriteLine($&quot;MyCustomKey: {configuration[&quot;MyCustomKey&quot;]}&quot;);<br>        Console.WriteLine($&quot;AnotherKey: {configuration[&quot;AnotherKey&quot;]}&quot;);<br>    }<br>}</pre><p>In this example, we create a <strong><em>Dictionary&lt;string, string&gt;</em></strong> to hold custom key-value pairs and pass it to the configuration builder via our extension method. The values can then be accessed using the <strong><em>IConfiguration </em></strong>interface.</p><h4>Extending the Custom Configuration Provider</h4><p>If you want to load configurations from a database, you can modify the Load method in <strong><em>CustomConfigurationProvider </em></strong>to query the database for configuration values:</p><pre>public override void Load()<br>{<br>    using (var connection = new SqlConnection(&quot;your-connection-string&quot;))<br>    {<br>        connection.Open();<br>        var command = new SqlCommand(&quot;SELECT Key, Value FROM Configuration&quot;, connection);<br>        var reader = command.ExecuteReader();<br><br>        while (reader.Read())<br>        {<br>            Data[reader.GetString(0)] = reader.GetString(1);<br>        }<br>    }<br>}</pre><p>This code fetches configuration values from a database table and loads them into the configuration system.</p><h4>Advanced Configuration Techniques</h4><ul><li><strong>Database-Driven Configuration</strong>: Fetch settings from a database table, enabling dynamic configuration updates without modifying the application’s code.</li><li><strong>Azure Key Vault Integration</strong>: Retrieve sensitive configuration values securely from Azure Key Vault, enhancing your application’s security posture.</li><li><strong>Distributed Configuration:</strong> Manage configuration settings across multiple environments using a distributed configuration service.</li><li><strong>Custom Configuration Formats:</strong> Parse and load configuration data from custom file formats or even programmatic sources.</li></ul><h4>Conclusion</h4><p>Creating a custom configuration provider in .NET Core gives you the flexibility to manage configurations from various non-standard sources like databases, APIs, or custom file formats. By implementing a custom <strong><em>IConfigurationSource </em></strong>and <strong><em>IConfigurationProvider</em></strong>, you can easily extend the configuration system to meet your application’s specific needs.</p><p>Now that you have the foundation, feel free to customize this approach for different data sources and requirements. Custom configuration providers in .NET Core are a powerful tool for developers building complex, scalable applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15e2e1615f9d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>