How to fix the Azure and Bicep setup when Defender for Storage is enabled, but on-upload malware scanning still refuses to turn on.

Enabling Microsoft Defender for Storage with Bicep sounds straightforward. In practice, you can end up in a frustrating half-working state: the Defender resource is created, you are Owner on the subscription, and on-upload malware scanningstill stays disabled.

One of those Azure moments: everything looks right, you are Owner on the subscription, the Bicep deployment says the Defender resource was created, and the portal still shows this gem:

Could not enable on-upload malware scanning

Even better, the storage account is already StorageV2, soft delete is enabled, and you are now deep enough into the rabbit hole that every new portal blade feels slightly personal.

We ran into exactly that while enabling Microsoft Defender for Storage with Bicep. The good news: it does work. The annoying news: there are a couple of hidden prerequisites and one very misleading failure mode.

If you found this post because you searched for one of these errors, you are in the right place:

• Could not enable on-upload malware scanning
• Plan enablement partially succeeded. Could not enable on-upload malware scanning: Failed to update malware scanning.
• The attempt to configure storage notifications for the provided storage account failed
• Failed to retrieve credentials ... listAccountSas ... InvalidAuthenticationToken
• at least one of the claims 'puid' or 'altsecid' or 'oid' should be present

TL;DR

If you just want the shortest possible answer, this is what solved it for us:

1. Register Microsoft.EventGrid on the subscription
2. Set dataScannerResourceId explicitly in the Bicep resource
3. Re-run the same deployment once after provider registration

If step 2 is missing, the Defender resource may be created while malwareScanning.onUpload still stays disabled. If step 1 was done only moments ago, the first retry can still fail with Storage Notification or InvalidAuthenticationToken before the next run succeeds.

🧩 The Setup

The goal was straightforward:

• Enable Defender for Storage on a StorageV2 account via Bicep
• Turn on on-upload malware scanning
• Write scan results to blob index tags / Log Analytics
• Use built-in remediation with BlobSoftDelete

In our case the Bicep module looked roughly like this:

That any(...) is not there because I enjoy chaos. It is there because the API accepted a property the published Bicep type definition did not properly expose in our case.

var defenderDataScannerResourceId = '/subscriptions/${subscription().subscriptionId}/providers/Microsoft.Security/datascanners/StorageDataScanner'

var defenderMalwareScanning = {
  onUpload: {
    isEnabled: true
    capGBPerMonth: 100
    filters: {}
  }
  blobScanResultsOptions: 'BlobIndexTags'
  automatedResponse: 'BlobSoftDelete'
}

resource defenderForStorageSetting 'Microsoft.Security/defenderForStorageSettings@2025-09-01-preview' = {
  scope: storageAccount
  name: 'current'
  properties: any({
    isEnabled: true
    dataScannerResourceId: defenderDataScannerResourceId
    malwareScanning: defenderMalwareScanning
    overrideSubscriptionLevelSettings: true
    sensitiveDataDiscovery: {
      isEnabled: false
    }
  })
}

😵 The Misleading Part: Owner Was Not Enough

At first glance, this smelled like a permissions problem. After all, the error included things like Unauthorized, InvalidAuthenticationToken, and listAccountSas.

Naturally, the first reaction is:

“But I am Owner on the subscription. What else do you want from me, Azure?”

Turns out: quite a bit.

Being Owner was not the actual missing piece in our case.

The real blocker was that Azure still needed infrastructure behind the scenes for storage notifications, and that depends on the Microsoft.EventGrid resource provider being registered in the subscription.

So yes, RBAC and resource provider registration are two different levers. Azure was perfectly happy to let us be Owner while still refusing to make progress because the required provider was not available. Classic.

🔌 The Hidden Dependency: Microsoft.EventGrid

Even though our Bicep did not directly declare an Event Grid resource, Defender for Storage on-upload malware scanning still depended on it.

Why? Because Azure configures storage notifications behind the scenes for on-upload scanning.

This was the missing step:

az provider register -n Microsoft.EventGrid

And if you want to verify it:

az provider show -n Microsoft.EventGrid --query registrationState -o tsv

If the provider is not registered, you can end up with errors like:

Could not enable on-upload malware scanning: The attempt to configure storage notifications for the provided storage account failed.

That is the part I wish someone had highlighted in giant flashing letters somewhere in the docs.

🕵️ Another Gotcha: dataScannerResourceId

There was one more surprise.

After working with support, it turned out that for our scenario the API only reliably persisted the malware scanning values when we explicitly set:

dataScannerResourceId: '/subscriptions/${subscription().subscriptionId}/providers/Microsoft.Security/datascanners/StorageDataScanner'

Without that, Defender itself could be enabled, but the malwareScanning.onUpload block was not applied the way we expected.

So if your deployment says the resource was created, but the portal still shows something like this:

{
  "isEnabled": false,
  "capGBPerMonth": -1
}

then this property is worth checking.

⚠️ The Weird 401 After EventGrid Registration

Here comes the part that made this feel extra fun.

After registering Microsoft.EventGrid, we expected the next deployment to just work.

It did not.

Instead, we got this:

Could not enable on-upload malware scanning: The attempt to configure storage notifications for the provided storage account YourStorageAccountName failed.

Failed to retrieve credentials for request=RequestUri=https://management.azure.com/.../listAccountSas

StatusCode=401
InvalidAuthenticationToken
The received access token is not valid: at least one of the claims ‘puid’ or ‘altsecid’ or ‘oid’ should be present.

ErrorCode: Storage Notification

At that point the natural instinct is to start tearing apart your Bicep, your identities, your storage config, your life choices, and possibly your coffee machine.

In our case, that would have been the wrong move.

The actual fix was simply:

1. Register Microsoft.EventGrid
2. Re-run the same deployment

That second run worked.

So at least in our case, this looked very much like a race condition or delayed backend propagation immediately after provider activation.

🧪 Things We Checked That Were Not the Root Cause

To save you a few hours of random portal clicking:

• The storage account was already StorageV2
• Soft delete was enabled manually and it still did not fix the problem
• The subscription role was already Owner

Soft delete is relevant for BlobSoftDelete remediation, but it was not the missing prerequisite that made on-upload malware scanning start working.

✅ What Finally Worked

For us, the practical checklist was:

1. Use Microsoft.Security/defenderForStorageSettings@2025-09-01-preview
2. Set the malwareScanning block explicitly in Bicep
3. Set dataScannerResourceId explicitly
4. Make sure the subscription has Microsoft.EventGrid registered
5. If the first deployment after provider registration fails with Storage Notification / InvalidAuthenticationToken, run the same deployment again

🔗 Related Azure Posts

If you are currently wiring up more of your Azure platform around the same solution, these might also be useful:

• Configuring Scoped Settings from App Configurations or Secrets and injecting them into Providers
• Simplify Your Life: Logging to Application Insights with Serilog

📌 Final Takeaway

If you are trying to enable Defender for Storage on-upload malware scanning with Bicep and it keeps failing, the short version is:

• Owner on the subscription does not automatically mean all hidden platform prerequisites are in place
• Microsoft.EventGrid may be required even if your Bicep never mentions Event Grid directly
• dataScannerResourceId can be the difference between "resource created" and "settings actually applied"
• A 401 right after Event Grid registration may just be a transient first-run failure, not proof that your entire setup is broken

In other words: if Azure tells you that malware scanning could not be enabled, it might not be your Bicep that is wrong. Sometimes Azure just wants one more provider registration, one more backend sync, and one more deployment run for emotional support.

Happy deploying, and may your onUpload.isEnabled finally stay true.

Defender for Storage Bicep Fix: Malware Scanning was originally published in Medialesson on Medium, where people are continuing the conversation by highlighting and responding to this story.

A few days ago I listened to a podcast episode about knowledge graphs, ontologies and semantics. One of those episodes where you start with “yeah yeah, sounds academic” and end up thinking “wait… this explains half of the problems we currently have with AI”.

This post is my attempt to structure and digest what I learned. If you are curious, you can find the podcast episode here.

A big shoutout to Tobias from Tentris.io and also to Heisenware as the podcast hosts. The discussion was genuinely insightful and highly relevant for anyone building data driven or AI powered systems today.

From relational tables to connected knowledge

Most of us grew up with relational databases. Tables, rows, columns, foreign keys. SQL joins everywhere. And to be fair: relational databases are amazing at what they do. Transactional workloads, clear schemas, predictable queries.

But they start to struggle once relationships become the core of the problem.

Imagine dozens of tables, joined again and again, just to answer questions like:

• Who is related to whom?
• What belongs together?
• How does A influence B indirectly?

At some point the query becomes harder to read than to explain in plain English. And performance usually suffers as well.

This is exactly where knowledge graphs enter the stage.

A knowledge graph stores information as nodes and edges. Or in simpler terms: things and their relationships. Instead of flattening knowledge into tables, you model it closer to how humans actually think about the world. More like a big structured mind map.

Ontologies: the missing layer of meaning

What really made it click for me was the role of ontologies.

An ontology defines what things are and how they are allowed to relate to each other. For example:

• A car is a vehicle
• A car has exactly one engine
• An engine is not a vehicle
• An electric car is a car

This may sound trivial, but it introduces something most databases do not have: formal semantics.

Instead of just storing data, you store meaning and rules. The data inside the graph then follows this structure. Technically this is often represented as triples:

Subject — Predicate — Object

For example:

CarA — hasEngine — Engine1

Once this structure exists, machines can start reasoning about data instead of just retrieving it.

Why graphs outperform joins at scale

Relational databases can model relationships. But every additional join increases complexity and cost. Deeply connected data quickly becomes painful.

Graph databases are optimized for exactly this problem. Traversing relationships is cheap. Asking questions like “show me all entities connected within three hops” is a natural operation, not an edge case.

That does not mean graph databases replace relational ones. Each has its sweet spot:

• Relational databases: structured data and transactions
• Document stores: scalability and flexibility
• Vector databases: similarity search
• Knowledge graphs: relationships and semantics

There is no silver bullet. Only better tools for specific problems.

Real world examples you already use

One of the nicest parts of the podcast was how grounded the examples were.

You are probably using knowledge graphs every day without knowing it:

• Netflix recommendations
• Spotify music discovery
• Google search and knowledge panels
• Large ecommerce platforms that actually understand what you are looking for

In the episode, Netflix was contrasted with Amazon Prime. Not as a product comparison, but as an architectural one. Netflix heavily relies on knowledge graphs to connect users, content, genres, actors and behavior. This allows both better recommendations and better business decisions.

Knowledge graphs meet AI

This is where things get really interesting.

Most current AI systems use Retrieval Augmented Generation. Documents are chunked, converted into vectors and stored in vector databases.

Queries are matched via similarity search and the retrieved chunks are injected into the prompt.

This works surprisingly well for small and medium datasets.

But it has limitations:

• No real understanding of relationships
• No explainability
• Increasing noise with growing data

The podcast introduced the idea of Graph RAG.

Instead of retrieving chunks purely by vector similarity, the system first navigates a knowledge graph. The ontology already encodes structure and meaning. The retrieved subgraph is then passed to the language model.

The benefits are huge:

• Far fewer hallucinations
• Explainable results, because the graph path is visible
• Better scaling across different domains and data silos

A vector space can tell you what looks similar. A knowledge graph can tell you what is related and why.

Why this suddenly matters now

Knowledge graphs are not new. The idea goes back to the early days of the Semantic Web.

What is new is the combination with modern language models.

LLMs are fantastic at language. But they are not knowledge systems. They need structured, meaningful context. Knowledge graphs can provide exactly that.

In a way, LLMs become the voice. Knowledge graphs become the memory.

Where this goes next

This post is intentionally conceptual. I wanted to first understand why knowledge graphs are relevant again before touching code.

In the next post I want to explore this hands on:

• Build a tiny knowledge graph
• Define a minimal ontology
• Query it from a .NET application
• And eventually combine it with an AI workflow

Because as always: things only really click once you build them yourself.

Knowledge Graphs and AI: Why Semantics Matter Again was originally published in Medialesson on Medium, where people are continuing the conversation by highlighting and responding to this story.

Why you should care about structured prompting

Working with LLMs is fun. At first you write a prompt, get a response, copy-paste, maybe tweak it by hand. But once you start building real systems (chatbots, pipelines, data extraction tools…), suddenly you’re dealing with a mix of inconsistent outputs, parsing failures, manual cleanup … and a creeping sense you might not hire yourself in six months if things grow.

Free-form prompting feels flexible, but in the long run it becomes breakable. Small changes to phrasing or context can lead to big shifts in output structure. That means fragile regex-parsers, subtle bugs, and unpredictable integration behavior. This ambiguity is exactly what structured prompting helps you avoid.

If you followed my earlier deep dive into TOON
(JSON vs. TOON: A New Era of Structured Input), you already know how a more compact and semantically clean format can reduce ambiguity before the prompt even reaches the model. TOON is not just a clever encoding trick, it is an example of how structured input and structured output naturally complement each other. If you combine both ideas, you get a workflow that is more predictable, more token-efficient, and less fragile as your system grows.
For this post I am taking the next step in that direction and focusing on structured output, backed by strict schemas and automatically validated responses. You can also find the full sample project on GitHub.

Enter structured prompting: You treat your prompt as a data contract: you define exactly what the output should look like (fields, types, maybe even constraints), you ask the LLM to honour that contract, and then you parse/validate the output just like you would any other external data source. At this point, you can think of an simple API you are requesting for data.

If you build your LLM integrations right, with schema definitions, validation and a robust workflow, you get reliability, predictable output, and cleaner integration.

Practical benefits & where it pays off

Structured prompting shines especially in these scenarios:

• Data extraction from unstructured text (emails, support tickets, logs, scraped HTML…) → output an object, not a story
• Automating pipelines: Let LLMs produce JSON (or another structured format), then feed that directly into databases, APIs or further processing → no manual “cleanup work” needed
• Generating content with a fixed template (e.g. product descriptions, metadata, summaries, configuration files) → you can constrain length, force required fields, enforce enums instead of free-text chaos
• Maintaining and versioning prompts over time. When prompts are structured, you can treat them more like code → diffable, testable, maintainable

In larger projects or production environments, structured prompting often isn’t “nice to have”, it becomes essential.

Checklist: Best Practices for Structured Prompting 🧰

Here’s a practical checklist for you (or any developer) to follow when using structured prompts:

• Define a clear schema
  Use JSON Schema, or your language’s equivalent (e.g. typed data models) to define exactly what output you expect — field names, data types, required vs. optional, enums or value constraints where appropriate.
• Provide a template / example output in your prompt
  Instead of saying “please output JSON”, show a minimal example (empty or with placeholder values). This guides the LLM on structure, order, and expected types, and dramatically increases consistency.
• Keep schemas as simple as possible initally
  Start with only a few required fields. Once basic outputs work reliably, you can extend the schema. This reduces initial complexity and debugging pain.
• Validate every output programmatically
  Do not assume the LLM always follows the schema. Always parse and validate. If parsing fails, implement retry logic or fallback handling.
• Make schema and prompt versioned & maintainable
  Treat prompt and schema like source code. Put them under version control. Keep them modular and documented. That way changes are traceable, and you avoid “works on my machine” secnarios.
• Know when not to use structured prompting
  For creative, open-ended tasks (storytelling, marketing copy, brainstorming, image generation) a rigid schema may hamper quality and expressiveness. Use structured prompting when you want precision, consistency, and machine-readable outputs, otherwise free-form might still win.

Minimal .NET / pseudocode example — how it could look in a real project

// Define a schema (DTO + JSON Schema)
public class SalaryByDepartment
{
    public string Department { get; set; }
    public decimal AverageSalary { get; set; }
}

public class SalaryByDepartmentResponse
{
    public List<SalaryByDepartment> Items { get; set; }
}

// Create JSON schema for the model (simplified)
var schema = @"
{
  ""type"": ""object"",
  ""properties"": {
    ""Items"": {
      ""type"": ""array"",
      ""items"": {
        ""type"": ""object"",
        ""properties"": {
          ""Department"": { ""type"": ""string"" },
          ""AverageSalary"": { ""type"": ""number"" }
        },
        ""required"": [""Department"", ""AverageSalary""],
        ""additionalProperties"": false
      }
    }
  },
  ""required"": [""Items""],
  ""additionalProperties"": false
}";

// Configure the model to *always* return structured output
var options = new ChatCompletionOptions
{
    ResponseFormat = ChatResponseFormat.CreateJsonSchemaFormat(
        "SalaryByDepartmentResponse",
        BinaryData.FromString(schema),
        "Schema for average salary grouped by department",
        true
    )
};

// Prompt template
var messages = new List<ChatMessage>
{
    new SystemChatMessage(
        "You are a precise data analyst. " +
        "Return ONLY valid JSON matching the JSON Schema."
    ),
    new UserChatMessage("Compute the average salary per department from this data: ...")
};

// Send request
var result = await chatClient.CompleteChatAsync(messages, options);
var rawJson = result.Value.Content[0].T

This is rudimentary, but it is precisely this approach that makes your LLM code maintainable, robust, and automatable.

What structured prompting does not solve — caveats & limitations

A schema can guarantee structure, but not correctness. The model may still output garbage, structurally valid JSON with wrong data. Structured prompting helps with format, not with semantics oder qualitative kindness.

Complex or deeply nested schemas increase the risk of parsing failures or token limits. With very large outputs, the model may intentionally or unintentionally cut off parts.

When used in creative scenarios, structure can limit freedom, sometimes you just want free text, not rigid objects.

Final Thoughts

For me, structured prompting is no longer a toy. It is a serious tool when working with LLMs, especially if you want more than just to “try it out quickly.”

If you expect outputs to be further processed, versioned, or automated, then the effort is immediately worthwhile. The small overhead of schema definition and validation quickly pays off, through fewer errors, more consistency, and more elegant integration into real software.

If you continue as before with JSON vs. TOON, for token efficiency, performance, and clean data formats, then structured prompting could be your next upgrade.

Structured Prompting in real projects — checklist & best practices was originally published in Medialesson on Medium, where people are continuing the conversation by highlighting and responding to this story.

When you build a file upload feature, you usually think about one progress bar: the one showing how much of your file has already made it to the server. But what if the real work starts only after the upload?

Maybe you‘re transcoding a video, importing rows into a database, or sending the file to another service. In these cases, you end up with two progress sources:

1. The client upload (how much data is already sent) 📤
2. The backend processing (how much work the API has done so far) 🔄

Let‘s talk about how to combine both in one beautiful, synchronized progress bar. 🎯

The problem 😬

Most apps stop updating progress once the upload finishes. The file is on the server, and now the user just… waits. It‘s not great UX. Users want to see what‘s happening — even if it‘s just „Processing 73%“.

The trick is to link your upload progress (client → backend) with your backend progress (API → client).

The idea 💡

The frontend tracks how many bytes have been sent. Meanwhile, the backend emits Server-Sent Events (SSE) with progress updates while it processes the uploaded file.

So we:

• Report upload progress during the upload (0–50%) 🎬
• Report backend progress via SSE (50–100%) 📈

Both values feed into one single IProgress<double> on the client side.

The clien️t 🖥️

On the client, we wrap the upload stream in a ProgressableStreamContent so each chunk reports how much of the file is uploaded. Once the upload is done, the same progress reporter is used for backend events received via SSE.

var streamContent = new ProgressableStreamContent(stream, progress: progress);
streamContent.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");

var body = new MultipartFormDataContent
{
    { streamContent, "file", fileName },
};

var response = await httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
var responseStream = await response.Content.ReadAsStreamAsync();

var parser = SseParser.Create(responseStream, SseItemParser);
foreach (var sseItem in parser.Enumerate())
{
    if (sseItem.Data is ProgressEventMessage message)
    {
        progress.Report(message.Progress.Percentage);
    }
}

The magic part is that both upload and backend processing share the same progress reporter. The user sees one continuous bar moving from 0 to 100%. 🎉

The backend 🧩

On the backend, we stream the uploaded file to storage and push progress updates via SSE:

var throttler = new ProgressThrottler(TimeSpan.FromMilliseconds(200), minPercentageStep: 1.0);
var sseProgress = new Progress<ReportProgress>(p =>
{
    if (!throttler.ShouldSend(p.Percentage))
    {
        return;
    }

    var message = new ProgressEventMessage(p);
    channel.Writer.TryWrite(new EventMessageBase(EventMessageType.Progress, JsonSerializer.Serialize(message)));
});

Then, as we process the file (e.g. upload to S3 or do other work), we map its progress range to 50–100% and report that through the same channel. 🧮

Why this works so well 🏆

• Single source of truth for progress: the client doesn‘t have to guess what‘s happening.
• Real-time UX: users never see a „frozen“ bar after the upload finishes.
• Extensible: works with any long-running API action — imports, analysis, background tasks, etc.

And yes, technically, you‘re „marrying“ two progress bars. But in this case, it‘s a happy marriage. 💍

Final thoughts 📝

This approach gives your users feedback through the entire lifecycle of an upload — from „sending file“ to „processing complete“.

You can adapt this pattern even if your backend doesn‘t upload the file further. As long as the backend sends progress via SSE, you can merge it with the client‘s upload progress and create a seamless experience. ✨

Combining Upload Progress with API Progress 🚀 was originally published in Medialesson on Medium, where people are continuing the conversation by highlighting and responding to this story.

Why structure matters more than ever

In times where prompts are growing larger and AI models are getting more powerful, one question keeps coming up: how can we keep costs and processing times low?

When working programmatically with LLMs, structured outputs have already become a standard approach. You can tell the AI to respond in a certain format, for example JSON. By defining a model schema , and ideally describing what each field means , the AI tries to understand the context and fill in the output “to the best of its knowledge and belief” (or however close an LLM gets to that).

This has made it easier than ever to work with the results of an AI.
But what about the input?

Even though we can make the output neat and structured, most of us still dump huge JSON, YAML, or even plain text datasets into prompts. That’s not only slow and expensive, it’s also far from token-efficient.

So it was only a matter of time until a new format would appear to fix exactly that.
And here comes TOON 🎉

Meet TOON — The token-friendly cousin of JSON

TOON is a new file format that sits somewhere between JSON and CSV. It’s still human-readable, but optimized for LLMs and tokenization efficiency. According to its creators, it can reduce token counts by 30–60% which, given how token pricing works, can translate into serious cost savings.

Here’s what makes TOON so interesting:

• 💸 Token-efficient: typically 30–60% fewer tokens than JSON
• 🤿 LLM-friendly guardrails: explicit lengths and fields enable validation
• 🍱 Minimal syntax: removes redundant punctuation (braces, brackets, most quotes)
• 📐 Indentation-based structure: like YAML, uses whitespace instead of braces
• 🧺 Tabular arrays: declare keys once, stream data as rows

JSON

[
  {"id": 1, "name": "Alice", "department": "Engineering", "salary": 120000},
  {"id": 2, "name": "Bob", "department": "Marketing", "salary": 95000},
  {"id": 3, "name": "Charlie", "department": "Engineering", "salary": 110000}
]

TOON

[3]{Id,Name,Department,Salary}:
  1,Alice,Engineering,120000
  2,Bob,Marketing,95000
  3,Charlie,Engineering,110000

If you look at it closely, it feels a bit like YAML met CSV for a coffee and decided to raise a structured baby together.

Benchmarks from the creators already show impressive results:
👉 Toon Benchmarking and Key Features

So… why should I care?

If you’re building anything that regularly feeds structured data into LLMs, for example, chatbots, AI-assisted code generation, or multi-step workflows, TOON can significantly reduce prompt size.

It’s not just about the money (although saving around 50% on token usage isn’t bad at all).
It’s also about speed. Fewer tokens mean faster inference and potentially lower latency, especially in real-time systems or when using streaming APIs.

And the best part: it’s already available for multiple languages (including):

🟦 .NET: ToonSharp

🐍 Python: python-toon / pytoon

🦫 Go: gotoon

Real-world evaluation

So I’ve built a small benchmark tool to see how TOON actually performs compared to JSON.

Using a simple dataset of employees, I asked GPT to analyze the data and calculate the average salary by department. The tool measures prompt size, completion token count, and overall response time.

A big thanks to Sebastian Jensen for the neat ConsoleHelper class, it made the output look way more professional than I expected from a console app! 😄

Here are the results from my run:

+---------------------------------------------------------------------------+
| Type         | Prompt Tokens   |  Completions Tokens  |  Duration         |
|--------------+-----------------+----------------------+-------------------|
| JSON         | 1344            |  3475                |  00:00:28.3932721 |
| TOON         | 589             |  2928                |  00:00:23.4953152 |
+---------------------------------------------------------------------------+

That’s roughly a 56% reduction in prompt tokens and a noticeable 5-second speed improvement, with the same output quality from the model.
So yes, TOON doesn’t just look good on paper. It’s actually faster, cheaper, and still easy to read.

Closing thoughts

It’s fascinating to see how we’ve come full circle: we spent years teaching AIs to output structured data, and now we’re optimizing our input to speak their language better.

Whether TOON becomes the new standard or just another clever niche idea, it’s definitely worth keeping an eye on. Especially if you care about performance, cost, and efficiency (and let’s be honest, who doesn’t?).

Please see the complete Project on my GitHub.

JSON vs TOON — A new era of structured input? was originally published in Medialesson on Medium, where people are continuing the conversation by highlighting and responding to this story.

When you trigger a long-running operation through an API — like generating a document, compiling a report, or uploading a processed file to cloud storage, users hate waiting in silence.
Wouldn’t it be great if your backend could continuously stream progress updates back to the browser in real time, without needing WebSockets or polling?

That’s exactly what this post is about.

We’ll explore how to build a robust, async-safe SSE architecture in ASP.NET Core to stream progress updates for any background task, and how to avoid common pitfalls like interleaved writes and lost events.

🎯 The Use Case

Let‘s take a typical example:

A user triggers a document generation request via your API.
The server generates the document, uploads it to a storage service,
and streams progress updates back to the client during each phase.

At the end, the client receives either a success event with the storage path,
or an error event if something failed.

🧩 Core Components

We‘ll build our system around three reliable helpers:

SseResponseWriter handles safe, thread-protected writes to the SSE response stream.

ProgressThrottler ensures we only send updates every few milliseconds or after meaningful percentage changes.

Channel<EventMessageBase>queues events to decouple your long-running task from the network I/O layer.

These three pieces work together to ensure your streaming response is smooth, structured, and cancellation-safe.

💡 Why We Use a Channel

When you generate a document and upload it to a storage service, multiple async operations run in parallel — sometimes in background threads.

If all of them tried to write directly to the HTTP response, you would quickly hit:

• race conditions
• out-of-order JSON messages
• broken connections when the controller finishes before the last event is sent

That‘s why we use a Channel — a thread-safe in-memory queue from System.Threading.Channels .

✅ Your background tasks write progress, success, and error messages into the channel.
✅ A single background consumer reads and sends them through the SseResponseWriter.
✅ The controller only returns once all messages are sent and the connection is cleanly closed.

⏱️ Progress Throttling Done Right

If your API processes thousands of steps or bytes, you don’t want to send every single one to the browser.
Our ProgressThrottler handles this elegantly using:

• time-based throttling (for example, every 200 milliseconds)
• percentage-based throttling (for example, on at least 1 percent change)

It uses a monotonic clock (Stopwatch.GetTimestamp) to avoid bugs caused by system time changes. This means your throttling logic will remain stable even on long-running tasks or VMs.

🧭 The ProgressThrottler

Here’s the full implementation you can directly reuse:

/// <summary>
/// Helper class to throttle progress updates based on time intervals and percentage steps.
/// Uses a monotonic clock (Stopwatch) to avoid issues when system time changes.
/// Thread-safe.
/// </summary>
public sealed class ProgressThrottler
{
    private readonly TimeSpan _minInterval;
    private readonly double _minPercentageStep;
    private long _lastSentTimestamp;
    private double _lastPercentage = -1;
    private readonly object _sync = new();

    public ProgressThrottler(TimeSpan minInterval, double minPercentageStep = 0)
    {
        if (minInterval < TimeSpan.Zero) throw new ArgumentOutOfRangeException(nameof(minInterval));
        if (minPercentageStep < 0) throw new ArgumentOutOfRangeException(nameof(minPercentageStep));

        _minInterval = minInterval;
        _minPercentageStep = minPercentageStep;
    }

    public bool ShouldSend(double currentPercentage)
    {
        lock (_sync)
        {
            var nowTs = Stopwatch.GetTimestamp();

            if (_lastSentTimestamp == 0)
            {
                _lastSentTimestamp = nowTs;
                _lastPercentage = currentPercentage;
                return true;
            }

            if (currentPercentage < _lastPercentage)
            {
                return false;
            }

            var elapsed = Stopwatch.GetElapsedTime(_lastSentTimestamp, nowTs);
            var percentageDiff = Math.Abs(currentPercentage - _lastPercentage);

            bool timeThresholdMet = elapsed >= _minInterval;
            bool percentageThresholdMet = _minPercentageStep > 0 && percentageDiff >= _minPercentageStep;

            if (timeThresholdMet || percentageThresholdMet || currentPercentage >= 100.0)
            {
                _lastSentTimestamp = nowTs;
                _lastPercentage = currentPercentage;
                return true;
            }

            return false;
        }
    }
}

You can tune the timing and precision to fit your workload.
For example, new ProgressThrottler(TimeSpan.FromMilliseconds(200), 1.0) will send at most five updates per second or whenever the percentage increases by one percent.

⚙️ Example Controller
Let’s take a simplified version of our actual implementation:

[HttpPost("generate-document")]
public async Task<IActionResult> GenerateDocumentAsync(
    [FromHeader(Name = "X-Sending-App")] string sender,
    [FromQuery][Required] string documentName,
    [FromQuery] string? targetFolder = null)
{
    if (!long.TryParse(Request.Headers.ContentLength?.ToString(), out var totalBytes))
        return BadRequest("Missing or invalid Content-Length");

    using var sseWriter = new SseResponseWriter(HttpContext.Response);
    var cancellation = HttpContext.RequestAborted;

    // Create an unbounded channel for event messages
    var channel = Channel.CreateUnbounded<EventMessageBase>(new UnboundedChannelOptions
    {
        SingleReader = true,
        SingleWriter = false
    });

    // Background task to process messages and send them via SSE
    var sseTask = Task.Run(async () =>
    {
        await foreach (var msg in channel.Reader.ReadAllAsync(cancellation))
        {
            await sseWriter.WriteEventAsync(msg, cancellation);
        }
    }, cancellation);

    // Throttle progress updates
    var throttler = new ProgressThrottler(TimeSpan.FromMilliseconds(200), minPercentageStep: 1.0);
    var progress = new Progress<ReportProgress>(p =>
    {
        if (!throttler.ShouldSend(p.Percentage)) return;

        var message = new ProgressEventMessage(p with { Percentage = Math.Floor(p.Percentage) });
        channel.Writer.TryWrite(new EventMessageBase(EventMessageType.Progress, JsonSerializer.Serialize(message)));
    });

    // Generate document (simulate long-running process)
    var generatedDocument = await documentService.GenerateAsync(documentName, progress, cancellation);

    // Upload generated file to external storage
    var uploadProgress = new Progress<ReportProgress>(p =>
    {
        var adjusted = p with { Percentage = 50 + p.Percentage * 0.5 };
        progress.Report(adjusted);
    });

    var uploadResult = await storageService.UploadFileAsync(sender, targetFolder, generatedDocument, uploadProgress, cancellation);

    // Send final success message
    channel.Writer.TryWrite(new EventMessageBase(EventMessageType.Success,
        JsonSerializer.Serialize(new SuccessEventMessage(uploadResult.RelativeFilePath))));

    channel.Writer.Complete();
    await sseTask;
    await HttpContext.Response.CompleteAsync();

    return Empty;
}

🧠 Key Takeaways

🧱 1. Use Channels to Decouple Event Flow

Channels let you queue up progress and result messages safely and keep network I/O off your business logic thread.

🧩 2. Protect SSE Writes with a Semaphore

ASP.NET Core’s HttpResponse isn’t thread-safe for concurrent writes.
Our SseResponseWriter ensures only one event is written at a time.

⚙️ 3. Always Throttle Updates

Throttle by time and by percentage step to keep updates smooth without flooding the network.

🕒 4. Use Stopwatch Instead of DateTime.UtcNow

It’s a monotonic clock that never jumps backward or forward, essential for stable throttling.

🧵 5. Wait for All SSE Writes Before Closing

The controller should only finish once the channel is drained and all messages are flushed.
Otherwise, the client might miss the final event.

🧭 The SseResponseWriter

public sealed class SseResponseWriter : IDisposable
{
    private readonly HttpResponse _response;
    private readonly SemaphoreSlim _writeLock = new(1, 1);

    public SseResponseWriter(HttpResponse response)
    {
        _response = response ?? throw new ArgumentNullException(nameof(response));
        _response.Headers.CacheControl = "no-cache";
        _response.Headers.Connection = "keep-alive";
        _response.Headers["X-Accel-Buffering"] = "no";
        _response.ContentType = "text/event-stream";
    }

    public async Task WriteEventAsync(EventMessageBase message, CancellationToken token)
    {
        await _writeLock.WaitAsync(token);
        try
        {
            await _response.WriteAsync($"event: {message.Type.ToString().ToLowerInvariant()}\n", token);
            await _response.WriteAsync($"data: {message.Body}\n\n", token);
            await _response.Body.FlushAsync(token);
        }
        finally
        {
            _writeLock.Release();
        }
    }

    public void Dispose() => _writeLock.Dispose();
}

💬 Client-Side Example

Here’s how to consume those events from your frontend:

const evtSource = new EventSource("/api/v1/documents/generate-document?documentName=report.pdf");

evtSource.addEventListener("progress", e => {
  const data = JSON.parse(e.data);
  console.log(`Progress: ${data.progress.percentage}%`);
});

evtSource.addEventListener("success", e => {
  const data = JSON.parse(e.data);
  console.log("Document ready at:", data.relativeFilePath);
  evtSource.close();
});

evtSource.addEventListener("error", e => {
  console.error("Error:", e.data);
  evtSource.close();
});

✅ Final Thoughts

This approach is not just about document generation, it’s a general pattern for any long-running server process:

• Generating PDFs or reports
• AI model inference
• Video or audio processing
• Background imports or exports

By combining Channels, Throttling, and SSE, you get real-time progress reporting that’s reliable, scalable, and surprisingly simple.

You don’t need WebSockets.
You don’t need SignalR.
Just smart use of built-in .NET features.

Happy coding, and may your uploads always reach 100% — smoothly.

⚡ Real-Time Progress Updates for Long-Running API Tasks with Server-Sent Events (SSE) in ASP.NET was originally published in Medialesson on Medium, where people are continuing the conversation by highlighting and responding to this story.

Amazon S3 (Simple Storage Service) is one of the most popular cloud storage solutions for modern applications. If you’re a .NET developer looking to integrate S3 into your application using clean architecture principles and dependency injection, this comprehensive guide will walk you through the entire process step by step.

Why Use Amazon S3 with .NET?

Amazon S3 offers numerous benefits for .NET applications:

• Scalability: Handle virtually unlimited storage needs
• Durability: 99.999999999% (11 9’s) data durability
• Cost-effective: Pay only for what you use
• Global accessibility: Access files from anywhere in the world
• Security: Enterprise-grade security features
• Integration: Seamless integration with other AWS services

Prerequisites

Before we begin, ensure you have:

• .NET 9.0 or later installed
• An active AWS account
• Basic knowledge of C# and dependency injection
• Visual Studio, VS Code, or JetBrains Rider

Setting Up AWS S3 and IAM

Step 1: Create an S3 Bucket

1. Log into the AWS Management Console
2. Navigate to the S3 service
3. Click “Create bucket”
4. Choose a unique bucket name (e.g., your-app-storage-bucket)
5. Select your preferred AWS region
6. Configure bucket settings (keep defaults for this tutorial)
7. Click “Create bucket”

Step 2: Create an IAM User

1. Navigate to the IAM service in AWS Console
2. Click “Users” → “Create user”
3. Enter a username (e.g., s3-dotnet-user)
4. Select “Programmatic access”
5. Click “Next: Permissions” and select your corresponding permissions like Amazon S3 FullAccess

Step 3: Generate Access Keys

1. Complete the user creation process
2. Save the Access Key ID and Secret Access Key securely
3. Important: Never commit these keys to version control

Project Setup and Dependencies

Create a new .NET console application and install the required NuGet packages:

dotnet new console -n AmazonS3Sample
cd AmazonS3Sample
dotnet add package AWSSDK.S3
dotnet add package Microsoft.Extensions.Hosting
dotnet add package Microsoft.Extensions.Configuration
dotnet add package Microsoft.Extensions.Configuration.Json
dotnet add package Microsoft.Extensions.DependencyInjection

Implementation Overview

Our implementation follows clean architecture principles with:

• Configuration model for AWS settings
• Service interface for abstraction
• Service implementation with proper error handling
• Dependency injection setup for clean IoC
• Logging integration for monitoring

Creating the Configuration Model

First, let’s create a configuration model to handle AWS settings:

namespace AmazonS3Sample.Configuration;

public sealed class AwsStorageConfiguration
{
    public static string SectionName => "Aws:Storage";
    
    public string AccessKeyId { get; set; } = null!;
    public string SecretAccessKey { get; set; } = null!;
    public string Region { get; set; } = null!;
    public string Bucket { get; set; } = null!;
}

Implementing the Storage Service

Service Interface

Create a clean interface that defines our S3 operations:

using System.Net.Mime;
using Amazon.S3.Model;

namespace AmazonS3Sample.Services.Interfaces;

public interface IAwsStorageClientService
{
    ValueTask<IEnumerable<string>> GetFileNamesAsync();
    ValueTask<GetObjectResponse> GetFileAsync(string fileName);
    ValueTask UploadFileAsync(string fileName, byte[] data, ContentType contentType);
    ValueTask DeleteFileAsync(string fileName);
}

Service Implementation

Now, implement the service with comprehensive error handling and logging:

using System.Net.Mime;
using Amazon;
using Amazon.S3;
using Amazon.S3.Model;
using AmazonS3Sample.Configuration;
using AmazonS3Sample.Services.Interfaces;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

namespace AmazonS3Sample.Services;
public class AwsStorageClientService : IAwsStorageClientService
{
    private readonly IAmazonS3 _client;
    private readonly IOptions<AwsStorageConfiguration> _options;
    private readonly ILogger<AwsStorageClientService> _logger;
    public AwsStorageClientService(IOptions<AwsStorageConfiguration> options, ILogger<AwsStorageClientService> logger)
    {
        _options = options;
        _logger = logger;
        _client = CreateAmazonS3Client();
    }
    
    public async ValueTask<IEnumerable<string>> GetFileNamesAsync()
    {
        _logger.LogInformation("Getting file names from bucket {Bucket}", _options.Value.Bucket);
        
        try
        {
            var result = await _client.ListObjectsV2Async(new ListObjectsV2Request
            {
                BucketName = _options.Value.Bucket,
            });
            
            if (result.S3Objects == null || !result.S3Objects.Any())
            {
                _logger.LogInformation("No files found in bucket {Bucket}", _options.Value.Bucket);
                return Array.Empty<string>();
            }
            return result.S3Objects.Select(x => x.Key);
        }
        catch (Exception e)
        {
            _logger.LogError(e, "Error getting file names from bucket {Bucket}", _options.Value.Bucket);
            throw;
        }
    }
    public async ValueTask<GetObjectResponse> GetFileAsync(string fileName)
    {
        _logger.LogInformation("Getting file '{FileName}' from bucket {Bucket}", fileName, _options.Value.Bucket);
        try
        {
            var response = await _client.GetObjectAsync(_options.Value.Bucket, fileName);
            return response;
        }
        catch (Exception e)
        {
            _logger.LogError(e, "Error getting file '{FileName}' from bucket {Bucket}", fileName, _options.Value.Bucket);
            throw;
        }
    }
    public async ValueTask UploadFileAsync(string fileName, byte[] data, ContentType contentType)
    {
        _logger.LogInformation("Uploading file '{FileName}' to bucket {Bucket}", fileName, _options.Value.Bucket);
        try
        {
            using var stream = new MemoryStream(data);
            var request = new PutObjectRequest
            {
                BucketName = _options.Value.Bucket,
                Key = fileName,
                InputStream = stream,
                ContentType = contentType.ToString()
            };
            
            var response = await _client.PutObjectAsync(request);
            _logger.LogInformation("File '{FileName}' uploaded successfully with status {StatusCode}", 
                fileName, response.HttpStatusCode);
        }
        catch (Exception e)
        {
            _logger.LogError(e, "Error uploading file '{FileName}' to bucket {Bucket}", fileName, _options.Value.Bucket);
            throw;
        }
    }
    public async ValueTask DeleteFileAsync(string fileName)
    {
        _logger.LogInformation("Deleting file '{FileName}' from bucket {Bucket}", fileName, _options.Value.Bucket);
        try
        {
            await _client.DeleteObjectAsync(_options.Value.Bucket, fileName);
            _logger.LogInformation("File '{FileName}' deleted successfully", fileName);
        }
        catch (Exception e)
        {
            _logger.LogError(e, "Error deleting file '{FileName}' from bucket {Bucket}", fileName, _options.Value.Bucket);
            throw;
        }
    }
    
    private AmazonS3Client CreateAmazonS3Client()
    {
        var options = _options.Value;
        return new AmazonS3Client(options.AccessKeyId, options.SecretAccessKey, 
            RegionEndpoint.GetBySystemName(options.Region));
    }
}

Setting Up Dependency Injection

Create an extension method to register our services cleanly:

using AmazonS3Sample.Configuration;
using AmazonS3Sample.Services;
using AmazonS3Sample.Services.Interfaces;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;

namespace AmazonS3Sample.DependencyInjection;
public static class AwsStorageDependencyInjection
{
    public static IServiceCollection AddAwsStorageClientService(this IServiceCollection services, IConfiguration configuration)
    {
        services.Configure<AwsStorageConfiguration>(configuration.GetSection(AwsStorageConfiguration.SectionName));
        services.AddTransient<IAwsStorageClientService, AwsStorageClientService>();
        
        return services;
    }
}

Configuration and Environment Variables

appsettings.json Setup

Create your configuration file (never commit real credentials):

{
    "Logging": {
        "LogLevel": {
            "Default": "Information",
            "System": "Warning",
            "Microsoft": "Warning"
        }
    },
    "Aws": {
        "Storage": {
            "AccessKeyId": "YOUR_ACCESS_KEY_ID",
            "SecretAccessKey": "YOUR_SECRET_ACCESS_KEY",
            "Region": "us-east-1",
            "Bucket": "your-bucket-name"
        }
    }
}

Important: Use UserSecrets to store sensetive data during development.

Using the Service in Your Application

Here’s how to set up and use the service in your Program.cs:

using AmazonS3Sample.DependencyInjection;
using AmazonS3Sample.Services.Interfaces;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

var builder = Host.CreateApplicationBuilder(args);

builder.Configuration
    .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
    .AddEnvironmentVariables();
builder.Services.AddAwsStorageClientService(builder.Configuration);

using var host = builder.Build();

// Use the service
var storageService = host.Services.GetRequiredService<IAwsStorageClientService>();

// Example operations
try
{
    // List files
    var fileNames = await storageService.GetFileNamesAsync();
    Console.WriteLine("Files in S3 bucket:");
    foreach (var fileName in fileNames)
    {
        Console.WriteLine($"- {fileName}");
    }

    // Upload a file
    var testContent = "Hello, S3 from .NET!"u8.ToArray();
    await storageService.UploadFileAsync("test-file.txt", testContent,
        new System.Net.Mime.ContentType("text/plain"));
    Console.WriteLine("File uploaded successfully!");

    // Download the file
    var downloadedFile = await storageService.GetFileAsync("test-file.txt");
    using var reader = new StreamReader(downloadedFile.ResponseStream);
    var content = await reader.ReadToEndAsync();
    Console.WriteLine($"Downloaded content: {content}");

    // Clean up
    await storageService.DeleteFileAsync("test-file.txt");
    Console.WriteLine("File deleted successfully!");
}
catch (Exception ex)
{
    Console.WriteLine($"Error: {ex.Message}");
}

await host.RunAsync();

Complete Example Repository

You can find the complete working example on GitHub: Amazon S3 .NET Sample Repository

Conclusion

Integrating Amazon S3 with .NET using dependency injection provides a clean, maintainable, and testable solution for cloud storage needs. By following the patterns and best practices outlined in this guide, you’ll have a robust foundation for handling file operations in your .NET applications.

The combination of proper dependency injection, configuration management, and error handling ensures your S3 integration is production-ready and follows modern .NET development practices.

Remember to always follow security best practices, never commit credentials to version control, and implement proper monitoring and logging for production environments.

How to Easily Connect to Amazon S3 with .NET and Dependency Injection: A Complete Guide was originally published in Medialesson on Medium, where people are continuing the conversation by highlighting and responding to this story.

Quick wins, real benchmarks, and when to trade developer joy for raw speed.

Sponsored Content Disclosure

This article contains a paid sponsored section that was added as part of a commercial collaboration with Iron Software. The sponsored content is clearly separated from the original editorial content and does not form part of the original article.

Original article starts below.

In a recent project we needed to take form data and inject it into an existing Excel template — replacing placeholder variables and producing filled spreadsheets automatically. We evaluated the available .NET libraries (looking at maintenance, maturity and community support) and found only two relevant contenders for this scenario:

<PackageReference Include="ClosedXML" Version="0.105.0" />
<PackageReference Include="DocumentFormat.OpenXml" Version="3.3.0" />

We compared both ergonomics (how easy is it to open and manipulate an Excel file?) and performance (measured with BenchmarkDotNet). Below is a pragmatic walkthrough of what we did, the demo code we used, the benchmark results, and a recommendation you can reuse.

TL;DR

• OpenXmlSdk = lower-level, faster, less memory allocation → best for high-throughput, server-side processing.
• ClosedXML = higher-level, developer-friendly, faster to implement → great for prototyping and templates with formatting.
• Benchmarks show OpenXmlSdk is ~2.9–3.5× faster and uses ~1/3 of the allocations compared to ClosedXML in our tests.

A quick note about my background

I already had some experience with OpenXmlSdk — it’s powerful, but you end up “sliding through rows and columns” more compared to ClosedXML’s niceties. That background helped when we needed to optimize for performance, but it also showed why ClosedXML is so popular for day-to-day work.

What the demo does

1. Ensures a template exists (Assets/Template.xlsx).
2. Prompts the user for a set of variables (e.g. vehicleRegistration, revenueQ1, status_A, budget_A).
3. Copies the template into two files and performs token replacement using:

• a low-level OpenXmlSdk approach (replacing text inside the shared string table), and
• a high-level ClosedXML approach (iterating used cells and replacing values).

Both demo methods replace tokens like ##Revenue_Q1## etc. with provided values (or defaults).

Demo entry point (RunDemo)

public static void RunDemo()
{
    // Ensure template exists
    ExcelTemplateGenerator.EnsureTemplateExists();
    Console.WriteLine("=== Excel Replacement Demo ===");
    Console.WriteLine();

    // Collect all variables
    Console.WriteLine("Please enter value for 'Vehicle Registration':");
    var vehicleRegistration = Console.ReadLine();
    Console.WriteLine("Please enter value for 'Dashboard':");
    var dashboard = Console.ReadLine();
    Console.WriteLine("Please enter value for 'Defect Description':");
    var defectDescription = Console.ReadLine();
    Console.WriteLine("Please enter value for 'Revenue Q1':");
    var revenueQ1 = Console.ReadLine();
    Console.WriteLine("Please enter value for 'Profit Q1':");
    var profitQ1 = Console.ReadLine();
    Console.WriteLine("Please enter value for 'Status A':");
    var statusA = Console.ReadLine();
    Console.WriteLine("Please enter value for 'Budget A':");
    var budgetA = Console.ReadLine();

    var fileId = Guid.NewGuid();
    // Demo OpenXML SDK
    DemoOpenXmlSdk(fileId, vehicleRegistration, dashboard, defectDescription, revenueQ1, profitQ1, statusA, budgetA);
    // Demo ClosedXML
    DemoClosedXml(fileId, vehicleRegistration, dashboard, defectDescription, revenueQ1, profitQ1, statusA, budgetA);
    Console.WriteLine();
    Console.WriteLine("Demo completed!");
    Console.WriteLine($"Files generated:");
    Console.WriteLine($"- {fileId}_OpenXML.xlsx");
    Console.WriteLine($"- {fileId}_ClosedXML.xlsx");
}

Open XML SDK approach (low-level, precise)

This approach opens the XLSX package, iterates the shared string table and replaces placeholders directly in SharedStringItem s:

private static void DemoOpenXmlSdk(Guid fileId, string? vehicleRegistration, string? dashboard, string? defectDescription, string? revenueQ1, string? profitQ1, string? statusA, string? budgetA)
{
    var fileName = $"{fileId}_OpenXML.xlsx";
    File.Copy("Assets/Template.xlsx", fileName, true);
    Console.WriteLine($"Created a new file with ID: {fileName}");
    Console.WriteLine("Using Open XML SDK to replace variables in the Excel file...");
    Stopwatch stopwatch = Stopwatch.StartNew();
    using (FileStream fileStream = new(fileName, FileMode.Open, FileAccess.ReadWrite))
    using (SpreadsheetDocument document = SpreadsheetDocument.Open(fileStream, true))
    {
        WorkbookPart? workbookPart = document.WorkbookPart;
        WorksheetPart? worksheetPart = workbookPart?.WorksheetParts.First();
        SheetData? sheetData = worksheetPart?.Worksheet.GetFirstChild<SheetData>();
        if (sheetData == null)
        {
            Console.WriteLine("No sheet data found in the workbook.");
            return;
        }
        foreach (Row row in sheetData.Elements<Row>())
        {
            foreach (Cell cell in row.Elements<Cell>())
            {
                if (cell.CellValue is not null && cell.DataType != null && cell.DataType.Value == CellValues.SharedString)
                {
                    int id = int.Parse(cell.CellValue.InnerText);
                    SharedStringItem cellTextItem = workbookPart!.SharedStringTablePart!.SharedStringTable.Elements<SharedStringItem>().ElementAt(id);
                    string text = cellTextItem.InnerText;
                    text = text.Replace("##VehicleRegistration##", vehicleRegistration ?? "DefaultVehicle")
                        .Replace("##Dashboard##", dashboard ?? "DefaultDashboard")
                        .Replace("##DefectDescription##", defectDescription ?? "DefaultDefect")
                        .Replace("##Date##", DateTime.Now.ToString("MM/dd/yyyy"))
                        .Replace("##Revenue_Q1##", revenueQ1 ?? "450,000")
                        .Replace("##Profit_Q1##", profitQ1 ?? "85,000")
                        .Replace("##Costs_Q1##", "365,000")
                        .Replace("##Margin_Q1##", "18.9")
                        .Replace("##Revenue_Q2##", "520,000")
                        .Replace("##Profit_Q2##", "95,000")
                        .Replace("##Costs_Q2##", "425,000")
                        .Replace("##Margin_Q2##", "18.3")
                        .Replace("##Revenue_Q3##", "580,000")
                        .Replace("##Profit_Q3##", "110,000")
                        .Replace("##Costs_Q3##", "470,000")
                        .Replace("##Margin_Q3##", "19.0")
                        .Replace("##Revenue_Q4##", "620,000")
                        .Replace("##Profit_Q4##", "125,000")
                        .Replace("##Costs_Q4##", "495,000")
                        .Replace("##Margin_Q4##", "20.2")
                        .Replace("##Status_A##", statusA ?? "Completed")
                        .Replace("##Budget_A##", budgetA ?? "75,000")
                        .Replace("##Status_B##", "In Progress")
                        .Replace("##Budget_B##", "120,000")
                        .Replace("##Status_C##", "Planned")
                        .Replace("##Budget_C##", "200,000");
                    cellTextItem.GetFirstChild<Text>()!.Text = text;
                }
            }
        }
        worksheetPart?.Worksheet.Save();
    }
    stopwatch.Stop();
    Console.WriteLine($"Replacement completed in {stopwatch.ElapsedMilliseconds} ms using Open XML SDK!");
}

Why choose it: efficient, minimal allocations, fine-grained control.
Downside: more verbose and more code to maintain.

ClosedXML approach (high-level, ergonomic)

This approach opens the workbook with ClosedXML, iterates used cells and replaces values:

private static void DemoClosedXml(Guid fileId, string? vehicleRegistration, string? dashboard, string? defectDescription, string? revenueQ1, string? profitQ1, string? statusA, string? budgetA)
{
    var fileName = $"{fileId}_ClosedXML.xlsx";
    File.Copy("Assets/Template.xlsx", fileName, true);
    Console.WriteLine($"Created a new file with ID: {fileName}");
    Console.WriteLine("Using ClosedXML to replace variables in the Excel file...");
    Stopwatch stopwatch = Stopwatch.StartNew();
    using (var workbook = new XLWorkbook(fileName))
    {
        var worksheet = workbook.Worksheet(1);
        foreach (var cell in worksheet.CellsUsed())
        {
            if (cell.HasFormula) continue;
            var value = cell.GetString();
            value = value.Replace("##VehicleRegistration##", vehicleRegistration ?? "DefaultVehicle")
                         .Replace("##Dashboard##", dashboard ?? "DefaultDashboard")
                         .Replace("##DefectDescription##", defectDescription ?? "DefaultDefect")
                         .Replace("##Date##", DateTime.Now.ToString("MM/dd/yyyy"))
                         .Replace("##Revenue_Q1##", revenueQ1 ?? "450,000")
                         .Replace("##Profit_Q1##", profitQ1 ?? "85,000")
                         .Replace("##Costs_Q1##", "365,000")
                         .Replace("##Margin_Q1##", "18.9")
                         .Replace("##Revenue_Q2##", "520,000")
                         .Replace("##Profit_Q2##", "95,000")
                         .Replace("##Costs_Q2##", "425,000")
                         .Replace("##Margin_Q2##", "18.3")
                         .Replace("##Revenue_Q3##", "580,000")
                         .Replace("##Profit_Q3##", "110,000")
                         .Replace("##Costs_Q3##", "470,000")
                         .Replace("##Margin_Q3##", "19.0")
                         .Replace("##Revenue_Q4##", "620,000")
                         .Replace("##Profit_Q4##", "125,000")
                         .Replace("##Costs_Q4##", "495,000")
                         .Replace("##Margin_Q4##", "20.2")
                         .Replace("##Status_A##", statusA ?? "Completed")
                         .Replace("##Budget_A##", budgetA ?? "75,000")
                         .Replace("##Status_B##", "In Progress")
                         .Replace("##Budget_B##", "120,000")
                         .Replace("##Status_C##", "Planned")
                         .Replace("##Budget_C##", "200,000");
            cell.Value = value;
        }
        workbook.Save();
    }
    stopwatch.Stop();
    Console.WriteLine($"Replacement completed in {stopwatch.ElapsedMilliseconds} ms using ClosedXML!");
}

Why choose it: readable code, fast to implement, great for formatting and formulas.
Downside: higher memory allocations and lower throughput for bulk processing.

Benchmarks (real runs)

We ran BenchmarkDotNet for both methods. Here are the trimmed results relevant to decision-making:

+------------+-------------+----------+----------+------------+-------------+
|   Method   | Environment |   Mean   |   Gen0   | Allocated  | Alloc Ratio |
+------------+-------------+----------+----------+------------+-------------+
| OpenXmlSdk | Job-YETQRT  | 3.101 ms | 46.8750  | 317.09 KB  | 1.00        |
| ClosedXml  | Job-YETQRT  | 4.733 ms | 171.8750 | 1098.7 KB  | 3.46        |
| OpenXmlSdk | .NET 9.0    | 1.057 ms | 50.7813  | 316.62 KB  | 1.00        |
| ClosedXml  | .NET 9.0    | 3.264 ms | 171.8750 | 1099.18 KB | 3.47        |
+------------+-------------+----------+----------+------------+-------------+

Conclusion from benchmarks: OpenXmlSdk is notably faster and uses far fewer allocations. For many-file generation scenarios, that difference matters.

✅ Recommendation (practical)

Choose Open XML SDK when:

• You need the best performance and minimal allocations.
• You generate many files concurrently or work at scale.
• You don’t mind more detailed code and extra boilerplate.

Choose ClosedXML when:

• Developer speed matters and you want readable, maintainable code.
• Templates include complicated formatting / ranges you want to manipulate easily.
• Throughput demands are moderate.

Sponsored Section — A Third Option for Excel Processing with IronXL

The following content was provided as part of a paid collaboration with Iron Software and is published separately from the original article.

A Third Option for Broader Pipelines: IronXL

Worth adding a footnote to the two-contender comparison: there’s a third library that wasn’t in scope for a pure performance benchmark but fits the same developer-friendly tier as ClosedXML — with a wider output surface if template filling isn’t the end of the pipeline.

IronXL is a commercial .NET library from Iron Software. The honest performance positioning up front: it sits in the same tier as ClosedXML, not OpenXmlSdk. If raw throughput and minimal allocations are the deciding criteria, OpenXmlSdk remains the recommendation from the benchmarks above. IronXL earns its place when the project needs more than template filling from the same workbook object.

The Same Token Replacement, with IronXL

using IronXL;

WorkBook workbook = WorkBook.Load("Assets/Template.xlsx");
WorkSheet sheet = workbook.DefaultWorkSheet;

foreach (var row in sheet.Rows)
{
    foreach (var cell in row)
    {
        if (cell.HasFormula) continue;

        string value = cell.StringValue;
        value = value
            .Replace("##VehicleRegistration##", vehicleRegistration ?? "DefaultVehicle")
            .Replace("##Dashboard##",           dashboard           ?? "DefaultDashboard")
            .Replace("##DefectDescription##",   defectDescription   ?? "DefaultDefect")
            .Replace("##Date##",                DateTime.Now.ToString("MM/dd/yyyy"))
            .Replace("##Revenue_Q1##",          revenueQ1           ?? "450,000")
            .Replace("##Profit_Q1##",           profitQ1            ?? "85,000")
            .Replace("##Status_A##",            statusA             ?? "Completed")
            .Replace("##Budget_A##",            budgetA             ?? "75,000");

        cell.Value = value;
    }
}

workbook.SaveAs($"{fileId}_IronXL.xlsx");

The structure is almost identical to the ClosedXML version — same iteration pattern, same .Replace() chain, same .SaveAs(). The code volume is comparable.

Where the Scope Widens

The case for IronXL over ClosedXML isn’t template replacement — they’re equivalent there. It’s what happens after the template is filled. The same WorkBook object covers:

// After token replacement — export the same workbook in multiple formats
workbook.SaveAsCsv($"{fileId}.csv");      // downstream systems that expect CSV

// Aggregate the revenue cells that were just filled
decimal totalRevenue = sheet["C5:C8"].Sum();   // LINQ range aggregates

// Hand off to a DataTable for a SQL insert or report
DataTable dt = sheet.ToDataTable(true);

// Evaluate formulas that reference the replaced values
sheet["D9"].Formula = "=SUM(D5:D8)";

None of that needs a second library. For a pipeline where the filled template also needs to be exported, aggregated, or fed into another system — which is common in form-data-to-spreadsheet workflows like the one this article describes — that consolidated surface area reduces dependencies.

Updated Recommendation Table

Folding IronXL into the recommendation from the benchmarks:

Choose OpenXmlSdk when: maximum throughput, minimum allocations, high-concurrency file generation, you’re comfortable with more verbose code.

Choose ClosedXML when: developer speed matters, readable code is the priority, moderate throughput is fine, free/open-source is a requirement.

Consider IronXL when: the project also needs multi-format output (CSV, JSON, DataTable) from the same workbook, formula evaluation, or LINQ range aggregates — and a commercial license is acceptable.

Honest Trade-Off

Both libraries compared in the article are free. IronXL is commercial — that’s the primary reason it wasn’t in scope for a “two relevant contenders” evaluation and the primary reason to not shortlist it when open-source is a constraint. Free 30-day trial available if the broader feature set justifies evaluating it.

👉 IronXL for .NET · Excel template filling · Free 30-day trial

📊 Fast Excel Template Replacement in .NET — OpenXmlSdk vs ClosedXML was originally published in Medialesson on Medium, where people are continuing the conversation by highlighting and responding to this story.

Recently, I tried installing .NET on my Mac using the official script provided by Microsoft. While it works, I ran into a couple of small quirks that I’d like to share — so you don’t have to stumble over the same things.

Step 1: Download the Script

Microsoft provides a handy install script: dotnet-install.sh.

You can either:

• Use curl to grab it, or
• Download it directly in the browser.

⚠️ Important: Make sure the file isn’t empty — you’ll want a real dotnet-install.sh script, not a zero-byte placeholder. 😉

Step 2: Put It Somewhere Safe

I moved the script into my home directory, so it won’t get deleted accidentally when cleaning up my Downloads folder (you know how that goes…).

Step 3: Run the Script

Running the script directly will install (depending on the configuration of the tool) the current LTS (Long Term Support), which (at the time of writing) is .NET 8.

Step 4: Update Your Terminal Path

To make .NET globally available, you’ll need to update your shell config.

I’m using zsh, so I edited my .zshrc and added the following lines at the bottom:

export PATH="$HOME/.dotnet/tools:$PATH"
export PATH="$HOME/.dotnet:$PATH"

Then reload the file:

source ~/.zshrc

Now you can check if .NET is installed:

dotnet --list-sdks

Step 5: Installing .NET 9 and .NET 10 Preview

For me, it was important to have not only .NET 8, but also .NET 9 and .NET 10 Preview.

.NET 9 is easy:

./dotnet-install.sh -c 9.0

.NET 10 Preview requires one more trick: you need to specify the quality parameter.

⚠️ Heads-up: the short form -q didn’t work for me. You also need to put --quality before the -c parameter.

./dotnet-install.sh --quality preview -c 10.0

Bonus: Script Documentation 📖

If you’re curious what else the install script can do, check out the official documentation:
👉 dotnet-install script reference

Takeaway 🎯

If you’re installing .NET on macOS:

• Download the script,
• Place it somewhere safe,
• Update your shell config, and
• Use the right parameters for .NET 9 and 10.

And just like that, you’re running multiple .NET versions side by side on your Mac. 🚀

🖥️ Installing .NET on macOS with dotnet-install.sh (and a Few Lessons Learned) was originally published in Medialesson on Medium, where people are continuing the conversation by highlighting and responding to this story.

Goodbye custom converters, hello clean polymorphic JSON in .NET 7+

Ever tried to split lots of different actions into inheritance-friendly classes and interfaces, then make them configurable through a connected frontend?

Sounds easy, right?
Yeah… until you try to serialize and deserialize them with all those different generic parameters floating around. Then things get ugly. 😅

The First Attempts (a.k.a. “How to Make Your Life Harder”)

Our first approach was to just use dynamic.
 .NET: "Sure, I can do that… wait, no, I can’t make you a nice mixed list of completely unrelated types without breaking everything."

Then we tried building custom converters.
They worked, though the approach was both time-consuming and prone to errors.

The Hero Arrives: JsonPolymorphic ✨

Starting from .NET 7, System.Text.Json got a new superpower — polymorphic serialization and deserialization without having to roll your own converter factory from scratch.

Here’s the magic:

[JsonPolymorphic(TypeDiscriminatorPropertyName = "$type")]
[JsonDerivedType(typeof(Configuration), "config")]
[JsonDerivedType(typeof(Configuration2), "config2")]
public interface IConfiguration { }

public class Configuration : IConfiguration
{
    public int Number { get; set; }
    public string Name { get; set; }
    public bool IsActive { get; set; }
}

public class Configuration2 : IConfiguration
{
    public string Configuration { get; set; }
}

Serializing

var obj = new List<IConfiguration>
{
    new Configuration { Number = 42, Name = "MainConfig", IsActive = true },
    new Configuration2 { Configuration = "Special settings" }
};

var options = new JsonSerializerOptions
{
    WriteIndented = true
};

var json = JsonSerializer.Serialize(obj, options);
Console.WriteLine(json);

The output now includes a $type discriminator:

[
  {
    "$type": "config",
    "Number": 42,
    "Name": "MainConfig",
    "IsActive": true
  },
  {
    "$type": "config2",
    "Configuration": "Special settings"
  }
]

Why This Is Awesome 🚀

• No more hand-crafted converters (and their inevitable bugs).
• Works out-of-the-box with System.Text.Json.
• Keeps your interfaces intact, so your architecture stays clean.

When deserializing, .NET will look at the $type and automatically pick the right implementation for you.

TL;DR 📝

If you’ve been fighting with serializing/deserializing interfaces or abstract classes in .NET and ended up in dynamic hell or converter purgatory,
➡️ try JsonPolymorphic and JsonDerivedType.
Your future self will thank you. 🙌

🧩 Taming Generics, Interfaces & JSON in .NET — Meet JsonPolymorphic was originally published in Medialesson on Medium, where people are continuing the conversation by highlighting and responding to this story.