Asynchronous programming is the backbone of responsive applications — both in rich UIs and high-throughput server backends. But with flexibility comes a hidden cost: uncontrolled concurrency.

A user types into a search box and triggers a request on every keystroke. A backend service issues duplicate fetches because multiple workflows request the same resource at the same time. These are not edge cases — they happen constantly.

Without control, concurrent async calls lead to redundant work, race conditions, wasted compute, and unpredictable behavior. In UIs this shows up as flickering and stale data. On servers it becomes extra load and hard-to-reproduce bugs. Typical fixes — locks, cancellation tokens, ad-hoc checks — are often incomplete and fragile.

Coalescing is a more structured approach: merge overlapping async operations and define how they execute and how results are delivered. Instead of letting every request spawn its own task, you shape concurrency explicitly.

.NET 9 (finally) introduced basic task coalescing in its in-memory cache. Multiple concurrent cache loads collapse into a single execution. This is equivalent to what I call the Use First strategy: the first request runs, others await its result.

But real systems often need different behavior. Sometimes the latest request should win. Sometimes execution should wait until input stabilizes. Sometimes requests must be processed sequentially.

In this article, I’ll introduce five async coalescing strategies that cover these scenarios and discuss their trade-offs.

Coalescing Strategies Covered

1. Use First — run once, reuse the result
2. Use Last — cancel previous, keep the newest request
3. Use Last with Debounce — wait for quiet input, then execute once
4. Queue — process sequentially, return the last result
5. Aggregate — combine multiple requests during a short buffer period

Resources

1. Playground: runnable demos that simulate concurrent requests and show how each pattern behaves.
2. Video: visual explanation of concurrency timelines and pattern behavior:

Let’s break them down.

1. Use First — Run Once, Reuse Result

In this strategy, when multiple async requests target the same resource concurrently, only the first request is executed. All subsequent overlapping requests are not started — they simply wait for the first request to complete and receive its result. This ensures that no redundant work happens for overlapping calls.

This is useful when:

• You want to deduplicate overlapping requests
• The result is stable and does not need refreshing
• Re-running the operation is expensive
• Cache lookups: when fetching a cached value, the first request triggers the load, and all concurrent requests simply await the same result rather than triggering additional loads

Typical examples include startup initialization, configuration loading, or shared data fetches when multiple UI components initialize simultaneously. With Use First, concurrent overlapping requests are never executed — they simply await the result of the first request.

Use First — Pseudo Code

To understand how Use First works, we can reduce it to a minimal idea. The strategy simply keeps a reference to the currently running task. If a new request arrives while that task is still in progress, it does not start a new operation. Instead, it returns the existing task.

// Shared per resource
Task<T>? _runningTask;

public Task<T> Sync(Func<Task<T>> operation)
{
    // Thread-safety omitted for clarity

    // If a request is already running, return it.
    if (_runningTask != null && !_runningTask.IsCompleted)
        return _runningTask;

    // Otherwise start a new one.
    _runningTask = operation();

    return _runningTask;
}

• Only the first request actually starts execution.
• Overlapping requests reuse the same Task.
• No cancellation is involved.
• No concurrent execution is possible.
• Once the task completes, the next request may start a new one.

2. Use Last — cancel previous, keep the newest request

In this strategy, when multiple async requests target the same resource concurrently, only the most recent request is considered the “active” one. Previous overlapping requests are notified to cancel, but they may continue running if cancellation does not happen immediately (or at all). All callers receive the result of the most recently started request once it completes, regardless of which request they originally initiated.

This is useful when:

• You want to always prioritize the latest request
• Earlier requests become irrelevant once a newer one is issued
• Keeping previous operations running is unnecessary or costly
• Dynamic UI updates: such as real-time filtering, search-as-you-type, or slider adjustments, where only the final value matters

Typical examples include search boxes, live preview filters, or any scenario where user input changes rapidly. With Use Last, multiple requests may execute concurrently for a short time while cancellation is being processed. Previous overlapping requests may keep running (which could potentially affect unprotected resources, so be careful), but they do not affect the final result delivered to callers.

Use Last — Pseudo Code

// Shared per resource
TaskCompletionSource<T>? _completion;
CancellationTokenSource? _cts;
Task<T>? _latestTask;

public Task<T> Sync(Func<CancellationToken, Task<T>> operation)
{
    // Thread-safety omitted for clarity

    // Cancel previous execution (if any)
    _cts?.Cancel();
    _cts = new CancellationTokenSource();

    // Start new execution
    var task = operation(_cts.Token);
    _latestTask = task;

    // Create shared completion source
    _completion = new TaskCompletionSource<T>();

    task.ContinueWith(t =>
    {
        // Ignore if this is not the latest task
        if (_latestTask != t)
            return;

        if (t.IsFaulted)
            _completion.TrySetException(t.Exception!);
        else
            _completion.TrySetResult(t.Result);

    });

    return _completion.Task;
}

• A new request cancels the previous one.
• Only the latest task is allowed to complete the shared result.
• Earlier tasks may continue running if they ignore cancellation.
• All callers receive the result of the most recently started request.

3. Debounce — wait for quiet input, then execute once

In this strategy, each incoming async request starts a short waiting period (e.g., 1–2 seconds). If no new request arrives during this time, the most recent request at the end of the waiting period is executed. If a new request arrives before the waiting period ends, the timer resets.

If a request is already running when new input arrives, the running request is notified to cancel, and a new debounce waiting period begins.

This is useful when:

• You want to avoid executing multiple operations while input is rapidly changing
• Only the final value after a period of inactivity matters
• Operations are expensive and should not run unnecessarily
• User-driven input, such as search boxes, sliders, or filter adjustments

Typical examples include type-ahead search, live filtering, or form validation. During the waiting period, no execution happens. Once execution starts, a new request will always notify the running operation to cancel and begin a new debounce cycle. If the running operation does not honor cancellation, it may continue executing in parallel, so care is required if it modifies shared or unprotected resources.

Use Last with Debounce — Pseudo Code

To understand how Debounce works, we combine two ideas:

1. Wait for a short period of inactivity before executing.
2. If execution is already running and new input arrives, notify the running operation to cancel and start a new debounce cycle.

The simplified pseudo code looks like this:

// Shared per resource
CancellationTokenSource? _cts;
TaskCompletionSource<T>? _completion;
Task<T>? _runningTask;
Timer? _timer;

public Task<T> Sync(Func<CancellationToken, Task<T>> operation)
{
    // Thread-safety omitted for clarity

    // Reset debounce timer
    _timer?.Dispose();

    _completion ??= new TaskCompletionSource<T>();

    _timer = new Timer(_ =>
    {
        // Cancel currently running task (if any)
        _cts?.Cancel();
        _cts = new CancellationTokenSource();

        var task = operation(_cts.Token);
        _runningTask = task;

        task.ContinueWith(t =>
        {
            if (t.IsFaulted)
                _completion.TrySetException(t.Exception!);
            else
                _completion.TrySetResult(t.Result);

            _completion = null; // new cycle next time

        }, TaskContinuationOptions.ExecuteSynchronously);

    }, null, debounceDelay, Timeout.InfiniteTimeSpan);

    return _completion.Task;
}

• Each incoming request resets the debounce timer.
• No execution happens until the timer expires.
• When execution starts, a new request cancels the running task.
• Only the final request after inactivity produces a result.
• All callers in the same debounce cycle receive the same result.

4. Queue — process sequentially, return the last result

In this strategy, incoming async requests targeting the same resource are added to a queue and executed one at a time, in order of arrival. Each request waits for the previous one to complete before starting. When several requests are queued together, all callers wait, and only the final request’s result is delivered to everyone.

This is useful when:

• You need controlled, sequential execution to avoid race conditions or resource conflicts
• Requests must be processed in order, but only the final outcome matters
• Operations are expensive and should not run concurrently
• Spooler-like scenarios, such as writing to a shared file, updating a database, or processing sequential tasks

Typical examples include queued background jobs, batched updates, or any scenario where concurrent writes could cause conflicts. With Queue, multiple requests may be waiting while the current one is running, but execution is strictly sequential. Be careful if earlier queued operations have side effects — they will still run, even if their results are not used.

Queue — Pseudo Code

The Queue strategy ensures that requests targeting the same resource are executed strictly one at a time, in order of arrival.

Callers do not receive the result of their individual request. Instead, all callers that arrive while the queue is being processed share a single “batch” completion. When the queue becomes empty, that batch completes and everyone receives the final result from the sequence.

Below is simplified pseudo code illustrating the idea.

Queue<Func<Task<T>>> _queue = new();
TaskCompletionSource<T>? _tcs;
bool _processing;

public Task<T> Sync(Func<Task<T>> op)
{
    lock (_queue)
    {
        // Add request to the queue
        _queue.Enqueue(op);

        // Start a new batch if needed
        _tcs ??= new TaskCompletionSource<T>();

        // If not already processing, start queue worker
        if (!_processing)
        {
            _processing = true;
            _ = ProcessQueue(_tcs); // capture current batch
        }

        // All callers in this batch wait for the same result
        return _tcs.Task;
    }
}

async Task ProcessQueue(TaskCompletionSource<T> batchTcs)
{
    T lastResult = default!;

    try
    {
        while (true)
        {
            Func<Task<T>>? op;

            lock (_queue)
            {
                // If queue is empty, finish the batch
                if (_queue.Count == 0)
                {
                    _processing = false;
                    _tcs = null; // next request creates new batch
                    break;
                }

                // Dequeue next request
                op = _queue.Dequeue();
            }

            // Execute sequentially
            lastResult = await op();
        }

        // Complete batch for ALL callers
        batchTcs.TrySetResult(lastResult);
    }
    catch (Exception ex)
    {
        // If something fails, fault the entire batch
        lock (_queue)
        {
            _processing = false;
            _tcs = null;
        }

        batchTcs.TrySetException(ex);
    }
}

• Only one request executes at a time (strict sequential processing).
• Multiple callers may be waiting while the current request runs.
• Earlier queued operations still execute, even if their individual results are not returned.
• All callers that joined the same processing cycle receive the final result of that sequence.
• A new request after the queue drains starts a new batch with a new completion source.

5. Aggregate — combine multiple requests during a short buffer period

In this strategy, incoming async requests are collected for a short buffer period (e.g., 1–2 seconds). Instead of executing each request individually, their input values are accumulated using a user-defined aggregation function. When the buffer period ends, a single operation is executed using the aggregated value.

If new requests arrive while the aggregated operation is running, they are added to a new accumulator. Once the current execution completes, the next aggregated batch is processed. All callers within the same aggregation window receive the result of the combined operation.

This is useful when:

• Multiple small requests can be merged into a single larger operation
• The underlying work benefits from batching
• Reducing the number of executions is more important than immediate responsiveness
• Accumulated state (e.g., sums, collections, updates) can be safely combined

Typical examples include batching database writes, grouping telemetry events, combining UI updates, or implementing spooler-like processing where many inputs can be merged before execution. With Aggregate, only one execution happens per buffer window, but care is required if aggregated operations modify shared state. The aggregation function must be deterministic and safe, since multiple requests are combined into a single logical operation.

Aggregate — Pseudo Code

Aggregate collects incoming requests during a short buffer window (for example, 1.5 seconds), combines their values using an aggregation function, and then executes a single operation with the combined value.

If new requests arrive while the operation is running, they are accumulated for the next execution cycle.

on request(value):

    if not running:
        add value to accumulator
        reset buffer timer

    else:
        add value to nextAccumulator

when buffer timer expires:

    running = true
    result = execute(accumulator)
    complete all callers in this batch with result

    accumulator = nextAccumulator
    nextAccumulator = empty

    if accumulator not empty:
        execute immediately (no new delay)
    else:
        running = false

• Requests are grouped during a short buffer window.
• Only one execution happens per buffer cycle.
• While running, new requests form the next batch.
• When the current run finishes, the next batch runs immediately.
• Each caller receives the result of the batch they joined.

Summary of Coalescing Strategies

The five strategies differ in how they control execution order, cancellation behavior, and result delivery. The table below summarizes their core semantics.

| Strategy  | Execution Model                                       | Cancellation Behavior                                      | Concurrency                          | Result Delivery                                     | Typical Use Case                            |
| ----------| ----------------------------------------------------- | ---------------------------------------------------------- | ------------------------------------ | --------------------------------------------------- | ------------------------------------------- |
| Use First | First request runs; overlapping requests do not start | No cancellation                                            | No concurrent execution              | All callers receive first result                    | Cache loading, initialization               |
| Use Last  | Latest request becomes active                         | Previous requests are notified to cancel                   | Possible during cancellation overlap | All callers receive last result                     | Search, dynamic UI updates                  |
| Queue     | Requests execute sequentially in order                | No cancellation                                            | No concurrent execution              | All callers receive final result of the sequence    | Spoolers, ordered updates                   |
| Debounce  | Execution delayed until inactivity period ends        | Running request is notified to cancel if new input arrives | Possible during cancellation overlap | All callers receive final result after quiet period | Rapid user input, sliders, filters          |
| Aggregate | Requests are buffered and combined before execution   | No implicit cancellation between windows                   | One execution per buffer window      | All callers receive aggregated result               | Batching writes, telemetry, grouped updates |

FlowSync — Applying Coalescing as a Strategy

The patterns described above are not theoretical. They are implemented in FlowSync, a small library that formalizes async coalescing as an explicit strategy.

Instead of manually wiring TaskCompletionSource, CancellationTokenSource, queues, timers, and locking logic for every use case, FlowSync separates:

• Business logic (what the operation does)
• Concurrency semantics (how overlapping calls behave)

This separation is the key design principle of the library.

Repository: https://github.com/0x1000000/FlowSync

Core Idea

A method is written as a normal async operation, but returns a FlowSyncTask<T>:

public async FlowSyncTask<int> FetchAsync(int id)
{
    var ctx = await FlowSyncTask.GetCancellationContext();
    // ctx.CancellationToken covers both:
    // 1) explicit external cancellation
    // 2) strategy-enforced cancellation due to coalescing/overlap
    // ctx.IsCancelledLocally is true only for case (2).
    
    // Do work here and respect ctx.CancellationToken if needed.
    await Task.Delay(Random.Shared.Next(50, 501), ctx.CancellationToken);
    return id + 42;
}

The method itself contains no synchronization logic.
It simply respects the cancellation context provided by FlowSync.

Then, at the call site, you choose the strategy:

// Strategy keeps per-group coalescing state across calls, so share one instance.
static readonly IFlowSyncStrategy<int> Strategy = new UseLastCoalescingSyncStrategy<int>();

public async Task<int> CallerAsync(int id)
{
    // CallerAsync can be invoked concurrently (for example from different threads).
    // For the same groupKey, each call awaits strategy resolution and completes when
    // the coalesced pipeline finishes (or is canceled); a given invocation may never start its own FetchAsync execution.
    return await FetchAsync(id).CoalesceInGroupUsing(Strategy, groupKey: id);
}

Each coalescing pattern is implemented as a strategy:

• UseFirstCoalescingSyncStrategy
• UseLastCoalescingSyncStrategy
• QueueCoalescingSyncStrategy
• DeBounceCoalescingSyncStrategy
• AggCoalescingSyncStrategy

All strategies follow the same abstraction but differ in semantics:

• Should previous calls be ignored?
• Should they be cancelled?
• Should execution be delayed?
• Should calls be queued?
• Should inputs be aggregated?

The operation does not change. Only the synchronization policy changes (Aggregation is a small exception).

This makes concurrency behavior explicit and configurable instead of implicit and scattered across code.

Asynchronous concurrency is not just a technical detail — it shapes how systems behave under real load. Without explicit control, overlapping async calls lead to race conditions, wasted work, and unpredictable results. Coalescing introduces structure: it defines which requests matter, when execution happens, and how results are delivered. Use First, Use Last, Queue, Debounce, and Aggregate are not the only possible strategies, but they represent common and practical patterns that cover many real-world scenarios. Making the chosen behavior explicit makes async code easier to reason about, easier to test, and more reliable.

Links

• FlowSync GitHub Repository: https://github.com/0x1000000/FlowSync
• Interactive Playground (Live Demo): https://0x1000000.github.io/FlowSync/
• YouTube Video Version: https://www.youtube.com/watch?v=wwSU83Qpjts

5 Common Async Coalescing Patterns was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

Polymorphism is a fundamental concept in object-oriented programming and serves as the basis for many commonly used design patterns in software development. However, in runtimes that use garbage collection, such as .Net, Java, and Go, many common design patterns that imply using polymorphism can have some overhead from a performance perspective, as they often entail memory allocations with consequent garbage collection cycles, which in turn can be crucial for high-load applications.

The initial approach that comes to mind for avoiding memory allocation is to utilize structures located on the stack. However, a critical issue arises: the structure data (when on the stack) does not inherently form an actual object with a link to its type and a list of virtual functions. Consequently, this limitation prevents the direct use of structures in situations that involve polymorphic calls, as they necessitate late binding performed through the virtual function table.

Nevertheless, C# offers a workaround that enables structures to implement interfaces, allowing them to be used in situations where actual classes would otherwise be necessary. This workaround is known as “boxing”. The concept behind boxing is that whenever a structure is cast to its interface (implicitly or explicitly), .NET creates an object on the heap and copies the structure data into it. This object now possesses a link to its type and, consequently, a link to the virtual function table, enabling polymorphic calls.

While the formal goal is achieved, the performance consequences are critical. In addition to the fact that memory allocation still occurs, it can now happen every time a structure is cast to its interface. Given that this allocation occurs implicitly, detecting this performance glitch can be challenging. I have even come across recommendations for beginning C# programmers not to use structures, since the likelihood of ruining performance is quite high.

However, there is a technique that prevents .NET from boxing and can ultimately lead to a significant performance boost. If the sole purpose of boxing is to establish a link to the type, it can be avoided by passing the link through other means, such as using a generic parameter.

So instead of a function contract like this:

void Do(ISomeInterface interfaceImplementation) {
    interfaceImplementation.SomeMethod();
}

the following contract should be used:

void Do<TSome>(ref TSome interfaceImplementation) where TSome : ISomeInterface {
    interfaceImplementation.SomeMethod();
}

Now we can pass a structure to the method, and no boxing will occur, as the virtual function table can be obtained from the generic parameter TSome.

To illustrate the concept, let’s examine an example in which the well-known design pattern ‘Visitor’ is implemented using both classes and structures, considering the implications associated with each version.

Example

I frequently utilize the “Visitor” design pattern to safely switch over an enumeration. I have even created a source generator that automatically implements this pattern.

The concept is as follows: there exists an enumeration, along with an interface that includes a corresponding method for each enumeration member:

enum Color
{
    Red,
    Green,
    Blue
}

interface IColorVisitor<out T, in TArg>
{
    T CaseRed(TArg arg);
    T CaseGreen(TArg arg);
    T CaseBlue(TArg arg);
}

There exists a helper method that receives a value from the enumeration and an interface implementation. This method checks the value and invokes the corresponding interface method:

static T Accept<T, TArg>(this Color source, IColorVisitor<T, TArg> visitor, TArg arg)
{
    switch (source)
    {
        case Color.Red:
            return visitor.CaseRed(arg);
        case Color.Green:
            return visitor.CaseGreen(arg);
        case Color.Blue:
            return visitor.CaseBlue(arg);
        default:
            throw new ArgumentOutOfRangeException(nameof(source), source, null);
    }
}

The advantage of this approach is that the ‘switch’ operator can be isolated from the actual logic, resulting in cleaner and safer code.

Now let’s create a generic version of the method:

static T Accept<T, TVisitor, TArg>(this Color source, ref TVisitor visitor, TArg arg)
    where TVisitor : IColorVisitor<T, TArg>
{
    switch (source)
    {
        case Color.Red:
            return visitor.CaseRed(arg);
        case Color.Green:
            return visitor.CaseGreen(arg);
        case Color.Blue:
            return visitor.CaseBlue(arg);
        default:
            throw new ArgumentOutOfRangeException(nameof(source), source, null);
    }
}

and run the benchmark for each situation:

[MemoryDiagnoser]
public class Program
{
    public static void Main()
    {
        BenchmarkRunner.Run<Program>();
    }

    const int Iterations = 1000;

    [Benchmark]
    public void Class()
    {
        var visitor = new VisitorClass();
        for (int i = 0; i < Iterations; i++)
        {
            Color.Green.Accept(visitor, true);
        }
    }

    [Benchmark]
    public void StructBoxing()
    {
        var visitor = new VisitorStruct();
        for (int i = 0; i < Iterations; i++)
        {
            Color.Green.Accept(visitor, true);
        }
    }

    [Benchmark]
    public void StructNoBoxing()
    {
        var visitor = new VisitorStruct();
        for (int i = 0; i < Iterations; i++)
        {
            Color.Green.Accept<string, VisitorStruct, bool>(ref visitor, true);
        }
    }
}

public class VisitorClass : IColorVisitor<string, bool>
{
    public string CaseRed(bool eng) => eng ? "Red" : "Rojo";

    public string CaseGreen(bool eng) => eng ? "Green" : "Verde";

    public string CaseBlue(bool eng) => eng ? "Blue" : "Azul";
}

public struct VisitorStruct : IColorVisitor<string, bool>
{
    public string CaseRed(bool eng) => eng ? "Red" : "Rojo";

    public string CaseGreen(bool eng) => eng ? "Green" : "Verde";

    public string CaseBlue(bool eng) => eng ? "Blue" : "Azul";
}

The results are following:

Utilizing structures with boxing yields significantly worse results, particularly from a memory allocation perspective. It consumes 1000 times more memory, as allocation occurs during each iteration. Conversely, the version without boxing incurs no allocations and runs much faster than any other version.

Note: The performance boost visible in the “StructNoBoxing” version, compared to the “Class” version, is attributed to the fact that the JIT compiler aggressively inlines method invocations with struct parameters.

Internals

Examining the IL code clarifies the mechanics of polymorphic calls with structures:

static T Accept<T, TVisitor, TArg>(this Color source, ref TVisitor visitor, TArg arg)
    where TVisitor : IColorVisitor<T, TArg>
{
    switch (source)
    {
...
        case Color.Green:
    // IL_00xx: constrained. !!1/*TVisitor*/
    // IL_00xx: callvirt     instance !0/*T*/ class EnumVisitorGenerator.IntegrationTests.IColorVisitor`2<!!0/*T*/, !!2/*TArg*/>::CaseGreen(!1/*TArg*/)
            return visitor.CaseGreen(arg);
...

    }
}

The constrained prefix instructs the callvirt instruction to utilize the virtual function table of the specified type. If the type is a value type, the pointer to the beginning of the structure (located on the stack) serves as the ‘this’ pointer, eliminating the need for boxing.

— — — —

While the technique described here cannot be universally applied as it functions only when the type is known in advance and the scope is confined to a synchronous function call. Furthermore, the resulting code lacks clarity and requires close attention to avoid performance degradation. Nevertheless, it remains applicable in many situations where performance is of utmost importance.

Source code: https://github.com/0x1000000/EnumVisitorGenerator/blob/main/EnumVisitorGenerator.IntegrationTests/Program.cs

Achieving Allocation-Free Polymorphism in C# was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

Without a doubt, Typescript provides a wide range of possibilities for describing object types. In fact, the range is so extensive that even experienced developers may find it challenging to comprehend what the possibilities are for. For a long time, I didn’t understand why Typescript has such a complex type system until I encountered a practical task where Typescript’s capabilities proved to be extremely useful.

I am developing a state management library that create an immutable snapshot of the properties of some class when at least one of its properties has been changed. This snapshot is compared with the previous one, and if the library detects that a property associated with some modifier (essentially a static function) has changed, then that modifier is called with the current snapshot as an argument. The result of this call applies to the properties of the class. Here is an example of how it can look like:

class Component {
  constructor(
     readonly $userId: Observable<number>,
     readonly userService: UserService) {

     initializeImmediateStateTracking<Component>(this);
  }

  firstName: string;

  lastName: string;

  readonly greetings = new Subject<string>();

  @With<Component>('firstName', 'lastName')
  static greet(state: ComponentState<Component>): StateDiff<Component> {
    return {
      greetings: `Hi, ${state.firstName} ${state.lastName}`
    };
  }

  @WithAsync<Component>('$userId')
  static async loadUser(getState: AsyncContext<Component>)
    : Promise<StateDiff<Component>> {

     const state: ComponentState<Component> = getState();

     if(state.$userId) {
       const userData = await state.userService.getById(state.$userId);

       return {
          firstName: userData.firstName,
          lastName: userData.lastName  
       };
     }
     return null;
  }
}

The most interesting part (from the point of view of the topic) is the types ComponentState<T> and StateDiff<T> which transform the original class T in the following way:

ComponentState<T>
• All methods are filtered out.
• All subjects and observables are unwrapped.
• All remaining properties are marked as read-only.
so that

type ComponentState<Component> = {
  readonly $userId: number;
  readonly userService: UserService;
  readonly firstName: string;
  readonly lastName: string;
  readonly greetings: string;
}

StateDiff<T>
• All methods are filtered out.
• All observables are filtered out.
• All subjects are unwrapped.
• All remaining properties are marked as optional.
so that

type StateDiff<Component> = {
  userService? UserService;
  firstName? string;
  lastName? string;
  greetings? string;
}

Note: Typescript does not perform the transformation it just reflects that the library does

Next, I will show how such type conversions can be performed.

First of all, I want to share my remark that Typescript actually consists of two languages: the first is a well-known JavaScript extension that adds typing capabilities, the second is a type conversion language with its own rules and constructions that do not affect the JavaScript runtime in any way. In what follows, only this second “language” will be discussed.

Typescript provides a list of type conversion (or type manipulation) features which are well described in its documentation. Here I will just briefly overview the most important of them:

keyof T

This operator generates a union of literal types where each type corresponds to a property key of the type.

type T = keyof { p1: string, p2: number, p3: ()->void } 
//equals
type T = 'p1' | 'p2'| 'p3'

Type Indexing

If we take a look at a typical type definition, it can resemble a JSON initialization of a JavaScript object where the value of a property is its type:

type User = {
  userId: number;
  firstName: string;
  lastName: string;
  dob: Date;
}

Typescript takes advantage of this similarity and enables you to obtain the type of a property by indexing it as if it were a regular JavaScript object:

type T1 = User['userId']
//equals
type T1 = 'number'

type T2 = User['firstName']
//equals
type T2 = 'string'

type T3 = User['dob']
//equals
type T3 = Date

However, this type indexing has more capabilities than the JavaScript one, allowing you to index multiple items simultaneously and receive a union of types:

type T = User['userId'|'firstName'|'lastName'|'dob']
//equals
type T = User[keyof User]

//equals
type T = number|string|Date

Conditional types

Typescript’s data manipulation language includes a ternary operator that can be used in the following way:

TypeA extends TypeB ? TypeExpression1 : TypeExpression2

The TypeA extends TypeB operator can be thought of as a boolean expression that verifies whether the following statements are true: TypeA is equivalent to TypeB, or TypeB is assignable to TypeA, since TypeB is a subset of TypeA.

TypeExpression can be an exact type or any other type manipulation expression, such as another conditional type.

Example:

type TypeA = {
    typeAid: string;
}

type TypeB = {
    typeBid: number;
}

function getId<T extends TypeA|TypeB>(obj: T): T extends TypeA 
    ? T['typeAid'] 
    : T extends TypeB 
        ? T['typeBid'] 
        : never {

     return ((obj as TypeA).typeAid ?? (obj as TypeB).typeBid) as any;
}

const idS: string = getId({typeAid: 'id5'});
const idN: number = getId({typeBid: 5});

The function’s resulting type depends on an object that is passed as an argument.

Mapped Types

The type mapping expression is the most powerful feature of the type manipulation language. It allows you to generate almost any type using a union of types as an input argument. In pseudo language, this expression can be represented as a pure function like this:

TOutput = <mapping expression>(<some union of types>, … other types or generic parameters)

or more precisely

TOutput = {
   <+-readonly> fk(x1, ...other types)<+-?>: ft(x1, ...other types)
   <+-readonly> fk(x2, ...other types)<+-?>: ft(x2, ...other types)
...
   <+-readonly> fk(xN, ...other types)<+-?>: ft(xN, ...other types)
}

where

• x — is an item of the union of types
• fk — is an expression that returns a type property key that should be on the following types: symbol, string, number or never * (in this case the output type will not contain a property for the item of the union of types)
• ft — is an expression that returns a type property key

The type mapping expression has the following syntax in Typescript:

{  
  <optional key readonly modifier>
  [<type tag> in <union of types> as <property key expression>]
    <key optionality modifier>: <property type expression>
}

where

• <union of types> is any expression that returns a type union e.g.:

'p1'|'p2'|'p3'
//or
keyof T
//or
Dog|Cat

• <type tag> is an identifier that represent an item in the union of types (it corresponds to “x” in the pseudo language definition)
• <property key expression> is an expression that generates a property name from an item of the input union of types represented by the type tag (it corresponds to “fk” function in the pseudo language definition). In this expression you can use conditional types and string interpolation (see the examples below). It is optional unless an item of the input union is not string literal type, in this case the expression must return some string literal type.
• <property type expression> is an expression that generates a property type from an item of the input union of types represented by the type tag (it corresponds to “fk” function in the pseudo language definition). In this expression you can use any Typescript expression that returns a type (e.g. generics, exact types, conditional types etc.)
• <key readonly modifier> can be +readonly or -readonly. It ads or removes readonly modifier for a property key
• <key optionality modifier> can be +? or -? . It ads or removes “?” modifier for a property key.

Example #1

type T = { [K in 'p1'|'p2'|'p3' ]: number }

//equals

type T = {
    p1: number;
    p2: number;
    p3: number;
 }

where

• <union of types> is ‘p1’|’p2'|’p3'
• <type tag> is K
• <property key expression> is omitted since type tag K already represents the required string literal type.
• <property type expression> is number

Example #2

type T = { [K in keyof User as `set${Capitalize<K>}` ]: (value: User[K]) => void }

//equals

type T2 = {
    setUserId: (value: number) => void;
    setUserFirstName: (value: string) => void;
    setUserLastName: (value: string) => void;
    setUserBob: (value: string) => void;
 }

where

• <union of types> is keyof User which is the same as ‘userId’|‘fistName’|‘lastName’|‘dob’
• <type tag> is K
• <property key expression> is `set${Capitalize<K>}`, which is an example of using string interpolation and intrinsic string manipulation functions to generate property keys of the resulting type.
• <property type expression> is (value: User[K]) => void, which is an example of type indexing.

Example #3

type WithTypeTag = {
    $type: string;
}

type Cat = {
    $type: 'cat';
}

type Dog = {
    $type: 'dog';
}

type Visitor<T> = { 
    [K in T as K extends WithTypeTag ? `visit${Capitalize<K['$type']>}` : never]
    : (value: K) => void 
}

type T = Visitor<Cat|Dog|User>;

//equals

type T = {
    visitCat: (value: Cat) => void;
    visitDog: (value: Dog) => void;
}

• <union of types> is Cat|Dog|User (through the generic parameter). Unlike the previous examples, this union is comprise of regular types, but not string literals.
• <type tag> is still K which is one of Cat|Dog|User.
• <property key expression> is K extends WithTypeTag ? `visit${Capitalize<K[‘$type’]>}` : never. It checks if a type has the type tag property. If the property is found then its type is used to generate a visitor method name, otherwise nothing will be emitted into the resulting type (the keyword is never).
• <property type expression> is (value: K) => void. Since the union is comprise of regular types then it is possible to use the tag as it is (the type indexing is not required here).

Solutions

Now that we have a basic understanding of type manipulation in Typescript, we can use this knowledge to solve the tasks that were introduced at the beginning of this post:

ComponentState<T>
• All methods are filtered out:

type WithoutFunctions<T> = 
    {[K in keyof T as T[K] extends Function ? never : K]: T[K]}

• All subjects and observables are unwrapped:

type Unwrap<T> = {
    [K in keyof T]: T[K] extends Observable<infer TO> 
        ? TO 
        : (T[K] extends Subject<infer TS> ? TS : T[K])
};

• All remaining properties are marked as read-only:

type Readonly<T> = {
    +readonly [K in keyof T]: T[K]
};

All together:

type ComponentState<T> = {
    +readonly [K in keyof T as T[K] extends Function ? never : K]
        : T[K] extends Observable<infer TO>
            ? TO
            : T[K] extends Subject<infer TS> ? TS : T[K]
};

type T = ComponentState<Component>;

//equals

type T = {
    readonly $userId: number;
    readonly userService: UserService;
    readonly firstName: string;
    readonly lastName: string;
    readonly greetings: string;
}

Similar for StateDiff<T>:

type StateDiff<T> = {
    -readonly [
        K in keyof T as 
        T[K] extends Function ? never 
          : T[K] extends Observable<any> ? never: K 
     ]+?: T[K] extends Subject<infer TS> ? TS : T[K]
};

type T = StateDiff<Component>;

//equals

type T = {
    userService?: UserService;
    firstName?: string;
    lastName?: string;
    greetings?: string;
}

—

While Typescript’s type manipulation language can be challenging to understand, it can be very useful in certain situations. For example, if you’re creating a library that other developers will use and may not be familiar with your codebase, TypeScript can help ensure that your code is more robust and less error-prone. It’s important to note that the type manipulation language does not affect JavaScript output and is only used during the compilation process.

Links:

ng-set-state — is the state management library described in this post

Advanced Typescript Techniques: State Tracking as an Example was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

In development, there is a very common situation when it becomes necessary to work with dynamic attributes for some entities, even more — the entities themselves can be fully dynamic. I think the most famous example of such application is JIRA where ticket attributes can be added or removed by administrators and then everybody will be able to work with them (review or modify their values for tickets). At the same time, JIRA provides ample opportunities for filtering and sorting tickets using the dynamic attributes which suggests that work with the dynamic attributes is deeply integrated into JIRA data storage, otherwise it would hardly be possible to achieve good performance when dealing with a large number of objects. For example, if there are thousands (millions) of stored objects (e. g. JIRA tickets) and if the filtering were not implemented in the data storage itself, then it would be necessary to read each of the objects into the application memory to check if it matches the filtering and sorting conditions. Obviously, it does not look very efficient.
I encountered a similar problem in several projects that I worked on, and using that experience I want to offer a solution that is quite effective, in my opinion.
Note. In the following, we will focus only on SQL databases. Most likely, many will agree with the statement that SQL databases nowadays are still the most popular way to store application data.

Table Structure

Before filtering data, you must first figure out how to properly organize data in SQL tables. In the simplest scenario table structure for dynamic attributes might look as follows:

• Product — List of objects which are going to be extended with dynamic attributes
• Attribute — list of dynamic attributes (name and type)
• ProductAttribute — dynamic attribute values for the products

Let’s look at an example where we have a list of mobile phone models and we want to dynamically add some additional attributes, such as ‘vendor’, ‘internal memory (Gb)’, ‘cellular protocols’, ‘release date’ etc.

In this case, the data might look like this:

There is one point in this simple structure that can cause certain problems — note that the type of the Value column is a string. At first glance, it seems logical since the attribute type can be different, but any of the types can always be represented as a string. However, in terms of filtering and sorting, this is not the best choice, because in general, string comparison does not correspond to encoded value comparison. So for example, when comparing strings ‘2.11’ and ‘11.2’ and comparing numbers 2.11 and 11.2, the exact opposite result will be obtained — ‘2.11’ is “greater” than ‘11.2’ but 2.11 is less than 11.2

Another problem with using strings is that the same data can be encoded differently. I think that many developers have met with problems caused by different date formats: 5/29/2022, 2022–05–22, 05.29.2022 — the same date, but different strings.

Often, attribute values can be more complex structures, such as lists, and if they are encoded as a single string, such values become almost unmanageable at the database level.

With the above in mind, consider the new table structure:

Now the value table (ProductAttribute) has one column for each attribute type (reference, number, date). List values are moved into a separate table ProductAttributeItem.

Using the structure, we can try to select some products that meet certain criteria.

SQL Filtering

Let’s consider a “complex filter”, for example:

In SQL it can be expressed as the :

(
  [1]/*Vendor*/ = 1/*Apple*/ 
  AND
  [2]/*Internal Memory*/ >= 64
  AND 
  [2]/*Internal Memory*/ <= 256
)
OR
([1]/*Vendor*/ = 2/*Samsung*/)

The idea is to inject the expression into some SQL query so that the database server does the filtering on its own. In order to do this it needs:

1. Analyze what attributes are used in the filter
2. Join a value table (ProductAttribute) as many times as many unique attributes are present in the filter

The filter contains two attributes:

• [1] Vendor — reference to the enumeration
• [2] Internal Memory — number

So, the final query should contain a sub query where ProductAttribute is joined two times with proper column aliases:

SELECT
    [ATTRIBUTES].ProductId
FROM
(
    SELECT
        [P].ProductId,
        [AT_1].ValueItem [1],
        [AT_2].ValueInt [2]
    FROM Product
        [P]
    LEFT JOIN ProductAttribute
        [AT_1] ON
        [AT_1].AttributeId = 1/*Vendor*/
        AND
        [AT_1].ProductId = [P].ProductId
    LEFT JOIN ProductAttribute
        [AT_2] ON
        [AT_2].AttributeId = 2/*Internal Memory*/
        AND
        [AT_2].ProductId = [P].ProductId
) [ATTRIBUTES]
WHERE
(
  [1]/*Vendor*/ = 1/*Apple*/ 
  AND
  [2]/*Internal Memory*/ >= 64
  AND 
  [2]/*Internal Memory*/ <= 256
)
OR
([1]/*Vendor*/ = 2/*Samsung*/)

As we can see, the original filter expression (like any other with the same attributes) can be injected into the query and the filtering will be performed directly in the SQL database. In addition to the fact that this filtering will be very efficient (we will add indexes, right?), we also get the ability to select a range in a sorted data set (pagination with FETCH OFFSET), which would require all the data to be in the application’s memory if filtering was done at the application level.

Building dynamic SQL

Theoretically, this looks good, but it is not clear how to use this approach in a real application, since the following tasks must be solved first:

1. Converting filtering criteria into a SQL boolean expression
2. Preparing a proper base selection query with all required joins

Hardly your ORM is flexible enough to perform these tricks and the first idea that comes to mind is to build dynamic SQL queries as text, which is possible of course but I would recommend using some sql builders that allow working with SQL syntax trees — that would be much safer and flexible solution.

Further, as an example, I will use SqExpress library for the .Net platform. Of course, you can use other similar libraries — the principles will remain the same.

So let’s start by building a boolean expression:

Note: here it is hard-coded but nothing prevents to create it dynamically from some filter model.

Small check that it is what we want:

Console.WriteLine(TSqlExporter.Default.ToSql(filter));

//[1]=1 AND [2]>=64 AND [2]<=256 OR [1]=2

Using syntax tree traversal we can find all unique attributes ids in the expression:

When we know all the attribute ids we can read their types:

Now using the information about attributes in the filter expression we can build a sub-query where ProductAttribute table will be joined once per an unique attribute:

Combining all together:

This works fine with simple attributes, but if an attribute is “Set” (value is a set of items), then more effort will be required. For example, we want to filter by cellular protocols:

To be executed by the database server, this filter query must be modified as follows: IN operator should be replaced with EXISTS sub-query, so that ProductAttributeItem table is used. It can be done by modification the filter syntax tree:

EXISTS
(
  SELECT 1 
  FROM [dbo].[ProductAttributeItem] 
    [A0] 
  WHERE 
     [A0].[AttributeId]=3 
     AND 
     [A0].[ProductId]=[A1].[ProductId] 
     AND 
     [A0].[AttributeItemId] IN(3,5)
)

After the modification the filter can be used in the query:

Using the techniques and approaches described above, you can not only create applications with dynamically expanding entities, but also provide a rich functionality for filtering these entities according to various criteria.

Links:

• SqGoods — it is a demo web application that demonstrates dynamic filtering of dynamic entities;
• Source code code for this article
• SqExpress — sql query builder used in the article examples. (Video Tutorial on Youtube)

Filtering by Dynamic Attributes was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

There are a lot of control types that can be considered as popup controls, e.g. Select, Date-Time picker, Modal dialog etc. All these controls have parts that break out of the layout flow and overlap other UI elements. There are well-known methods for achieving the overlap behavior in HTML, but when developing your own component library in Angular, using these methods can be problematic. Next, we’ll take a look at potential problems with pop-up controls and some tricks to avoid them.

Overlays in HTML

The most simple and efficient way to create a local-pop is using relative position for a host element and absolute position for a popup:

<div id="host">
  <div id="control">
    Host
    <div id="popup">
      Popup
    </div>
  </div>
</div>
<style>
  #host{
    width: 90px;
    height: 50px;
  }

  #control{
    position: relative;
  }

  #popup{
    position: absolute;
    top: 30px;
    left: 10px;
  }
</style>

It works fine if the host element does not hide overflow, otherwise the popup might be clipped, and z-index will not help:

#host{
  width: 90px;
  height: 50px;
  overflow: hidden;
}

#control{
  position: relative;    
  z-index: 10000; /*It does not help*/
}

Library controls can be placed in any host, so the solution described above cannot be treated as universal.

Another solution is using fixed position for a popup:

<div id="host">
  <div id="control">
    Control
      <div id="popup">
        Popup
      </div>
  </div>  
</div>
<style>
  #host{
    width: 90px;
    height: 50px;
    overflow: hidden;
  }

  #popup{
    position: fixed;
    top: 50px;
    left: 50px;
  }
</style>

The disadvantage of this method is that the popup and control positions need to be synchronized via JavaScript, but the main problem is that the control host can initiate a new stacking context and, as a result, the popup can be overlapped by other elements and z-index again will not help:

<div id="host">
  <div id="control">
    Control
      <div id="popup">
        Popup
      </div>
  </div>  
</div>
<div id="overlap">
  Overlap
</div>
<style>
  #host{
    width: 90px;
    height: 50px;
    overflow: hidden;

    /*New stacking context*/
    position: relative;
    z-index: 0;
  }

  #popup{
    position: fixed;
    top: 50px;
    left: 35px;
    z-index: 1000;/*It does not help*/
  }

  #overlap{
    transform: translate(80px, -50px);
  }
</style>

The solution here is to render the popup in the root stacking context:

<div id="host">
  <div id="control">
    Control
  </div>  
</div>
<div id="overlap">
  Overlap
</div>
<div id="popup">
  Popup
</div>

This trick works, but what should the hapless creators of Angular controls do? These controls can be placed anywhere and how can their popups be moved after an application root? Further, I will show you how to do this.

Implementation in Angular

The first goal is rendering a popup part anywhere else except for the control itself.

For example we are developing a drop-down control, and we want the “down” part to be rendered after the application root component:

<div class="l-box">
  <ng-content select="lib-drop-down-label">
  </ng-content>
</div>
<div class="l-popup">
  <ng-content select="lib-drop-down-content">
  </ng-content>
</div>

Using an angular structural directive we can catch the popup part as a template and store it in some service:

The Service:

This is how the detective is used:

...
<div
  *libPopupRect="isOpen;top:rect.top;left:rect.left;width:rect.width;"
    class="l-popup">

<ng-content select="lib-drop-down-content">
  </ng-content>
</div>

Now we need a host component which will render the stored templates:

<ng-container *ngFor="let p of popups">
  <div 
    class="l-popup"
    [style.top.px]="p.rect.top" 
    [style.left.px]="p.rect.left" 
    [style.bottom.px]="p.rect.bottom" 
    [style.right.px]="p.rect.right" 
    [style.width.px]="p.rect.width" 
    [style.height.px]="p.rect.height"
    [style.z-index]="p.rect.zIndex"
  >
    <ng-container [ngTemplateOutlet]="p.tr"></ng-container>
  </div>
</ng-container>
<style>
  .l-popup {
    position: fixed;
    pointer-events: initial;
  }  
</style>

The component class:

@Component(...)
export class PopupHostComponent {

  popups: PopupData[];
    
  constructor(private readonly _cd: ChangeDetectorRef) { }

  update(popups: PopupData[]) {
    this.popups = popups;
    this._cd.detectChanges();
  }
}

Now, using the service, we can dynamically create the host component after the application root:

In DOM the opened drop-down will look as follows:

Positioning of popups

The popups use the fixed position and that means that they should be explicitly positioned relative to an anchor:

For the drop down we can use the component element ref as an anchor and position the popup when it appears:

The problem here is that we can not keep the drop-down always opened since any change in the host DOM layout (resizing, scrolling etc.) might lead to incorrect position. The simplest solution of the problem is closing the drop-down popup on any external activity:

Other popup controls can also be developed in the same manner. For example Modal Dialog.

—

The technique described above might look a bit over-complicated but it is a good solution to be 99% sure that your popups will not be overlapped by anything else.

Link to the example on stackblitz: https://stackblitz.com/edit/angular-popup-modals

Creating Popup Controls in Angular was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

In the previous article(“Angular Components with Extracted Immutable State”), I explained why changing component fields without any restrictions is not always good, and also presented a library that allows you to streamline a component state changes.

Since then, I’ve slightly modified its concept and made it easier to use. This time I’ll focus on a simple (at first glance) example of how it can be used in scenarios where rxJs usually would be needed.

Concept

There is an immutable object that represents all field values of some component:

Each time when some field value (or several field values) of the component is changed, then a new immutable object will be created. It will contain all the old unchanged values and the new ones:

By comparing these two objects, it can be determined what exact fields values have changed and if there are logical dependencies on these fields then the corresponding value evaluation should be performed. After the evaluation is done then a 3-rd object will be created and this 3-rd object will contain the original values and the newly evaluated ones.

Now we have the new object which also can be compared with the previous one and next dependencies can be evaluated if it is required. This scenario can be repeated many times until we have a fully consistent object. After that angular will render its data. Profit!

Simple Greeting Form

Let’s create a simple greeting form (source code on stackblitz):

Obviously, the greeting field depends on the userName field and there are several ways to express the dependency:

1. Converting greeting to a property with a getter, but in this case its value will be calculated on each change detection.
2. Converting userName to a property with a setter which will update the greeting filed value.
3. Creating an event handler on “(ngModelChange)” but it will make the markup more complex;

They will work, but if some other field depends on greeting (for example, “greetings counter”) or “greeting” depends on more than one field (e.g. greeting=f(userName, template) ), then none of these methods will help, so another approach is proposed:

First, the component should be decorated with @StateTracking decorator, or alternatively initializeStateTracking function should be called in the constructor (custom component decorators do not properly works in some old Angular versions):

@StateTracking decorator (or initializeStateTracking function) finds all the component fields which other fields might depend on and replaces them with properties with getters and setters, so that the library can keep track changes.

Next, some transition function should be defined e.g.:

Each transition function receives an object that represents a current component state and should return an object which will contain only updated fields. It will be merged into a copy of the current state object. Then the new updated copy will become a new component state.

Optionally, you can add a second argument which will receive a previous state object.

If you define a third parameter, it will receive a “difference” object between the current and previous states:

ComponentState and ComponentStateDiff are typescript mapped types which filter out methods and event emitters. Also ComponentState marks all fields as readonly (state is immutable) and ComponentStateDiff marks all fields as optional, since transition function can return any subset of the original state.

For simplicity let’s define the type aliases:

Decorator @With receives a list of field names whose value changes will trigger the decorated static (!) method. Typescript will check that the class has the declared fields and that the method is static (transitions should “pure”)

Tracking log

Now the form displays a corresponding greeting when user types a name. Let’s see how the component state changes:

onStateApplied is a hook function which is called each time when the component state becomes consistent — that mans that all transition functions have been called and no more changes detected:

Transition:
{} =>
{"userName":"B","greeting":"Hello, B!"}

Transition:
{"userName":"B","greeting":"Hello, B!"} =>
{"userName":"Bo","greeting":"Hello, Bo!"}

Transition:
{"userName":"Bo","greeting":"Hello, Bo!"} =>
{"userName":"Bob","greeting":"Hello, Bob!"}

As we can see, the component goes to a new state every time user types a next character and the greeting field is updated immediately. If it needs to prevent the greeting from updating on every name change then it can be easily accomplished by adding Debounce extension to the @With decorator:

Now the library waits 3 second after the last name change and only then executes the transition:

Transition:
{} =>
{"userName":"B"}

Transition:
{"userName":"B"} =>
{"userName":"Bo"}

Transition:
{"userName":"Bo"} =>
{"userName":"Bob"}

Transition:
{"userName":"Bob"} =>
{"userName":"Bob","greeting":"Hello, Bob!"}

Let’s add an indication that the form is waiting:

It seems working but an issue appears — if a user started typing and then decided to return the original name during the given 3 seconds, then from the “greet” transition perspective nothing has changed and the function will never called and the form will keep “thinking” forever unit you type a different name. It can be solved by adding @Emitter() decorator for userName field:

@Emitter()
userName: string;

which says to the library, that any assignment of any value to this field will be considered as a change, regardless of whether the new value is the same as the previous one.

However, there is another solution — when the form stops thinking, it can set userName to null and the user will have to start typing a new name:

Now let’s think about a situation when a user is impatient and wants to get the result immediately. Well, let’s let him press Enter ((keydown.enter)="onEnter()") to get the immediate result:

It would be nice to know how much time to wait if user does not press Enter — some kind of countdown counter would be very useful:

<h1 *ngIf="isThinking">Thinking ({{countdown}} sec)...</h1>

and it is how it looks like:

The countdown is also should be reset each time a new greeting is ready. It prevents the situation when the user immediately presses [Enter] and the countdown remains 3 at this moment — after that it stops working, since its value will never change again. For simplicity lets reset all the fields that depend on “Is Thinking” flag:

Change Detection

Obviously, the countdown woks asynchronously and this fact does not cause any issue with the Angular change detection as long as the component detection strategy is Default. However, if the strategy is OnPush, then nothing can tell the component that its state is changing while the countdown is in progress.

Fortunately, we’ve already defined a callback function which is called each time when the component state has just changed, so the only thing which is required is adding an explicit change detection there:

Now it works as expected even with OnPush detection strategy.

Output Properties

The library detects all component event emitters and calls them when a bound component field has just changed. By default the binding is performed using Change suffix for the event emitters:

greeting:  string;

@Output()
greetingChange = new EventEmitter<string>();

Shared State

Typically, when a component is destroyed (e.g. hidden by *ngIf) then all still pending asynchronous operations initiated by that component end in nothing. However, the library allows extracting a component state with its transitions into a separate object which can exist independently on the component. More other, such object can be shared between several components simultaneously!

Let’s convert the greeting form component into a service:

and add it to the module providers.

includeAllPredefinedFields means that all fields that have some initial value (even if it is null) will be automatically included into state objects.

To utilize the service inside a component, follow these steps:

1. Inject the service instance into the component
2. Point the state tracker to the service instance
3. Declare component fields which will be bound to the corresponding service field
4. Subscribe to the service state changes — it is necessary when the explicit change detection is required or change detection strategy is “On Push”

After these steps have been taken, the component will look like this:

The service instance is passed into initializeStateTracking function (using @StateTracking() decorator is also possible but it would take more efforts) which returns a state tracking handler that provides some methods to control the tracking behavior.

The subscription is required to call onStateApplied callback function when the service state is changed or to call a local transition function which depends on the shared state. If a component uses the default change detection strategy and there are no local transition functions, then subscription is not required.

Do not forget to unsubscribe it when the component is destroyed to avoid memory leaks. Alternatively, you can call handler.release() or releaseStateTracking(this) functions to release component subscriptions, but these methods also cancel pending asynchronous operations, which is not always desirable.

Complex Shared State

The library allows using shared states not only in components, but also in others services.

Let’s create a service which will will log all greetings and simulate sending them to a server:

On each greeting field value change it will be added to log array. logVersion will be increased each time the array is changed to provide change detection without making log array immutable:

The service will not send new greetings immediately, it will wait some time to collect a bunch of changes:

And finally saving:

This transition function differs from the others — it is asynchronous.

1. It decorated with WithAsync instead of just With.
2. The decorator has a concurrent launch behavior specification (OnConcurrentLaunchPutAfter in this case)
3. Instead of a current state object it receives a function which returns a current state at the moment it called.

In the same way, we can implement greeting deleting and restoring , but I will skip this part, since there is nothing new in it.

As a result, our form will look like this:

—

We’ve just reviewed an example of a user interface with a relatively complex asynchronous behavior. However, it turns out that implementing this behavior is not that difficult using the concept of a series of immutable states. At least it can be considered as an alternative to RxJs.

—

1. Link to the code: https://stackblitz.com/edit/set-state-greet

2. Link to the previous article: Angular Components with Extracted Immutable State

3. Link to ng-set-state source code: https://github.com/0x1000000/ngSetState

Angular Components State Tracking with ng-set-state was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

I noticed that when it becomes necessary to work with SQL databases in .Net applications, developers often chose some ORM library (.Net Entity Framework in most cases) without even considering other alternatives. This is a reasonable decision at first sight, since for popular ORM libraries there are many tutorials, great toolkit, and a lot of experienced developers. However, it is not always a good choice — I would at least consider other options if your application:

1. Mainly works with fact relationships rather than objects
2. Requires using dynamic (not predefined) queries

Fact Relationships

If your application mainly implements use cases similar to these ones:

1) “Give me the list of employees with the highest salary broken down by department”

2) “Give a discount to all customers registered in April”

3) “Reduce the price of T-shirts from last year’s collection by 20%”

then your application is focused on fact relationships rather than objects since all these scenarios involve handling an unpredictable amount of data. For example, organization might have 2 departments and 30 employees or hundreds of departments and hundreds of thousands of employees. If some ORM is used, it is assumed that all the data is loaded into memory, then analyzed and, in the case of write operations, uploaded back into the SQL database. Obviously, this cannot but affect performance.

Note. Modern ORM libraries have many ways to solve the performance problems such as LINQ expressions convertible to SQL, lazy loading, etc. But we must remember that these are just tricks that partially mitigate the conceptual issue and problems inevitably arise at some point.

On the other hand, SQL is specially designed to deal with fact relationships and the scenarios described above could be implemented with a couple relatively simple expressions. Obviously, it makes sense to use pure SQL without any ORMs here, but what options do we have? Not so many. The common ones will be:

1) Stored Procedures

2) Text based queries

3) Some LINQ based query builders (e.g. “LINQ to DB”)

4) My library “SqExpress” :)

Stored Procedures

Stored procedures are the first thing that comes to mind when you need to use the full power of native SQL, but this approach has several serious disadvantages:

1) Difficulties in maintenance — it is always necessary to make sure that the .Net code is synchronized with code of stored procedures in all database instances that your application works with. This means that you are doomed to mess with migration scripts in almost every deployment and inevitably something will go wrong at some point.

2) SQL is a great query language, but a very bad programming language — Once you start writing stored procedures, it will be difficult to resist the temptation to move as much logic into them as possible, because it is such an easy way to solve performance problems and quickly fix issues in application design. But the downside is that you will not be able to break your logic into small modules and test them in isolation (how often I have seen “super” procedures for 5 thousand lines with hundreds of variables inside)

3) Copy-paste — The code in procedures is very difficult to reuse (functions, however, partly solve this problem), so some common logic will be duplicated in many procedures.

4) It is difficult to implement batch processing. UpdateUserById — it is the typical name for a stored procedure. But what if I have 100 users? No problem! Let’s call the procedure 100 times! The only problem is that it will take 100 times longer (Some databases support Table-Valued Parameters, but such parameters are difficult to use and maintain).

5) Stored procedures don’t help if you need dynamic queries

6) If you actively use stored procedures, then migrating to another SQL database will be incredibly difficult.

Text based queries

Queries resulting from simple string concatenation are pure evil. Such code is difficult to write, read, and maintain. Security is also a huge issue — Sql Injections are still relevant today.

However, surprisingly, there are some positive aspects to this approach:

1) You can use the approach in addition to any other one

2) It is still the most popular way to create dynamic queries which is available to everyone.

LINQ based query builders

Such query builders (e.g. “LINQ to DB”) can be a good alternative when you need reach SQL abilities. “LEFT”, “RIGHT”, “FULL”, “CROSS” JOINS etc. are no longer a problem — you can express them directly in C# code (using LINQ):

//Full Join
var query =
    from c in db.Category
    from p in db.Product.FullJoin(pr => pr.CategoryID == c.CategoryID)
    where !p.Discontinued
    select c;

CTEs are also supported

//CTE
var employeeSubordinatesReport  =
   from e in db.Employee
   select new
   {
      e.EmployeeID,
      e.LastName,
      e.FirstName,
      NumberOfSubordinates = db.Employee
          .Where(e2 => e2.ReportsTo == e.ReportsTo)
          .Count(),
      e.ReportsTo
   };

var employeeSubordinatesReportCte = employeeSubordinatesReport
                                       .AsCte("EmployeeSubordinatesReport");

var result =
   from employee in employeeSubordinatesReportCte
   from manager in employeeSubordinatesReportCte
                      .LeftJoin(manager => employee.ReportsTo == manager.EmployeeID)
   select new
   {
      employee.LastName,
      employee.FirstName,
      employee.NumberOfSubordinates,
      ManagerLastName = manager.LastName,
      ManagerFirstName = manager.FirstName,
      ManagerNumberOfSubordinates = manager.NumberOfSubordinates
   };

However, when it comes to dynamic queries, LINQ is not the best solution. Theoretically it is possible to change the expression but it is not a simple action.

SqExpresss

For many years I was unhappy working with SQL from .Net, but then I wrote my own library (SqExpress) (Video Tutorial on Youtube) that allows me to write C # code as close to SQL as possible. It does not use LINQ — query building is implemented through helper functions and operator overloading. As a result, there are no problems with dynamic queries. Here’s an example:

Still ORM?

There is one more question left: “When ORM is a good solution?” Well.. let’s just invert those conditions under which ORM is not recommended to use:

1. In the vast majority of scenarios, you can accurately predict the number of rows in database tables that will be read or modified (that means that all the data can be mapped to objects)
2. No frequent need in dynamic queries

If these conditions are met, then most likely ORM is the right choice.

—

I hope this short article will help you decide which way of working with a SQL database is best for your application, or at least you will reflect and don’t make hasty decisions.

Links:

• linq to db on github
• SqExpress on github (Video Tutorial on Youtube)
• “Syntax Tree and Alternative to LINQ in Interaction with SQL Databases”- my previews article related to this topic

Writing SQL in C# or When You should not use ORM was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

Recently, when I was reviewing the new features that are going to be included in .Net 5, I came across an interesting one — C# Source Generators. This feature especially interested me since I have been using a similar approach for the last… 5 years and what Microsoft proposes is simply a deeper integration of this approach into the build process.

Further I’ll share my experience using Roslyn in code generation and I think this will help you better understand what exactly Microsoft offers and when you can use it.

First, let’s look at a typical code generation scenario. You have some external source of information such as a database, a JSON description of some REST service, another assembly (via reflection) etc. And using this information you can generate various types of source code such as DTOs, db model classes or proxies to REST services.

However, sometimes there are situations when there is no any external source of information and everything you need is contained in the source code of the project itself, where you want to add some generated code.

Coincidentally, I recently published an open source project that has an example of such a situation. In the project there are more than 100 classes that represent SQL syntax tree nodes and I needed to create visitors which would traverse and modify tree objects (You can find more information about the project in my previous article Syntax Tree and Alternative to LINQ in Interaction with SQL Databases).

The reason why the code-generation is a good choice here is that each time I make even a small change to the classes I need to remember to modify the visitors and these changes must be done very carefully. However, I cannot use reflection for the code generation, since the assembly containing the new changes simply does not exist yet, and if these changes are not compatible with the previous version and lead to compilation errors, then this assembly will never appear until I manually fix all the errors.

At first glance, this problem has no solution, but in fact, I can use the Roslyn compiler to solve it. I can pre-compile the model classes in advance and get information similar to what I could get via the reflection.

Let’s create a simple console application and add Microsoft.CodeAnalysis.CSharp package.

Note: Theoretically it could be done in t4 but I prefer not to struggle with dependencies and strange t4 syntax.

First, we need to read all the cs files that contain the model classes and extract syntax trees from them:

The trees contain a lot of information about the source code from text perspective (class names, method names, etc.), but often this information is not enough as we want to know what the text means, so we need to ask Roslyn to analyze the syntax trees and get some semantic data:

Using the semantic data we can get an object of INamedTypeSymbol type:

which can provide information about class constructors and properties:

Since all the model classes are immutable, all meaningful properties have to be set via theirs constructors, so let’s iterate all the constructors parameters and get their types:

Now it needs to analyze the parameter type and find out the following things:

1. Is the type a list?
2. Is the type Nullable (the project uses “Nullable reference types”)?
3. Whether the type inherits from the base type (in our case, the interface) for which we create “Visitors”.

The semantic model provides answers to these questions:

Note: The method “AnalyzeSymbol” extracts an actual type from collections and Nullables:

List<T> => T (list := true) 
T? => T (nullable := true) 
List<T>? => T (list := true, nullable := true)

Checking a base type in the semantic model is more complex than if you were using the reflection, but it is also possible:

Now we can put all the information in a simple container:

and use it in the code generation to create something like this.

In my project, I run the code generation as a console utility, but in .Net 5 you will be able to embed this generation into the project as a class marked with a special attribute that will be automatically run at compile time to add missing parts of the code. This is certainly more convenient than a standalone utility, but the idea is similar.

Finally, I want to say that you shouldn’t take this new feature of .Net 5 as an incredible innovation that will fundamentally change the approach to dynamic code generation that is used in libraries such as AutoMapper, ASP.Net Core, etc. (I’ve heard such opinions) It will not! The fact is that the code-generation works in a static context where everything is known in advance, but, for example, AutoMapper does not know what classes it will work with and it still will have to dynamically emit code. However, there are situations when such code generation is very useful (I described one of them in this article). So it’s worth knowing about this feature and understanding its principles and limitations.

(link to the source code on github)

No need to wait for .Net 5 to start using code generation with Roslyn was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

It was a legacy enterprise project and I was asked to improve its “advanced” filtering capabilities.

Before they had something like this:

But wanted something like this:

Let’s skip the UI part and focus on the data access layer. Like 99% of similar projects this one used a sql database (MS SQL but it is does not matter) and this project belonged to the part of the similar projects that does not struggle with Entity Framework but puts all the logic in stored procedures. The procedure which performed the “advanced” search looked like that:

CREATE PROCEDURE dbo.SomeContextUserFind
    (@ContextId int, @Filter nvarchar(max)) AS
BEGIN

DECLARE @sql nvarchar(max) = 
    N'SELECT U.UserId, U.FirstName, U.LastName
    FROM [User] U
    INNER JOIN [SomeContext] [C]
      ON ....
    WHERE [C].ContextId = @p1 AND ' + @Filter;

EXEC sp_executesql 
    
    @sql,
    N'@p1 int',
    @p1=@ContextId
END

and the code that generated the filter string looked something like that:

Of course it is not the greatest code you have ever seen, but unfortunately legacy projects (or even not yet legacy) often are full of similar things. Anyway, it is what it is and it should be somehow improved.

The first thought that came to my mind is just adding more fields into the “FilterItem” and make the building logic more complex, but I quickly realized that it a road to nowhere — it would be extremely difficult to maintain such code, and I would never achieve the desired functionality.

At that point, I remembered the “abstract syntax tree”, which is obviously the best choice in this case, and now I am going to explain what it is and how it helps.

Abstract Syntax Tree

First, let’s look at the filter strings we’re going to create, for example this one:

[FirstName]='John' AND ([LastName]='Smith' OR [LastName]='Doe')

And here we can notice some structure:

This structure is a tree and that means that we can create several simple classes which will describe it:

Using the classes, we can create an object which would represent the original filter:

In fact, this structure, called the “Abstract Syntax Tree”, can be used to represent more complex queries, but they will all have a single “root” that can be stored in a single object.

“Abstract Syntax Tree” is a result of parsing a phrase of some formal language (language of Boolean expressions in our case) with defined syntax rules. Such rules have a specific notation. For example the rules of our simple language (subset of boolean expressions) can be written as:

<eqPredicate> ::= <column> = <str>
<or> ::= <eqPredicate>|or|<and> OR <eqPredicate>|or|<and>
<and> ::= <eqPredicate>|(<or>)|<and> AND <eqPredicate>|(<or>)|<and>

Note: “Abstract” means that the syntax do not describe all the language details, for instance, grouping parentheses, extra spaces etc.

Parsing is a separate huge topic in and of itself, and it’s not that important right now since we already have syntax trees ready, so let’s focus on what we can do with them.

SQL Code Generation

Obviously, the most important task for us is converting the syntax tree back to text and we have several ways to do that.

The first way is to use pattern matching, which is pretty straightforward:

As a result, the builder will contain the following line:

[FirstName]='John' AND ([LastName]='Smith' OR [LastName]='Joe')

It seems this is what we need!

Visitor

Although I have a passion for functional programming, in this case the object-oriented approach may provide a more efficient solution — I’m talking about the “Visitor” pattern. The idea of this pattern is that we do not try to determine a type of an object, but give it a list of all possible actions (in the form of an interface) and the object itself chooses the action that is most appropriate for its type. Let’s define the list:

and any object (of our structure) can accept the selection:

Now we can extract the generation of sql code into a separate class:

And use it as follows:

As a result, we will get the desired string:

[FirstName]='John' AND ([LastName]='Smith' OR [LastName]='Joe')

Using the “Visitor” pattern has several advantages over the pattern matching. So, for example, the selection of concrete types is always exhaustive since adding a new type to the structure always leads to a change in the IExprVisitor interface and, as a result, the need to extend all its implementations (otherwise there will be compilation errors).

Traversal and parentheses

There are several aspects to pay attention to in this algorithm.

First, how does it work at all?

In fact, here we are doing a depth-first traversal of the syntax tree and the sql code is a trace of this traversal:

Second, what happens to the parentheses?

The point is that Boolean expressions have a certain order of evaluation. ‘AND’ operation is evaluated first, and ‘OR’ operation is evaluated second. The parentheses are needed to change this sequence, since any expression in parentheses has a higher priority in the evaluation. But in the syntax tree, the sequence of the evaluation is given by the structure itself (from branches to the root), so a separate type for parentheses is not required.

Extension of the syntax

In reality, of course, we also need other predicates, for example ‘Not Equal’ and to be able to use it, we just need to add a new class:

And since we have a new type, the compiler tells us that we need to implement the SQL code generation for it:

We’ve created just a very simple set of Boolean predicates and MS SQL supports many more, but as you see you can easily add all the required language structures.

By the way, the SQL Server documentation has all the SQL syntax rules that are very useful:

Operator Overloading

Obviously, creating syntax trees by calling the class constructors is not convenient at all. However, the C# operator overloading can help us with this.

Let’s do the following:

Now we can create a syntax tree in a very simple and clear way:

And the result again will be:

[FirstName]='John' AND ([LastName]='Smith' OR [LastName]='Doe')

Note: C# does not allow overloading of the && and || operators and actually it makes sense, since these operators stop further evaluation if the result is already known, but we need all the parts to build the syntax tree (it will be evaluated by the SQL database).

What is Next

We seem to have solved the problem with boolean filters, but what is about sorting and paging? It is also sometimes necessary to attach a sql view (or derived table) to filter (or sort) on a calculated field.

Not a problem! Let’s just implement all the SQL SELECT syntax:

Of course, you do not need to do it by yourself since there are several libraries (for example mine) where this is already implemented, so just consider this approach as an alternative way to interact with SQL databases.

Alternative to LINQ

What we’ve done with the operator overloading is somewhat similar to LINQ expressions, and in fact there are some actual similarities. The C# compiler generates syntax trees and then libraries such as Entity Framework or “LINQ to SQL” convert those trees to real SQL code. The main problem with this approach is that the compiler generates syntax tree of C# language, but we need SQL! Reflecting imperative C# into declarative SQL is not an easy task, and the results are often unpredictable.

I prefer a different approach — instead of using C # syntax as a basis, it can use real SQL syntax. And instead of the compiler, it can use operator overloading, extension methods and fluent builders.

With this approach,on the one hand, we get almost the same flexibility as if we were using stored procedures. On the other hand, we have strong typing, intellisense and business logic does not move to the database. And … no need to google how to do a left join using LINQ )

Another advantage is that the data update statements (INSERT, UPDATE, DELETE and even MERGE) can also be implemented in the same way, and there is no need to load thousands of records from the database to update a just single column.

This is an example of what can be done using real SQL syntax as a basis (taken from here):

Conclusion

Syntax tree is a very powerful data structure that you will likely come across sooner or later. Probably these will be LINQ expressions, or maybe you will need to create a Roslyn analyzer, or maybe you want to create your own syntax yourself, like I did a few years ago in order to refactor one legacy project. In any case, it is important to understand this structure and be able to work with it.

Link to SqExpress — a project that partially includes the code from this article.

Syntax Tree and Alternative to LINQ in Interaction with SQL Databases was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.

Almost 100% of asynchronous tasks that you deal with in your C# code are running or already completed regardless of whether you “await” them or not. This can be shown with the following example:

static async Task Main()
{
    var task = GetValueAsync();
    await Task.Delay(500);
    Console.WriteLine("Awaiting...");
    Console.WriteLine("Result is: " + await task);
}

static async Task<int> GetValueAsync()
{
    Console.WriteLine("Async Start");
    await Task.Delay(100);
    Console.WriteLine("Async End");
    return 42;
}

Output:

Async Start
Async End
Awaiting...
Result is: 42

As you see GetValueAsync finished before await but what if the behavior is not desired and you do not want a task to start executing until there is at least one “awaiter”? Of course, you can use Lazy — a generic class from the System namespace:

static async Task Main()
{
    var task = new Lazy<Task<int>>(GetValueAsync);
    await Task.Delay(500);
    Console.WriteLine("Awaiting...");
    Console.WriteLine("Result is: " + await task.Value);
}

static async Task<int> GetValueAsync()
{
    Console.WriteLine("Async Start");
    await Task.Delay(100);
    Console.WriteLine("Async End");
    return 42;
}

Output:

Awaiting...
Async Start
Async End
Result is: 42

However, there is another solution which will just slightly impact you client code — it is replacing of Task returned by your asynchronous function with LazyTask — a class that you can create by your own using the new C#7 feature — Generalized async return types

As a result, your code will look like as follows:

static async Task Main()
{
    var task = GetValueAsync();
    await Task.Delay(500);
    Console.WriteLine("Awaiting...");
    Console.WriteLine("Result is: " + await task);
}

static async LazyTask<int> GetValueAsync()
{
    Console.WriteLine("Async Start");
    await Task.Delay(100);
    Console.WriteLine("Async End");
    return 42;
}

Output:

Awaiting...
Async Start
Async End
Result is: 42

As you know C# compiler converts methods marked async into a state machine where each state represents a completion of an asynchronous operation marked await. Generalized async return types allow you to gain access to the state machine execution (I examined this mechanism in detail in one of my previous articles.) and in our case we just need to postpone the first step till we have at least one “awaiter”. It can be done by adjusting the Start method:

LazyTaskMethodBuilder.cs

public class LazyTaskMethodBuilder<T>
{
    public LazyTaskMethodBuilder() => this.Task = new LazyTask<T>();
    public void Start<TStateMachine>(
        ref TStateMachine stateMachine)..
    {
        //instead of stateMachine.MoveNext();
        this.Task.SetStateMachine(stateMachine);
    }
    ...
}

Instead of starting the first step, the Start method just stores a link to the state machine inside the LazyTask object.

The first step now can be called only when we have the first “awaiter”:

LazyTask.cs

public void OnCompleted(Action continuation)                               {
    ...
    this._asyncStateMachine.MoveNext();
    ...
}

Note: The interesting thing that the trick works even with methods which do not have any asynchronous calls inside (without “await” keyword).

That’s all!

In conclusion, I want to say that Generalized async return types is a powerful mechanism and LazyTask is just one the examples of how it can be utilized.

Links

1. Source code on Github;

2. “Maybe” monad through async/await in C# (No Tasks!) — an article where Generalized async return types are examined in detail;

3. Video version with more details.

Writing “Lazy Task” Using New Features of C# 7 was originally published in ITNEXT on Medium, where people are continuing the conversation by highlighting and responding to this story.