<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Afsaneh Narimani on Medium]]></title>
        <description><![CDATA[Stories by Afsaneh Narimani on Medium]]></description>
        <link>https://medium.com/@afsane.narimani?source=rss-4fbb23802b0e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*8p7Rj6KX_1jQ81qpxfa5Lw.gif</url>
            <title>Stories by Afsaneh Narimani on Medium</title>
            <link>https://medium.com/@afsane.narimani?source=rss-4fbb23802b0e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 17:52:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@afsane.narimani/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Understanding Concurrency in Python: From Threads to AsyncIO and Parallelism]]></title>
            <link>https://medium.com/@afsane.narimani/understanding-concurrency-in-python-from-threads-to-asyncio-and-parallelism-6423378d0fd7?source=rss-4fbb23802b0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/6423378d0fd7</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[multithreading]]></category>
            <category><![CDATA[asyncio]]></category>
            <category><![CDATA[concurrency]]></category>
            <dc:creator><![CDATA[Afsaneh Narimani]]></dc:creator>
            <pubDate>Tue, 17 Nov 2026 08:50:50 GMT</pubDate>
            <atom:updated>2026-11-17T08:50:50.082Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2vHTrQhIZrqR1_-WWV08ww.png" /></figure><p><em>An engineering guide to processes, thread safety, coroutines, event loops, synchronization, the GIL, and choosing the right Python concurrency model.</em></p><p>Concurrency is one of those topics that seems simple until you need to make a production service reliable under load. Should you use threads, processes, or asyncio? Does the GIL prevent parallelism? When do you need a lock or semaphore? And what actually happens when an async FastAPI endpoint calls a blocking OCR engine?</p><p>This guide builds the answers from first principles, then connects them to practical Python code and a realistic inference-service example. Examples target Python 3.11+ unless a newer version is explicitly mentioned. Last reviewed: November 2026.</p><h3>A Roadmap: From Execution to Production</h3><p>We will move from processes and threads to race conditions, the GIL, asyncio, executors, and production concurrency limits. The final case study brings the concepts together in an OCR inference API.</p><h3>1. The big picture</h3><p>Imagine a restaurant. A <strong>process</strong> is one operating restaurant, a <strong>thread</strong> is a worker executing instructions, a <strong>task</strong> is a particular order, and a <strong>semaphore</strong> is the rule that only a certain number of people can use a shared oven at once. An <strong>event loop</strong> coordinates tasks that can pause while waiting. These concepts solve different problems: <em>execution</em>, <em>coordination</em>, <em>correctness</em>, and <em>capacity</em>.</p><p><strong>Concurrency</strong> means making progress on multiple tasks during overlapping periods. <strong>Parallelism</strong> means actually executing multiple operations at the same instant (for example, on separate CPU cores). A concurrent program is not necessarily parallel.</p><p>Four separate questions help organize the subject:</p><ol><li><strong>What executes work?</strong> Processes, OS threads, and CPU cores.</li><li><strong>What represents work?</strong> Functions, coroutines, and tasks.</li><li><strong>What coordinates work?</strong> Event loops, schedulers, executors, and queues.</li><li><strong>What keeps work safe?</strong> Locks, semaphores, immutable state, isolation, and careful ownership.</li></ol><h3>2. How programs execute</h3><p>A <strong>program</strong> is code stored on disk. A <strong>process</strong> is a running instance with its own resources and address space. A <strong>thread</strong>is an execution path within a process. A process normally starts with at least one thread and may create more.</p><p>For example:</p><pre>print(&quot;start&quot;)</pre><pre>def process_image():<br>    print(&quot;processing&quot;)</pre><pre>print(&quot;loaded&quot;)<br>process_image()</pre><p>This prints start, loaded, processing: def creates a function object; it does not execute the function body. Importing a Python module generally executes its top-level statements once per interpreter under that module name. This matters when a module unexpectedly loads a machine-learning model during import.</p><p><strong>Synchronous</strong> execution means the caller waits for the called operation to complete before proceeding. <strong>Blocking</strong> means an operation occupies or suspends a thread so it cannot do other useful work during that wait. <strong>Asynchronous</strong> execution can separate initiation from completion. <strong>Non-blocking</strong> behavior allows a thread to continue useful work rather than waiting in place. Related terms are not synonyms: an async def function can still execute blocking code.</p><h3>3. Concurrency vs. parallelism</h3><p><strong>Sequential:</strong> A finishes, then B starts.</p><pre>Time ---&gt;<br>A: [=======]<br>B:          [=======]</pre><p><strong>Concurrent, not necessarily parallel:</strong> A and B take turns; their progress overlaps in time.</p><pre>Time ---&gt;<br>A: [run][wait......][run]<br>B:      [run][wait][run]</pre><p><strong>Parallel:</strong> A and B are running at the same instant on different execution resources.</p><pre>CPU 1: [AAAAAAA]<br>CPU 2: [BBBBBBB]</pre><p>Concurrency improves utilization when one task waits. Parallelism can reduce computation time when work can be divided across cores. In real systems the two often coexist.</p><h3>4. CPU-bound vs. I/O-bound work</h3><p><strong>I/O-bound</strong> tasks spend substantial time waiting for network responses, disks, databases, or remote services. Examples: HTTP calls and database queries. <strong>CPU-bound</strong> tasks spend their time computing. Examples: pure-Python mathematical loops, image processing, and certain inference workloads.</p><p>A task may be mixed: an OCR request reads an uploaded image (I/O), decodes and preprocesses it (CPU/native library), and runs ONNX inference (often native compute). Classify individual stages rather than the entire application with one label.</p><ul><li>For many cooperative I/O operations, consider <strong>asyncio</strong> with async-compatible libraries.</li><li>For synchronous blocking I/O, consider <strong>threads</strong> or a thread pool.</li><li>For pure-Python CPU-heavy computations under traditional CPython, consider <strong>processes</strong>.</li><li>For NumPy/ONNX/native code, benchmark: native operations may release the GIL and may have their own internal thread pools.</li></ul><h3>5. Processes and threads</h3><p>Threads inside one process share its memory and can communicate cheaply, but shared mutable objects can introduce bugs. Processes have separate address spaces, usually stronger isolation, greater startup/memory costs, and require explicit communication or shared memory to exchange information.</p><pre>Process A                         Process B<br>+---------------------------+    +----------------------+<br>| Main thread (event loop)  |    | Main thread          |<br>| Worker thread 1           |    | Worker thread 1      |<br>| Worker thread 2           |    |                      |<br>| Shared memory of A        |    | Separate memory of B |<br>+---------------------------+    +----------------------+</pre><p><strong>Context switching</strong> means changing which work executes; OS threads may be preempted by the operating system, while Python asyncio tasks generally yield at suspension points such as await. Switching and coordination have overhead.</p><h3>6. Race conditions and thread safety</h3><p>A <strong>race condition</strong> occurs when correctness depends on the timing or interleaving of concurrent operations. <strong>Thread safety</strong> means code remains correct when accessed concurrently by threads under its intended contract.</p><p>This example can lose increments because checking and changing shared state is a read-modify-write sequence:</p><pre>import threading</pre><pre>counter = 0<br>lock = threading.Lock()</pre><pre>def increment():<br>    global counter<br>    with lock:<br>        counter += 1</pre><pre>threads = [threading.Thread(target=increment) for _ in range(100)]<br>for thread in threads:<br>    thread.start()<br>for thread in threads:<br>    thread.join()<br>print(counter)  # 100</pre><p>The lock protects a <strong>critical section</strong>, a region in which shared state must be accessed atomically with respect to cooperating threads. Removing the lock can make the algorithm unsafe; the failure may be intermittent rather than reproducible on every run.</p><p>The GIL is <strong>not</strong> a substitute for an application-level lock. Even if some individual operations are atomic in a particular implementation, multi-step invariants are not automatically safe. Prefer immutable objects, message passing, or isolated ownership where practical.</p><p>Async code also has races when shared state changes across await points. The single-threaded nature of an event loop does not make every workflow safe.</p><h3>7. Synchronization primitives</h3><p>PrimitiveWhat it doesCommon useLock / mutexAllows one holder into a critical sectionProtect a mutable counter or cacheRLockReentrant lock for the same threadNested locking in carefully designed codeSemaphore(n)Allows up to n simultaneous holdersLimit concurrent model callsBoundedSemaphore(n)Like semaphore, detects over-releaseCapacity-limited resourcesEventSignals that a condition has occurredNotify workers to proceedConditionWaits for a state predicate while coordinating a lockProducer/consumer state changesBarrierWaits for a set of parties to reach a pointPhased thread computationQueueTransfers items between workersProducer–consumer pipeline</p><p>A semaphore limits <strong>entry into a section</strong>, not necessarily the number of threads or processes that exist.</p><pre>import threading<br>import time</pre><pre>semaphore = threading.Semaphore(2)</pre><pre>def handle(request_id: int) -&gt; None:<br>    with semaphore:<br>        print(&quot;start&quot;, request_id)<br>        time.sleep(1)<br>        print(&quot;end&quot;, request_id)</pre><pre>threads = [threading.Thread(target=handle, args=(i,)) for i in range(5)]<br>for thread in threads:<br>    thread.start()<br>for thread in threads:<br>    thread.join()</pre><p>Five threads exist, but only two can be inside the protected section simultaneously. A threading.Semaphore created separately in each server process does <strong>not</strong> create a global service-wide limit. multiprocessing.Semaphore can coordinate processes that actually share that synchronization object; independently deployed services require coordination through an appropriate external mechanism.</p><p><strong>Deadlock:</strong> participants wait for resources held by one another and cannot proceed. <strong>Starvation:</strong> one participant repeatedly fails to make progress. <strong>Livelock:</strong> participants keep reacting but make no useful progress. <strong>Contention:</strong>workers compete for the same resource. Use consistent lock ordering, short critical sections, timeouts where appropriate, and avoid holding locks across unrelated slow I/O.</p><h3>8. The GIL and free-threaded Python</h3><p>In a conventional CPython build, the <strong>Global Interpreter Lock (GIL)</strong> normally allows only one thread at a time to execute Python bytecode within one interpreter. This commonly limits CPU parallelism for pure-Python code in ordinary threads.</p><p>But this does <strong>not</strong> mean threads are useless. I/O often releases the GIL, and many native extensions release it while computing. ThreadPoolExecutor can therefore help synchronous I/O workloads and some native-code-heavy workloads.</p><p>Important recent developments:</p><ul><li><strong>Python 3.13</strong> introduced an optional experimental free-threaded CPython build.</li><li><strong>Python 3.14</strong> officially supports free-threaded builds, but they are still <strong>not the default</strong>. Disabling the GIL does not make mutable shared state magically safe, and not every extension supports this mode.</li><li><strong>Python 3.14</strong> added InterpreterPoolExecutor. Its workers have separate interpreters/GILs, allowing multi-core parallelism while requiring more explicit data isolation and serialization.</li></ul><p>Always document which Python version, build, and third-party runtime you benchmark. These details can change concurrency behavior.</p><h3>9. Python’s concurrency toolbox</h3><p>API / approachExecution modelGood fitImportant limitationthreading.ThreadOS threads in one processA few independently managed blocking tasksShared-state races; GIL in standard CPythonThreadPoolExecutorManaged thread poolMany sync I/O operations; blocking SDKsPool saturation; CPU-bound pure Python may not scaleasyncioCooperative tasks on an event loopHigh-concurrency async network I/OBlocking functions stall the loopasyncio.to_threadAsync wrapper over thread offloadCall sync blocking function from async codeDoes not automatically speed up pure Python CPU workmultiprocessingSeparate processesCPU parallelism, isolationSerialization and process overheadProcessPoolExecutorManaged process poolBatch CPU-bound functionsFunctions/arguments must be serializable; worker startupInterpreterPoolExecutor(3.14+)Threads with isolated interpretersSpecialized multi-core workIsolation, compatibility, serializationFree-threaded CPython (optional)Threads without the usual GILParallel threaded workloads with compatible librariesNot the default; correctness and extension support matter</p><p><strong>Libraries vs. models:</strong> concurrent.futures provides a common Executor/`Future` API for multiple worker types; it is not an independent hardware execution model. AnyIO, Starlette and FastAPI build higher-level abstractions on top of existing async/threading mechanisms.</p><h3>A simple producer–consumer queue</h3><p>A queue.Queue is often safer than sharing a mutable list and managing all locking yourself. It coordinates workers and lets producers hand off work. A bounded maxsize also provides a basic form of backpressure.</p><pre>import queue<br>import threading</pre><pre>jobs: queue.Queue[int | None] = queue.Queue(maxsize=4)</pre><pre>def worker() -&gt; None:<br>    while True:<br>        job = jobs.get()<br>        try:<br>            if job is None:<br>                return<br>            print(f&quot;processed {job}&quot;)<br>        finally:<br>            jobs.task_done()</pre><pre>thread = threading.Thread(target=worker)<br>thread.start()</pre><pre>for number in range(3):<br>    jobs.put(number)</pre><pre>jobs.put(None)<br>jobs.join()<br>thread.join()</pre><h3>Minimal threading example</h3><pre>import threading</pre><pre>def download_report(report_id: int) -&gt; None:<br>    print(&quot;download&quot;, report_id)</pre><pre>workers = [threading.Thread(target=download_report, args=(i,)) for i in range(3)]<br>for worker in workers:<br>    worker.start()<br>for worker in workers:<br>    worker.join()</pre><h3>Minimal thread pool example</h3><pre>from concurrent.futures import ThreadPoolExecutor<br>import time</pre><pre>def blocking_call(number: int) -&gt; int:<br>    time.sleep(0.2)<br>    return number * 2</pre><pre>with ThreadPoolExecutor(max_workers=4) as executor:<br>    results = list(executor.map(blocking_call, range(8)))<br>print(results)</pre><h3>Minimal process pool example</h3><p>Save the following code as a .py file and run it normally rather than relying on an interactive REPL:</p><pre>from concurrent.futures import ProcessPoolExecutor</pre><pre>def cpu_work(number: int) -&gt; int:<br>    return sum(i * i for i in range(number))</pre><pre>if __name__ == &quot;__main__&quot;:<br>    with ProcessPoolExecutor(max_workers=4) as executor:<br>        results = list(executor.map(cpu_work, [800_000] * 4))<br>    print(results)</pre><p>The if __name__ == &quot;__main__&quot; guard is important for multiprocessing compatibility, especially with the spawn start method on macOS/Windows. Start methods and defaults vary by platform and Python version; Python 3.14 changed the POSIX default away from fork.</p><h3>10. Asyncio, coroutines, tasks, and the event loop</h3><ul><li>async def defines a <strong>coroutine function</strong>.</li><li>Calling it creates a <strong>coroutine object</strong>; its body does not run yet.</li><li>await waits for an <strong>awaitable</strong>; if that awaitable suspends, the current coroutine yields control.</li><li>An <strong>event loop</strong> schedules ready async tasks and reacts to I/O and timers.</li><li>asyncio.Task schedules and manages a coroutine&#39;s execution.</li><li>asyncio.create_task() lets a coroutine run concurrently with other tasks.</li><li>asyncio.gather() waits for multiple awaitables and returns their results; TaskGroup provides structured task lifetimes and stronger failure management (Python 3.11+).</li></ul><pre>import asyncio</pre><pre>async def fetch(name: str, delay: float) -&gt; str:<br>    print(&quot;start&quot;, name)<br>    await asyncio.sleep(delay)<br>    print(&quot;done&quot;, name)<br>    return name</pre><pre>async def main() -&gt; None:<br>    async with asyncio.TaskGroup() as group:<br>        first = group.create_task(fetch(&quot;A&quot;, 2))<br>        second = group.create_task(fetch(&quot;B&quot;, 1))<br>    print(first.result(), second.result())</pre><pre>asyncio.run(main())</pre><p>Both delays overlap; the program takes about two seconds rather than three. asyncio.sleep() is non-blocking, while time.sleep() inside an async function blocks the event-loop thread.</p><h3>An async semaphore</h3><pre>import asyncio</pre><pre>async def main() -&gt; None:<br>    semaphore = asyncio.Semaphore(2)</pre><pre>    async def request(number: int) -&gt; None:<br>        async with semaphore:<br>            print(&quot;start&quot;, number)<br>            await asyncio.sleep(1)<br>            print(&quot;end&quot;, number)</pre><pre>    await asyncio.gather(*(request(i) for i in range(5)))</pre><pre>asyncio.run(main())</pre><p>Only two coroutines may be inside the section at once. Tasks waiting for the semaphore suspend without blocking the event loop.</p><h3>Cancellation, errors, and timeouts</h3><p>Cancellation is a normal part of async programming: clients disconnect, deadlines expire, and tasks fail. Cleanup belongs in finally; do not silently swallow asyncio.CancelledError. asyncio.timeout() can bound an async wait, but a timeout does <strong>not</strong> necessarily stop already-running synchronous work in a worker thread.</p><pre>import asyncio</pre><pre>async def operation() -&gt; None:<br>    try:<br>        async with asyncio.timeout(2):<br>            await asyncio.sleep(10)<br>    finally:<br>        print(&quot;cleanup&quot;)</pre><pre>asyncio.run(operation())  # Raises TimeoutError after cleanup</pre><h3>Task creation is not the same as awaiting sequentially</h3><p>This awaits one delay before starting the next:</p><pre>await fetch(&quot;A&quot;, 2)<br>await fetch(&quot;B&quot;, 1)</pre><p>This schedules both delays together:</p><pre>await asyncio.gather(fetch(&quot;A&quot;, 2), fetch(&quot;B&quot;, 1))</pre><p>These are snippets intended to appear inside an async def. The distinction is <strong>when the work gets scheduled</strong>, not merely whether the code contains await.</p><p><strong>Async does not equal parallel CPU execution.</strong> It primarily enables efficient coordination of waits. Async callbacks should avoid blocking the event-loop thread.</p><h3>11. Thread pools and process pools</h3><p>A <strong>pool</strong> manages reusable workers; an <strong>executor</strong> accepts jobs and produces <strong>Future</strong> objects that represent pending results. Benefits include limiting worker creation, reusing resources, and separating submission from completion.</p><p>Pool size is not the same as request concurrency or work capacity. A server may have thousands of suspended async requests while only four model inferences run. Conversely, one native inference can itself use many CPU threads. Backpressure and queue limits matter: a bounded worker pool with an unbounded input queue can still run out of memory.</p><p>In general:</p><ul><li>Thread pool: lower overhead and shared memory; suitable for blocking I/O.</li><li>Process pool: stronger isolation and CPU parallelism, but moving large images/models between processes can be expensive.</li><li>Interpreter pool: newer specialized option; explicitly isolated runtime state.</li></ul><h3>12. Combining async code and blocking functions</h3><p>Use asyncio.to_thread() to offload synchronous work from asyncio:</p><pre>import asyncio<br>import time</pre><pre>def blocking_operation() -&gt; str:<br>    time.sleep(2)<br>    return &quot;done&quot;</pre><pre>async def main() -&gt; None:<br>    result = await asyncio.to_thread(blocking_operation)<br>    print(result)</pre><pre>asyncio.run(main())</pre><p>In Starlette/FastAPI, run_in_threadpool() serves a similar role and uses AnyIO&#39;s thread offloading:</p><pre>from starlette.concurrency import run_in_threadpool</pre><pre>async def process_image(engine, image):<br>    return await run_in_threadpool(engine.recognize, image)</pre><p>The calling coroutine waits for the result; the event loop can run other tasks; the worker thread executes engine.recognize.</p><p><strong>Important cancellation nuance:</strong> cancelling the waiting coroutine does not forcibly kill a running Python worker thread. Production code must account for request cancellation, engine shutdown, and native library behavior.</p><h3>13. Choosing the right approach</h3><p>SituationFirst approach to considerWhyThousands of HTTP requests with true async clientsasyncioEfficient I/O waitingBlocking SDK from an async endpointrun_in_threadpool / asyncio.to_threadKeep loop responsiveBatch downloads with sync librariesThreadPoolExecutorSimple bounded I/O concurrencyLarge pure-Python calculationsProcessPoolExecutorUse multiple CPU cores despite standard GILNeed strong worker memory isolationmultiprocessingSeparate process address spacesShared state accessed by multiple threadsLock, Queue, immutable statePrevent races and ownership confusionLimit in-flight async API callsasyncio.SemaphoreCooperative capacity controlLimit sync workers performing model inferenceShared capacity limiter / semaphoreProtect scarce compute and memory resourcesNative ONNX/NumPy inferenceBenchmark threads, native settings, and processesNative scheduling/GIL release vary</p><p>Decision tree:</p><pre>Is the workload waiting on network/disk most of the time?<br>  Yes -&gt; Is an async-compatible API available?<br>           Yes -&gt; asyncio + async libraries<br>           No  -&gt; threads / thread pool<br>  No  -&gt; Is it CPU-heavy pure Python?<br>           Yes -&gt; processes (or evaluate newer interpreter/free-threaded options)<br>           No  -&gt; Native computation? Benchmark thread/process options and runtime settings</pre><pre>Need shared mutable state? -&gt; establish ownership or use synchronization.<br>Need a concurrency cap?  -&gt; use the appropriate scoped limiter.<br>Need a service-wide cap across replicas? -&gt; coordinate across processes/hosts.</pre><h3>14. FastAPI and OCR inference: a practical case study</h3><p>Consider a FastAPI service that loads ONNX OCR engines at startup and uses them to recognize uploaded images. Its execution involves three different concerns:</p><p><strong>Lifecycle:</strong> Models load during application startup, not opportunistically during module import. The application should not claim readiness before required engines are usable.</p><p><strong>Responsiveness:</strong> Blocking inference should not execute directly on the event-loop thread in an async def route.</p><p><strong>Resource safety:</strong> Limit simultaneous model calls based on CPU, RAM, native runtime threads, memory usage, and engine thread-safety requirements.</p><p>Illustrative route (not a drop-in application):</p><pre>from fastapi import APIRouter, Request<br>from starlette.concurrency import run_in_threadpool</pre><pre>router = APIRouter()</pre><pre>@router.post(&quot;/recognize&quot;)<br>async def recognize(request: Request):<br>    image = await request.body()  # Illustrative: real APIs should validate size and decode<br>    engine = request.app.state.engine<br>    result = await run_in_threadpool(engine.recognize, image)<br>    return {&quot;result&quot;: result}</pre><p>A simplified lifecycle:</p><pre>Uvicorn process begins<br>  -&gt; imports application modules<br>  -&gt; starts FastAPI lifespan<br>  -&gt; initializes model engines<br>  -&gt; startup completes / application becomes ready<br>  -&gt; request coroutine receives an image<br>  -&gt; worker thread executes synchronous OCR<br>  -&gt; request coroutine resumes and responds<br>  -&gt; shutdown closes engines</pre><h3>AnyIO thread limiter is not a global request limiter</h3><p>Starlette uses anyio.to_thread.run_sync for synchronous endpoints, file operations, and other sync work. Its default capacity limiter is shared by multiple such operations; the documented default is <strong>40 tokens</strong>, not necessarily 40 total HTTP requests. The effective setting is per relevant application runtime, not a distributed global quota.</p><pre>import anyio.to_thread</pre><pre>limiter = anyio.to_thread.current_default_thread_limiter()<br>limiter.total_tokens = 4</pre><p>This caps concurrent operations using that default AnyIO thread limiter in the current runtime. It does not constrain native ONNX internal threads, other process workers, or asynchronous endpoints that never use the thread limiter.</p><h3>Thread safety is different from limiting concurrency</h3><p>A limit of two concurrent inference calls is a <strong>capacity policy</strong>, not evidence that the engine is safe to invoke from two threads. If an engine is not reentrant, it may require a dedicated lock, single-owner worker, or a different instance-per-worker design. Conversely, a thread-safe engine may still need a concurrency limit to prevent memory exhaustion or CPU oversubscription.</p><h3>Why model-level safety matters</h3><p>An inference engine may or may not support simultaneous calls on the <strong>same instance</strong>. This is a library- and engine-specific contract; thread offloading alone does not establish safety. Also note that model loading and teardown can have different concurrency requirements from inference. Validate runtime documentation and load-test with realistic images.</p><p>With four server processes and a per-process limit of two inferences, up to eight inferences might run across the service, assuming enough requests and resources. If you need at most two <strong>globally</strong>, you need a service-wide coordination mechanism or a deployment design that enforces it.</p><h3>15. Common Misconceptions: Six Fast Corrections</h3><p><strong>“Async means another thread.”</strong> No. A coroutine can run on the existing event-loop thread; await does not automatically spawn an OS thread.</p><p><strong>“More threads always mean more speed.”</strong> No. Contention, scheduling overhead, the GIL, and native-library threading can make extra threads slower.</p><p><strong>“A semaphore makes shared state thread-safe.”</strong> Not necessarily. A semaphore with capacity greater than one permits simultaneous access; even a one-permit semaphore helps only if every relevant access follows the same discipline.</p><p><strong>“My API handles 1,000 requests, so it can run 1,000 inferences.”</strong> No. In-flight requests may be awaiting I/O or capacity tokens while only a few inference calls are running.</p><p><strong>“Timeout means the worker stopped.”</strong> No. Cancelling an async wait does not generally terminate an already-running worker thread.</p><p><strong>“FAR and throughput improve when I add threads.”</strong> These are different concepts: verification error rates are model-evaluation metrics, while throughput depends on execution resources and workload.</p><h3>16. Testing, benchmarking, and production pitfalls</h3><p>A correct concurrency design must be measured and tested, not selected solely from API names.</p><h3>Measure at least</h3><ul><li>Throughput (requests/second) and latency (P50, P95, P99).</li><li>Active requests, queued work, worker occupancy, and rejection/backpressure.</li><li>CPU usage, RAM, model memory and thread counts.</li><li>Errors, timeouts, cancellations, and shutdown behavior.</li><li>Results across concurrency levels <strong>1, 2, 4, 8, …</strong>, using representative input sizes.</li></ul><h3>Common mistakes</h3><ol><li>Calling time.sleep(), a sync HTTP client, or CPU-heavy Python code directly inside an async route.</li><li>Assuming async def makes any function non-blocking.</li><li>Assuming await means executing in another thread.</li><li>Assuming the GIL prevents all races or makes all list/dict workflows safe.</li><li>Assuming a semaphore creates, schedules, or limits the number of OS threads.</li><li>Assuming a per-process semaphore protects the entire deployment.</li><li>Increasing thread count without considering native runtime threads or memory.</li><li>Using processes without accounting for serialization, model duplication, and start-method behavior.</li><li>Assuming cancellation or a timeout forcibly stops a running thread.</li><li>Benchmarking with sleep-only workloads and extrapolating to real OCR inference.</li><li>Holding a blocking threading lock across await in event-loop code.</li><li>Creating unlimited tasks and confusing high concurrency with unlimited capacity.</li></ol><h3>What to record in a reproducible benchmark</h3><p>Record the Python version and build (including GIL mode), CPU architecture and core count, number of server workers, thread and process limits, native runtime thread configuration, representative request/input sizes, warm-up procedure, and whether time includes preprocessing or network I/O.</p><p>For a meaningful comparison, measure each implementation under the <strong>same workload</strong> and report response-time distributions as well as throughput. time.sleep examples teach scheduling behavior; they are not evidence that OCR inference will scale similarly.</p><h3>Production checklist</h3><ul><li>Identify CPU-bound, I/O-bound, and native stages separately.</li><li>Define what needs to be concurrent and what must be serialized.</li><li>Document shared mutable state and ownership.</li><li>Choose synchronization that matches <strong>threads</strong>, <strong>async tasks</strong>, or <strong>processes</strong>.</li><li>Know the scope of every limit: task, interpreter, process, host, or whole service.</li><li>Keep blocking operations off the event loop.</li><li>Bound queues, add appropriate timeouts, and decide overload behavior.</li><li>Define engine initialization, readiness, failure, and shutdown behavior.</li><li>Test cancellation, exceptions, race conditions, and worker saturation.</li><li>Measure with production-like workloads before tuning.</li></ul><h3>17. Glossary and references</h3><p>TermOne-sentence definitionProcessA running program instance with its own resources and memory space.ThreadAn OS-scheduled execution path inside a process.ConcurrencyProgress on multiple tasks during overlapping time periods.ParallelismMultiple computations executing at the same instant.CoroutineA resumable computation defined in Python using async def.TaskAn object that schedules and tracks a coroutine within asyncio.Event loopThe scheduler/reactor that runs ready async tasks and handles waits.FutureAn object representing a result that may not yet be available.Critical sectionCode that must coordinate access to shared state.Race conditionA correctness bug that depends on execution interleaving.LockMutual exclusion for a protected section.SemaphoreA bounded number of permits for concurrent entry.DeadlockA waiting cycle that prevents participants from progressing.BackpressureSlowing, rejecting, or buffering inputs to match capacity.GILA CPython interpreter lock relevant to Python bytecode execution.</p><h3>Official references</h3><ul><li><a href="https://docs.python.org/3.14/library/threading.html">Python </a><a href="https://docs.python.org/3.14/library/threading.html">threading</a></li><li><a href="https://docs.python.org/3.14/library/asyncio-task.html">Python </a><a href="https://docs.python.org/3.14/library/asyncio-task.html">asyncio coroutines and tasks</a></li><li><a href="https://docs.python.org/3.14/library/multiprocessing.html">Python </a><a href="https://docs.python.org/3.14/library/multiprocessing.html">multiprocessing</a></li><li><a href="https://docs.python.org/3.14/library/concurrent.futures.html">Python </a><a href="https://docs.python.org/3.14/library/concurrent.futures.html">concurrent.futures</a></li><li><a href="https://docs.python.org/3.14/howto/free-threading-python.html">Python free-threading guide</a></li><li><a href="https://docs.python.org/3.14/whatsnew/3.14.html">Python 3.14: what’s new</a></li><li><a href="https://www.starlette.io/threadpool/">Starlette thread pool</a></li></ul><p><strong>Final principle:</strong> Choose a concurrency mechanism based on where the time is spent, which resources are shared, what correctness guarantees are required, and where the capacity limit must apply. No single mechanism is optimal for all Python workloads.</p><p><em>Author’s note: This guide focuses on Python execution and engineering trade-offs. The FastAPI/OCR example is illustrative and should be adapted to the thread-safety, lifecycle and resource requirements of the actual inference engine.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6423378d0fd7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Benchmarking Face Verification Models for an eKYC System]]></title>
            <link>https://medium.com/@afsane.narimani/benchmarking-face-verification-models-for-an-ekyc-system-bbfa0e219040?source=rss-4fbb23802b0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/bbfa0e219040</guid>
            <category><![CDATA[computer-vision]]></category>
            <category><![CDATA[biometrics]]></category>
            <category><![CDATA[face-recognition]]></category>
            <category><![CDATA[ekyc]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Afsaneh Narimani]]></dc:creator>
            <pubDate>Wed, 29 Jul 2026 07:09:20 GMT</pubDate>
            <atom:updated>2026-11-17T08:52:02.949Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K3QFoA3nqI0v6GpXwZfEJA.png" /></figure><p><em>An engineering-focused comparison of six face recognition models using verification accuracy, security-related error rates, processing throughput, model size, and deployment constraints.</em></p><h3>Why Model Selection Matters in eKYC</h3><p>Face verification is an important component of electronic Know Your Customer (eKYC) systems. In a typical verification flow, a face captured from a user is compared with a reference image to determine whether they belong to the same person.</p><p>Choosing a model for this task is not simply a matter of maximizing accuracy. A false acceptance may allow an unauthorized match, while a false rejection can prevent a legitimate user from completing verification. In practical systems, computational cost, response time, model size, and deployment requirements matter as well.</p><p>To explore these trade-offs, I developed a Python-based benchmark framework to evaluate six face verification models for an eKYC-like use case. The objective was to select a promising model for further MVP validation, not to certify a production-ready identity verification system.</p><h3>Models Considered</h3><p>The initial comparison included <strong>ArcFace, MobileFaceNet, GhostFaceNet, EdgeFace, FaceNet, and AdaFace</strong>. These represent different trade-offs between recognition performance and deployment cost. Since names alone do not uniquely identify model architectures or training checkpoints, the results should be understood as comparisons of the model artifacts used in this benchmark, not universal rankings of these model families.</p><h3>Evaluation Pipeline</h3><p>The benchmark applied the following steps to each face-image pair:</p><ol><li><strong>Face detection and validation:</strong> identify and validate the relevant face in each image.</li><li><strong>Face alignment:</strong> normalize detected faces to provide more consistent model inputs.</li><li><strong>Embedding extraction:</strong> generate a numerical representation of each face using the selected recognition model.</li><li><strong>Similarity calculation:</strong> compare the two embeddings using cosine similarity.</li><li><strong>Threshold-based prediction:</strong> accept or reject a match by comparing the similarity score with a fixed threshold.</li><li><strong>Metric collection and reporting:</strong> aggregate verification outcomes and runtime statistics to compare candidate models.</li></ol><p>The initial benchmark report describes an LFW evaluation with <strong>6,000 requested image pairs</strong> and a balanced-pair sampling strategy. The repository implements a balanced sampler, but the publicly inspected runner does not establish whether that sampler was enabled for the reported run. Consequently, the exact evaluated class balance and the number of successfully processed versus skipped pairs should be confirmed from the original run artifacts. The comparison below reproduces the reported results rather than claiming a fresh independent execution.</p><p>The runner code uses a <strong>fixed cosine-similarity threshold of 0.5</strong> for its match predictions. This is a benchmark operating point, not a threshold calibrated or approved for deployment.</p><h3>Metrics: Accuracy Is Only Part of the Picture</h3><p><strong>Accuracy</strong> measures the overall proportion of correct match/non-match decisions. It is easy to understand, but it does not fully describe risk at the false-acceptance rates that an identity system may require.</p><p><strong>False Acceptance Rate (FAR)</strong> measures how often non-matching comparisons are accepted. <strong>False Rejection Rate (FRR)</strong> measures how often matching comparisons are rejected. For pair-level biometric matching, the related terminology <strong>False Match Rate (FMR)</strong> and <strong>False Non-Match Rate (FNMR)</strong> can be more precise; the benchmark report uses FAR and FRR.</p><p><strong>Equal Error Rate (EER)</strong> is the operating point where the false-match and false-non-match rates are equal or approximately equal. <strong>TAR@FAR</strong> describes the true acceptance rate at a specified false-acceptance target. These are useful for understanding how changing the threshold affects the security–usability trade-off.</p><p><strong>Processing throughput</strong> was calculated in the runner as the number of successfully processed image pairs divided by elapsed benchmark time. The original report calls this <em>FPS</em>; in this article it should be interpreted as <strong>processed pairs per second</strong>, including the relevant pipeline work, rather than standalone neural-network frames per second.</p><p><strong>Model size</strong> is the stored model artifact size in megabytes. This is useful for packaging and distribution, although it does not directly measure peak memory consumption. Finally, a <strong>weighted decision score</strong> combines selected metrics into an engineering ranking.</p><h3>Results: Six-Model Comparison</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jb1GondAa1QXUxMNdgo7aQ.png" /><figcaption><em>Figure 1. Reported results of the initial six-model benchmark on LFW, ordered by the weighted decision score. “FPS” in the source report denotes processed image pairs per second in the benchmark runner. Source: </em><a href="https://github.com/AfsanehNarimani/ekyc-face-verification-benchmark/blob/main/ekyc-face-verification-benchmark.md"><em>project benchmark report</em></a><em>.</em></figcaption></figure><p>The results highlight why a single metric is not enough for a deployment decision.</p><p><strong>FaceNet achieved the highest reported accuracy (96.28%)</strong>, but its model artifact was considerably larger than EdgeFace. <strong>GhostFaceNet recorded the highest reported processing throughput (2.00 pairs/s)</strong>, while <strong>EdgeFace was the smallest model (6.94 MB)</strong> and achieved the highest composite decision score (<strong>73.15</strong>).</p><p>EdgeFace was therefore selected as an <strong>initial candidate for further eKYC MVP validation</strong>. It did not win every individual metric; it ranked highest under the particular scoring policy used in this benchmark.</p><h3>How the Weighted Decision Score Was Calculated</h3><p>The repository’s DecisionMatrix assigns weights to six criteria:</p><p>CriterionWeightAccuracy35%FAR20%Speed15%Model size10%CPU latency proxy10%ONNX deployment indicator10%</p><p>These weights reflect an engineering preference for verification performance and low false acceptance while accounting for deployment footprint.</p><p>It is important to understand the implementation: the score uses heuristic transforms, including model size and FPS-based terms, and <strong>uses measured throughput again as a proxy for CPU latency</strong> rather than a separate latency measurement. ONNX availability is treated as a simple deployment indicator. Therefore, this score is a project-specific ranking tool — not an independently validated measure of security, real CPU latency, or production readiness. Different weights or a revised normalization strategy could produce a different ranking.</p><h3>Interpreting the Zero-FAR Results</h3><p>Several models reported <strong>0.00% FAR</strong> at the benchmark’s fixed operating threshold. This does <strong>not</strong> mean those models have zero security risk. It only means no false acceptance was reported among the evaluated non-matching comparisons under those conditions and at the precision shown. A finite test set cannot establish a zero false-acceptance probability in the real world.</p><p>A deployed eKYC solution requires threshold selection against a clearly defined risk target, followed by evaluation on independent, representative samples. A model selected at a fixed threshold on LFW may behave differently with mobile camera images, lower image quality, more diverse populations, or adversarial attempts.</p><h3>Lessons Learned</h3><p>This benchmark reinforced three engineering lessons.</p><p>First, <strong>verification accuracy is not enough</strong>. False matches and false non-matches need to be examined separately, and operational thresholds should reflect the application’s risk tolerance.</p><p>Second, <strong>model selection is a trade-off</strong>. A slightly larger or more accurate model is not automatically the better choice for a service with latency, memory, or deployment constraints. Small ONNX artifacts can simplify packaging, but runtime behavior must still be measured in the intended environment.</p><p>Third, <strong>benchmark methodology matters as much as the reported numbers</strong>. Sampling, preprocessing, threshold selection, hardware conditions, skipped inputs, and scoring rules can affect comparisons. Reporting these details is essential for a reproducible and defensible decision.</p><h3>Limitations and Next Steps</h3><p>This was an <strong>offline face verification benchmark</strong>, not an end-to-end eKYC security evaluation. The public summary does not provide enough information to reproduce all reported numbers independently, including complete hardware settings, resolved model checkpoint identities, original processed/skipped pair counts, and run-level sampling configuration.</p><p>LFW also does not represent every operating condition of a banking onboarding service. The next validation stages should include independent threshold calibration, diverse and representative capture conditions, pose and age variation, poor lighting and blur, real mobile video, and separate testing of presentation attacks and liveness mechanisms. Inference latency, throughput under concurrent load, peak RAM consumption, and error rates should be measured on the intended deployment hardware.</p><p>Face verification and liveness detection solve different problems. A strong similarity score does not prove that a live person is present in front of the camera.</p><h3>Conclusion</h3><p>The initial six-model comparison identified <strong>EdgeFace as a promising starting point</strong> for an eKYC MVP because it combined competitive reported verification accuracy with a small model artifact and the highest score under the project’s stated decision policy. The finding is a model-selection hypothesis to validate, not a claim of production-grade security.</p><p>More broadly, the project shows why practical AI engineering requires both <strong>model evaluation</strong> and <strong>systems thinking</strong>: performance metrics, risk-sensitive thresholds, reproducibility, runtime cost, and deployment constraints all contribute to selecting a model responsibly.</p><p><strong>Project repository:</strong> <a href="https://github.com/AfsanehNarimani/ekyc-face-verification-benchmark">ekyc-face-verification-benchmark</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bbfa0e219040" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>