<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sudhendu Rai on Medium]]></title>
        <description><![CDATA[Stories by Sudhendu Rai on Medium]]></description>
        <link>https://medium.com/@sudhendu.rai?source=rss-9fb5db03b1df------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*1Q--BO_PghydiWznqCpShw.jpeg</url>
            <title>Stories by Sudhendu Rai on Medium</title>
            <link>https://medium.com/@sudhendu.rai?source=rss-9fb5db03b1df------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 22:07:22 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sudhendu.rai/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Where to Intervene: A Framework for Transforming Complex Processes]]></title>
            <link>https://medium.com/@sudhendu.rai/where-to-intervene-a-framework-for-transforming-complex-processes-3ecf632f7607?source=rss-9fb5db03b1df------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ecf632f7607</guid>
            <category><![CDATA[lean-manufacturing]]></category>
            <category><![CDATA[systems-thinking]]></category>
            <category><![CDATA[operation-research]]></category>
            <category><![CDATA[process-improvement]]></category>
            <category><![CDATA[organizational-change]]></category>
            <dc:creator><![CDATA[Sudhendu Rai]]></dc:creator>
            <pubDate>Sat, 14 Nov 2026 03:26:17 GMT</pubDate>
            <atom:updated>2026-11-14T03:26:17.173Z</atom:updated>
            <content:encoded><![CDATA[<p>A framework developed retrospectively from print-production cases: remove what blocks flow, restructure what holds the present form in place, and model the behavior when you can</p><p><em>Sudhendu Rai | Independent Researcher</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*RLyXxqCgzG2YZY7TlMDlSA@2x.jpeg" /></figure><p>Knowing where to intervene is the hard part.</p><p>Much of my work over the years has involved transforming complex business processes. One recurring challenge is not implementing a change but discovering where to intervene. Once a successful transformation is in place, the mechanism can look obvious. Beforehand, with many possible levers and uncertain consequences, it rarely does.</p><p>Reflecting on these experiences, I have developed a framework built on two ideas: remove what obstructs flow, and restructure the relationships that hold the present configuration in place. When the complex behavior of a system can be modeled well enough, complexity science adds mathematical tools to analyze it.</p><p>These are not rival theories; they are ways to narrow a daunting field of possible interventions. I should also be explicit about the order of events. This framework was developed after the fact. It did not exist when the transformations described here were carried out, and it did not guide them. I offer it for future transformations, and I use the earlier cases only to test whether it would have helped.</p><p><strong>Why successful transformations look deceptively simple</strong></p><p>Consider a familiar scheduling rule. For a single machine with all jobs available at the outset, sequencing them by shortest processing time minimizes average completion time. Once that policy is adopted and the mean falls, the explanation seems simple: we put the shorter jobs first. But that is a result viewed from the endpoint, with the right rule already identified and its assumptions understood.</p><p>At the outset, a complex operation presents a much larger search problem. We might alter scheduling, batch sizes, staffing, cross-training, resource allocation, equipment layout, incentives, operating procedures, or the organization of work itself. Some interventions are parametric; others are structural, functional, or behavioral. They interact, and a change that helps one objective can harm another. Which lever should we try, and why? The fair test of a framework is not whether it explains a success afterward, but whether it narrows this search beforehand.</p><p>Models can narrow the possibilities, but they may be unavailable, incomplete, or misleading — particularly when informal cooperation and human incentives matter. The two ideas that follow help structure the search for where to act.</p><p><strong>Remove what obstructs the desired behavior</strong></p><p>Think of a river blocked by a log jam. Whoever clears the key log does not create the water or push it along. Removing the obstruction lets the water flow according to the conditions already present.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FvDMZ1dNNV7qy91ZBsEMkg@2x.jpeg" /></figure><p><em>Removing the right obstruction lets flow resume.</em></p><p>For system transformation, the heuristic is straightforward: identify a capability or behavior you want to see, investigate what prevents its expression, and remove or alter that obstruction. The intervention is directed at the barrier, not necessarily at the component that appears to be underperforming. Typical candidates are handoffs, rigid skill boundaries, batching rules, approval queues, and incentives.</p><p><strong>Restructure the relationships that hold the present form in place</strong></p><p>A second starting point is to ask why the present configuration persists. Behavior is often attributed to the parts, when it may be sustained by the relationships among them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*luWcGRJrujcYd9dTRrhTEQ@2x.jpeg" /></figure><p><em>Restructuring departments into connected cells changes how work flows.</em></p><p>Consider a whirlpool. It has a recognizable form and measurable behavior, but that form persists through an ongoing arrangement of flow. Change the currents, boundaries, or surrounding conditions, and the whirlpool may shift or disappear. Its recognizable identity is not independent of the relations through which it exists.</p><p>Ask which relationships, interactions, rules, and dependencies continually reproduce the configuration. An intervention can then target those sustaining relations instead of forcing each component to behave differently. The aim is to restructure them so that a more efficient underlying structure can emerge, though whether it does is something to test, not assume.</p><p><strong>When the behavior can be modeled</strong></p><p>When the behavior of a system can be modeled well enough, complexity science offers a further tool. Dynamical-systems and simulation models can show whether a system has more than one stable behavior, how changes in its parameters could shift it from one to another, and whether a feasible route exists to a better operating regime. This is more precise than the first two ideas, but it depends on the adequacy of the model. Organizational behavior, incentives, tacit skills, changing workloads, and informal cooperation can matter greatly, yet they are difficult to capture in a complete model.</p><p>Where a good model exists, use it to test and quantify the two ideas. Where it does not, use them as questions that guide observation and small experiments, not as substitutes for evidence.</p><p>The approaches can also work together. A barrier is often sustained by relationships, and a relationship may function as a barrier. Once we understand enough of the system, a model can help test whether altering either one is likely to produce the intended result.</p><p><strong>Testing the framework on print production</strong></p><p>Cells instead of departments. One setting for testing these ideas is our Lean Document Production (LDP) work at Xerox, which was carried out without this framework. In conventional transaction-print operations, printing and finishing equipment are often grouped in separate departments. Jobs wait between stages, and the organization of work can prevent available capacity and skills from being used effectively. We regrouped the same kinds of equipment into cross-trained cells and processed jobs in smaller batches. Simulation comparisons generally showed reductions in waiting and cycle time, although results depended on workload and operating assumptions.</p><p>Applying the framework in hindsight, I can map the two ideas onto the case. That is the vantage this article cautions against; the better test is whether they would have narrowed the search in advance. Departmental handoffs and rigid skill boundaries were obstructions to flow — the kind of issue the first idea makes visible. The existing configuration was also sustained by assignments, transfer rules, and divisions of responsibility — the relationships the second idea encourages us to question. Simulation then helped us compare the performance of alternative arrangements.</p><p>A contest that removed cooperation. In another transaction-print operation, cross-trained operators could help neighboring cells clear jams and resolve problems. We introduced a small weekly reward for the highest-performing cell. Cooperation within cells remained strong, but helping across cells declined. The operators had not lost their skills; the incentives governing how they used those skills had changed.</p><p>Here the test can be run forward. Suppose we had asked, before introducing the reward, what sustains cooperation across cells. The answer was visible in the operation: helping a neighboring cell carried no competitive cost until the contest ranked cells by output. That question would not have predicted the magnitude of the effect, but it might have identified the risk to cooperation and suggested testing the incentive across a small group of interacting cells before introducing it throughout the operation.</p><p>This illustrates the reverse lesson. A reward meant to improve performance had become an obstruction to cooperation, while also altering the relationships that had sustained it. Changing the conditions of a process can enable better behavior, but it can also undermine what already works.</p><p><strong>Choosing an intervention responsibly</strong></p><p>These approaches do not guarantee success. The hard work is identifying the right obstruction, dependency, or parameter — and understanding what else it affects. Removing a barrier can expose a system to harm; changing a relationship can have unexpected effects; and some transitions may be difficult to reverse. Candidate interventions still need to be assessed for expected benefit, feasibility, cost, risk, and reversibility, and simulation and controlled experiments can narrow the choices further. The framework does not eliminate the search problem; they direct attention toward mechanisms that might otherwise be overlooked.</p><p>A practical sequence is to define the desired behavior; identify what blocks it and what sustains the current configuration; formulate candidate changes; test them as safely and reversibly as possible; and, where data and models permit, quantify the consequences.</p><p><strong>Conclusion</strong></p><p>Two questions guide where to look. What obstructs the behavior I want? What relationships hold the current configuration in place? And when the behavior can be modeled well enough, models can test and quantify the answers. Changing the visible components is not the only way to change a system. This framework was developed after the fact, and I offer it for future transformations, not as an account of how past ones were done.</p><p>Sometimes transformation requires building something new. Sometimes it requires removing an obstruction. The skill is knowing which, and what the obstruction was holding up.</p><p><strong>Selected references</strong></p><p>Rai, S., Duke, C. B., Lowe, V., Quan-Trotter, C., &amp; Scheermesser, T. (2009). “LDP lean document production — OR-enhanced productivity improvements for the printing industry.” Interfaces, 39(1), 69 — 90.</p><p>Rai, S. (2026, November 9). “When the Prize Beats the Plant: A Game-Theoretic Analysis of How a Weekly Contest Broke Cooperation.” Medium. https://medium.com/@sudhendu.rai/when-the-prize-beats-the-plant-a-game-theoretic-analysis-of-how-a-weekly-contest-broke-cooperation-e7891cf2a093</p><p><em>The views expressed are my own.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ecf632f7607" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When the Numbers Balance, but the Transactions Do Not]]></title>
            <link>https://medium.com/@sudhendu.rai/when-the-numbers-balance-but-the-transactions-do-not-4d7b73826bfc?source=rss-9fb5db03b1df------2</link>
            <guid isPermaLink="false">https://medium.com/p/4d7b73826bfc</guid>
            <category><![CDATA[fintech]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[accounting]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Sudhendu Rai]]></dc:creator>
            <pubDate>Wed, 11 Nov 2026 16:59:53 GMT</pubDate>
            <atom:updated>2026-11-11T16:59:53.785Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Why complex financial reconciliation requires more than matching debits and credits</em></p><p><em>By Sudhendu Rai, Independent Researcher</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*KQXLK5uBsXXy0pIWb7Hwbw@2x.jpeg" /></figure><p><em>Reconciliation is reconstructing the relationships behind the numbers, not just finding amounts that offset.</em></p><p>A debit of $10,000 and a credit of $10,000 appear to be a perfect match. But are they related?</p><p>That is the fundamental challenge in financial reconciliation. Two transactions can balance perfectly and still have nothing to do with each other. Conversely, transactions that belong together may not resemble a conventional matching pair at all.</p><p>Consider a pattern familiar to anyone in operations. A $15,000 settlement arrives as a single line on an independent statement. In the ledger, it appears as three entries of $7,000, $5,000 and $3,000, posted on different days, none carrying the settlement’s reference. Every entry is correct. The books are in balance. And nothing matches.</p><p>A single financial event can generate several accounting entries, appear across different periods, pass through multiple systems, and acquire different identifiers along the way. The challenge is not simply finding numbers that add to zero. It is reconstructing the relationships behind those numbers.</p><p>This distinction led me to explore several computational approaches to reconciliation, each suited to a different class of problem. Used in sequence, they resolve most of the population cheaply and leave the genuinely hard cases visible.</p><p>In one instance, this approach reduced two weeks of manual review to minutes and automatically cleared more than 80% of the transactions.</p><p><strong>Start with what you know</strong></p><p>The simplest reconciliation problems are often the most numerous. Transactions may share an account, reference, category, or other identifying attribute. Grouping them by these characteristics and examining their net balances can resolve large populations efficiently.</p><p>This is essentially rule-based matching, supported by aggregation techniques similar to pivot tables. The mathematics is straightforward: the signed amounts within a valid group sum to zero (within an allowed tolerance).</p><p>The important part is not the summation. It is how the group is constructed. A well-designed grouping rule uses knowledge of the underlying process to identify transactions that plausibly belong together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*ipbflCjHRIShzkuSXQ2S_g@2x.jpeg" /></figure><p><em>Figure 1. Deterministic grouping and balance validation.</em></p><p>This makes deterministic matching both efficient and explainable. But it only works when the relevant relationships are sufficiently visible in the data.</p><p><strong>Sometimes the relationship is a chain</strong></p><p>Consider a transaction that references another transaction, which in turn references a third. None of the individual pairs may balance. Yet the complete chain may represent a single financial event whose entries collectively offset.</p><p>A conventional matching rule may miss this relationship because it looks for transactions sharing the same identifier or amount. A recursive algorithm takes a different approach: it starts with one transaction, follows its references to related records, and continues exploring until the relevant relationship has been reconstructed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*w-OdFK2JWHhgy9zFwiKU3w@2x.jpeg" /></figure><p><em>Figure 2. Recursive reconstruction of linked transaction relationships.</em></p><p>Mathematically, this resembles traversing a graph in which transactions are nodes and references form connections. Instead of asking which transactions have offsetting amounts, we ask which transactions are connected and whether their combined amounts satisfy the required financial relationship.</p><p><strong>When relationships are missing, search for combinations</strong></p><p>Not every transaction has reliable identifiers. Sometimes the only available evidence is a collection of amounts, dates, and other attributes. A transaction of $15,000 might be offset by three entries of $7,000, $5,000, and $3,000.</p><p>Finding such combinations is a version of the subset-sum problem, a well-known problem in combinatorial mathematics. With 30 candidate transactions, there are more than one billion possible nonempty subsets. Searching every combination is usually impractical.</p><p>But speed is only half the problem. The larger risk is false positives: as the candidate pool grows, so does the number of unrelated combinations that happen to balance. A bigger search does not just cost more. It produces more coincidences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*m_6gBEfqcNKkHs94k42cXA@2x.jpeg" /></figure><p><em>Figure 3. Constrained search for financially balancing combinations.</em></p><p>The solution is to use the structure of the financial problem to narrow the search. Transactions can be restricted by account, type, date range, or other compatible attributes. Searches can begin with small groups and narrow time windows, expanding only when necessary.</p><p>Often, the most important optimization is deciding what not to search.</p><p><strong>Search more efficiently with branch-and-bound</strong></p><p>Branch-and-bound algorithms eliminate entire groups of possibilities when the remaining transactions cannot produce the required balance. At each partial combination, the algorithm calculates whether the unused amounts could still reach the target. If not, it abandons that branch without enumerating its descendants.</p><p>A small example shows how. Suppose the target is $15,000 and the candidates, sorted largest first, are $9,000, $7,000, $5,000, $3,000 and $2,000. The search first tries including the $9,000 entry. Adding the $7,000 would reach $16,000, which overshoots, so that branch is dropped immediately. Skipping the $7,000 leaves no way to make the remaining $6,000 exactly, so the whole $9,000 subtree is abandoned.</p><p>Back at the top, the search skips the $9,000. Skipping the $7,000 as well would leave only $5,000 + $3,000 + $2,000 = $10,000 available against a $15,000 target, so that branch is cut without looking inside it. Taking the $7,000 and the $5,000 leaves $3,000 to find, and the $3,000 entry closes it. Only a handful of the 31 possible subsets were ever examined.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*BUzZ6VQpRVE6dyRvhImAfw@2x.jpeg" /></figure><p><em>Figure 4. Pruning infeasible branches during combinatorial search.</em></p><p>With five candidates the saving is modest. With thirty, it is the difference between a search that finishes and one that does not. The mathematics is established. The practical value comes from combining it with knowledge of the underlying financial process.</p><p><strong>Independent evidence changes the problem</strong></p><p>There is another limitation to searching within a single dataset. Sometimes the information needed to identify a relationship simply is not there.</p><p>An independent transaction record may contain an expected amount, reference, or other attribute that is missing from the accounting entries. One approach starts with the independent record and searches for the accounting entries that represent it. The inverse approach starts with unresolved accounting entries and searches for an independent record that explains them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*9GKm9vXgMWTzFhnJN_-2AQ@2x.jpeg" /></figure><p><em>Figure 5. Forward and inverse matching using independent records.</em></p><p>The independent record provides something numerical matching alone cannot: additional evidence that the transactions actually belong together. A zero-sum combination is only an arithmetic fact. A supported economic relationship is a reconciliation.</p><p><strong>Why one algorithm is not enough</strong></p><p>These approaches solve fundamentally different problems. Rule-based matching works when relationships are explicit. Recursive matching works when relationships are indirect but traceable. Combinatorial matching works when relationships must be inferred from constrained numerical combinations. Independent evidence helps when necessary information is absent from the original dataset.</p><p>The most effective architecture is therefore not necessarily a single sophisticated matching algorithm. It is a progression of complementary methods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/1*pm0WjEdxA6BKbcGxHwoDjQ@2x.jpeg" /></figure><p><em>Figure 6. Progressive reconciliation using complementary methods.</em></p><p>Start with high-confidence rules. Remove the transactions that can be reliably reconciled. Apply relationship-based searches to the remaining population. Use constrained combinatorial methods for more difficult cases. Introduce independent evidence where necessary. As straightforward transactions are resolved, the remaining population becomes smaller but often more complex.</p><p><strong>The danger of matching too aggressively</strong></p><p>There is a subtle risk in automating reconciliation: the more combinations an algorithm searches, the more accidental numerical matches it may discover. If enough transactions are available, some unrelated combinations will inevitably balance.</p><p>Maximizing the number of transactions cleared is not the same as maximizing the number correctly reconciled. A reconciliation system should be able to leave transactions unresolved when evidence is insufficient. In financial operations, an unexplained balance is visible. An incorrect match that has been cleared may be much harder to detect.</p><p><strong>The broader lesson</strong></p><p>Aggregation, graph traversal, combinatorial search, branch-and-bound and record linkage are all established ideas. None of them is new. What matters is recognizing which mathematical structure corresponds to the problem at hand, and being disciplined about where each one stops being trustworthy.</p><p>The same principle extends beyond financial reconciliation. Many operational problems involve incomplete records, fragmented events, and identifiers that do not line up, where relationships must be reconstructed from several sources.</p><p>The goal of reconciliation is not to make the numbers disappear. It is to establish why they belong together.</p><p><em>Disclaimer: The views expressed are my own and do not represent those of any employer or affiliated organization.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4d7b73826bfc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When the Prize Beats the Plant: A Game-Theoretic Analysis of How a Weekly Contest Broke Cooperation…]]></title>
            <link>https://medium.com/@sudhendu.rai/when-the-prize-beats-the-plant-a-game-theoretic-analysis-of-how-a-weekly-contest-broke-cooperation-e7891cf2a093?source=rss-9fb5db03b1df------2</link>
            <guid isPermaLink="false">https://medium.com/p/e7891cf2a093</guid>
            <category><![CDATA[incentives]]></category>
            <category><![CDATA[operation-research]]></category>
            <category><![CDATA[process-improvement]]></category>
            <category><![CDATA[game-theory]]></category>
            <category><![CDATA[lean-manufacturing]]></category>
            <dc:creator><![CDATA[Sudhendu Rai]]></dc:creator>
            <pubDate>Mon, 09 Nov 2026 15:20:35 GMT</pubDate>
            <atom:updated>2026-11-09T15:22:05.877Z</atom:updated>
            <content:encoded><![CDATA[<h3>When the Prize Beats the Plant: A Game-Theoretic Analysis of How a Weekly Contest Broke Cooperation Between Autonomous Cells</h3><p><em>Sudhendu Rai, Independent Researcher</em></p><p>· Nov 9, 2026</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*ylZZjaYnw7ICQPDtBm44wA@2x.jpeg" /></figure><p><em>A weekly output prize made helping a neighboring cell a losing move. (Illustration, AI-generated)</em></p><p>A small weekly contest between autonomous print cells made our most skilled operators stop helping each other, and the shop’s overall output suffered. This is a stylized reconstruction from memory, not a measured case study.</p><p>The shop was a large transaction print operation. It produced printed pages that were inserted into envelopes together with pre-printed inserts. We applied the Lean Document Production (LDP) framework and organized the floor into cells. Each cell held printing and finishing equipment and was staffed by cross-trained operators. We also coordinated procurement from the warehouse so that every pre-printed insert a job needed was at the cell before the job started. It worked well.</p><p>Then we added a competition. The cell with the highest output over a week would win a small cash reward and free pizza. We expected the incentive to make operators work harder.</p><p>Inside each cell, cooperation stayed excellent. Between cells, it broke down. Operators who knew how to clear a jam, or who could have helped with a production step, stayed in their own cells because helping a neighbor might cost them the prize.</p><p>The problem was not that operators stopped caring about production. They cared about winning. Helping another cell improved the shop’s output but could reduce their own chance of getting the prize. What was good for the plant had become bad for the individual cell.</p><p>Could game theory explain that behavior? And could a different incentive restore cooperation without eliminating competition? To find out, I built a small simulation of the shop. Everything below comes from that model, with illustrative parameters. The numbers describe the model, not our floor, and the 3.5% gain I report was never observed in the real operation.</p><p><strong>A stylized shop</strong></p><p>In the model, each cell is one player, because cooperation inside a cell was never the problem. The question is what a cell does when another cell’s equipment jams.</p><p>There are six cells. Three have an expert fixer who clears a jam in about 10 minutes. Three do not, and they take about 28. If an expert agrees to help, she walks over, fixes the jam, and her home cell loses 35% of its output while she is away. With these illustrative numbers, helping is efficient for the shop: the cell that gets help saves far more time than the helper’s cell loses. Each expert cell decides how often to say yes. Cells are scored on output relative to their own no-help target, and the best cell each week takes the whole prize.</p><p>I looked at the same shop through two lenses. In the first, each cell picks its own behavior to maximize its own payoff, which shows what the contest does. In the second, groups of cells that help each other create value together, which shows what cooperation is worth and how its gains could be shared.</p><p>The contest pays experts to withhold help</p><p>In the model, helping another cell is a losing move for an expert, even though full help would raise shop output by about 3.5%.</p><p>When nobody helps, the three expert cells win about 25% of the weeks between them. When all of them help, they win none, because the cells they helped overtake them. Economists call this structure a rank-order tournament (Lazear and Rosen, 1981): only rank is rewarded, not the value a person creates. Tournaments can draw out effort from independent contestants, but Lazear later pointed out (1989) that when contestants can affect each other’s results, a tournament rewards holding back help, and in the extreme sabotage.</p><p>Our cells were not independent. They shared fixers, equipment and floor space. Helping created value for the shop and for the cell that was helped, but it lowered the helper’s score and raised a rival’s. A ranked contest counts both against the helper.</p><p><strong>Cooperation survives only as a fragile equilibrium</strong></p><p>Game theorists call a stable outcome, one where no player gains by changing only her own choice, an equilibrium. The simulated contest has two of them.</p><p>No help is the firm one. An expert who starts helping loses about 8 percentage points of win probability. The simulation also finds a fragile full-help outcome sustained by indifference: when everyone helps, no expert gains by unilaterally withholding help, because none of them can win whatever she does.</p><p>That fragile state disappears as soon as helping costs anything at all, such as the walk, the interruption or the lost rhythm. Every expert then prefers to stop, and the experts also prefer the no-help state. “Work together” is not a stable instruction under this contest. A team bonus on total shop output removes the penalty, but each cell keeps only about a sixth of the value it creates, so that incentive is thin too.</p><p><strong>Crediting the helper restores cooperation</strong></p><p>If the helper is credited with part of the downtime she saves the recipient, helping can become the stable choice. In the model, I moved a share of the recovered output from the recipient’s score to the helper’s, like an assist in basketball. In the base case, no help stops being an equilibrium once that credit reaches about 36%. The credit moves contest score, not money.</p><p>A simple ratio gives a benchmark: the helper’s cost relative to what the recipient saves.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*p2nd96H2bkzL63wQlr7Fyw@2x.jpeg" /></figure><p>Here f is the fraction of the helper cell’s output lost while she is away, m_e is the expert’s fix time, w is the walk time, and m_s is the non-expert fix time. The base case gives 0.30, while the simulated tournament requires 0.36. The higher threshold is consistent with an additional penalty from relative ranking, although the difference also reflects the simulation’s scoring and payoff rules. Varying the inputs moves both numbers together.</p><p>The credit needed rises when helping is costly and falls when the recipient is much slower than the expert. When the gain from help is small, a large credit buys little, and the contest may not be worth running at all.</p><p><strong>Match the reward group to the dependency</strong></p><p>Paying teams of cells instead of single cells restores full cooperation, as long as each team contains the expertise it needs. In the model, I replaced the six-cell contest with a single winner-take-all contest among teams, scored the same way as before: output relative to the team’s own no-help target.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G-tmbrD1SBkH3dZBGQKFkA@2x.jpeg" /></figure><p>In the simulation, each expert’s best response among the strategies tested is full help inside her team and none to other teams. If she alone refuses to help her teammate, her team’s chance of winning drops from 33% to 8%, so helping a teammate is strictly in her interest and helping another team is not. The triples work too, though they are uneven. The team with two experts and one recipient gains less from in-team help, so it wins only about 17% of the weeks, and its two experts can cover for each other, which leaves them indifferent.</p><p>The contest between teams still blocks help across teams, so the gain depends on the dependency lying inside the team. If the needed expert sat in another team, the refusal would come back. The lever is the boundary of the reward group, not competition as such.</p><p><strong>What cooperation is worth, and who could claim it</strong></p><p>Through the cooperative lens, the full shop produces about 424 more units a week, 3.5%, than six cells working alone. Two standard benchmarks say how that gain could be shared. The Shapley value pays each cell its average marginal contribution. The core asks whether any group of cells would do better by walking away.</p><p>In the model, any one expert can serve all the recipients, so experts are substitutes for each other. A near-core allocation, one in which no group of cells has much to gain by leaving, therefore pays each expert roughly what she would produce alone and gives nearly all the surplus, about 140 to 144 units a week each, to the cells that receive help. The experts enable the gain but have little bargaining power. Their operational contribution is valuable, but their bargaining position is weak. The Shapley value is more generous to them, about 35 units each, but it falls outside the core: one expert and the three recipients could do about 72 units better on their own.</p><p>The contest used neither benchmark. It paid only the top cell by rank, whatever value helping another cell created. One caution: this analysis divides output value, so it assumes output can be converted into transferable economic value.</p><p><strong>Design lessons for process owners</strong></p><ol><li><strong>Draw the reward boundary around the dependency</strong>. Prizes for teams that contain the needed expertise removed the conflict between helping and winning.</li><li><strong>Ask whether the reward is relative or absolute</strong>. Ranking cells that share help, equipment or people turns every act of help into a gift to a rival.</li><li><strong>Credit the assist</strong>. Give the helper a share of what she saves the recipient. Size it from the ratio of her cost to the recipient’s saving, then add a margin for rivalry.</li><li><strong>Measure help so it can be credited</strong>. Log each help event with a scan or timestamp, in the same event-log style used for process mining, so assists are visible rather than anecdotal.</li><li><strong>Run the smallest incentive that works</strong>. If the saving from help is small compared with its cost, a contest may not be worth running.</li></ol><p><strong>The detail behind the results</strong></p><p>For readers who want to see the underlying simulation results, the figure below puts four of them side by side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/1*Tpd0jiPjiwPzLDL_L8wv5A@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/549/1*-plGiv0dvu9XjjrDqLPpPg@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/549/1*cydUk_RhvrMQmR3P53w-ug@2x.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/549/1*2rtTm-PJ_-ztCzJvYb3v6A@2x.jpeg" /></figure><p>Figure. (A) An expert’s expected weekly payoff by how much she helps. (B) Shop output at equilibrium as the assist credit rises. (C) Share of weeks won by expert cells and by cells needing help. (D) Gain over standing alone under the Shapley value and a near-core allocation (midpoint of the allowed range).</p><p><strong>Limits</strong></p><p>The numbers in this article come from a stylized model with illustrative parameters, not from shop data, so treat the mechanism as the finding and the thresholds as examples.</p><p>• The scenario is recalled from memory. Jam rates, fix times and the helper’s loss were chosen to be plausible, not measured.</p><p>• Experts are never double-booked. Their simulated utilization is about 11% of working time when all experts help, so this matters little here, but a real shop with scarce experts would differ.</p><p>• Equilibria come from a search over a coarse grid of help levels, checked against 10,000 simulated weeks. Where several equilibria exist, the search finds one near its starting point.</p><p><strong>The larger lesson</strong></p><p>The contest did not change the machines, the operators, their skills or the layout. It changed the incentives governing how those resources were shared, so the shop stayed physically integrated but became behaviorally fragmented.</p><p>In the simulations, competition itself was not the problem. Competition between groups whose performance depended on helping one another was. When the reward group was redrawn around the dependency, cooperation returned without removing the competition.</p><p>The production architecture determines where cooperation creates value. The incentive architecture shapes whether that value is realized. The design principle is to align the boundaries of incentives with the boundaries of interdependence.</p><p><em>The views here are my own and are not endorsed by my employer.</em></p><p><strong>References</strong></p><p>• Lazear, E. P., &amp; Rosen, S. (1981). Rank-order tournaments as optimum labor contracts. Journal of Political Economy, 89(5), 841–864.</p><p>• Lazear, E. P. (1989). Pay equality and industrial politics. Journal of Political Economy, 97(3), 561–580.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e7891cf2a093" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Research Lab to the Shop Floor: How a New Solution Got Institutionalized in a Large…]]></title>
            <link>https://medium.com/@sudhendu.rai/from-research-lab-to-the-shop-floor-how-a-new-solution-got-institutionalized-in-a-large-a73bfe7f93a2?source=rss-9fb5db03b1df------2</link>
            <guid isPermaLink="false">https://medium.com/p/a73bfe7f93a2</guid>
            <category><![CDATA[change-management]]></category>
            <category><![CDATA[lean-six-sigma]]></category>
            <category><![CDATA[innovation-management]]></category>
            <category><![CDATA[operations-research]]></category>
            <category><![CDATA[process-improvement]]></category>
            <dc:creator><![CDATA[Sudhendu Rai]]></dc:creator>
            <pubDate>Fri, 06 Nov 2026 18:27:54 GMT</pubDate>
            <atom:updated>2026-11-06T18:27:54.138Z</atom:updated>
            <content:encoded><![CDATA[<h2>From Research Lab to the Shop Floor: How a New Solution Got Institutionalized in a Large Corporation</h2><p><em>Sudhendu Rai, Independent Researcher.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*HqHI4t8LMPMjcXEzv9mXYA@2x.jpeg" /></figure><p><em>An idea becomes an organization’s way of working one handoff at a time: from lab to training to support to the shop floor.</em></p><p>A good algorithm is not a product. A working prototype is not an offering. And an offering that works in a few pilot sites is not yet something a large company can deliver, support and sell at scale.</p><p>I was part of the team at Xerox that took Lean Document Production (LDP) from a research idea to a method used in print shops across several countries. Most of what is written about LDP covers the technology and the results. Here I want to focus on the part that gets less attention: how a solution from a research lab became part of how a large corporation actually works. The facts below come from our 2009 Interfaces paper on LDP, which described the training and deployment in some detail.</p><p><strong>An unusual route to the field</strong></p><p>In most large companies, a new offering follows a familiar path. A product team conceives it, takes technology from the research lab, develops and tests it, and hands it to sales and service through a structured phase-gate process.</p><p>LDP did not follow that path. The research team developed it directly in customer sites, working side by side with the sales and service people who looked after those sites. At the same time, together with an engineering team from the global services organization, we built the tools and the training that would eventually roll the offering out to the sales and service forces. The technology, the toolkit, the training and the delivery infrastructure were all built at once, not one after another.</p><p>That decision shaped everything that followed. Because the people who would deliver LDP were involved from the start, we learned early what they could and could not do with it.</p><p><strong>Five phases over nearly a decade</strong></p><p>The rollout ran in five overlapping phases:</p><p>1.	<strong>Develop and prove it (1999 to 2001)</strong>. Researchers ran engagements to develop the process, algorithms and software, and demonstrated productivity gains in print shops that Xerox itself operated.</p><p>2.	<strong>Roll it out in the print-operations business (2002).</strong> The research team and the operations management trained a first small group of assessors. This is where the trouble started, as described below.</p><p>3.	<strong>Fold it into an existing corporate program (2002 to 2003). </strong>With the launch of Xerox Lean Six Sigma, LDP became one of the Black Belt tools.</p><p>4.	<strong>Hand over to a delivery organization (2003 to 2006).</strong> The global services organization took over program management, trained more Black Belts, built a complete toolkit package, and set up tiered support.</p><p>5.	<strong>Institutionalize it (2007). </strong>The team studied the barriers that remained and planned next steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/945/1*fzTGp76FWZJ18ZozokLuVg@2x.jpeg" /></figure><p><em>Leadership moved from the research group to a delivery organization over about nine years, while the research team stayed on as the last line of support.</em></p><p><strong>Lesson 1: Design for the people who will actually run it</strong></p><p>In the second phase, we trained four assessors. They had to build simulation models in a commercial simulation package and evaluate scenarios with proprietary algorithms. The work was labor-intensive and complex. It became clear that either future assessors would need much higher technical skills than was typical in the operations organization, or we had to simplify the tools and the work process.</p><p>We chose to simplify. The research team added a front-end interface to the modeling software and automated several model-building steps. We built a scheduling tool based on the LDP algorithms. Later, the delivery organization moved the platform to Java-based products, which made simulations run faster, and created an assessor certification process.</p><p>The pattern is worth stating plainly. A research prototype is built for the people who understand it. A deployable solution has to be built for the people who will use it.</p><p><strong>Lesson 2: Ride an existing corporate vehicle</strong></p><p>The most important decision may have been not to create a new channel. Xerox was launching Lean Six Sigma, which had a community of Black Belts who already had the right profile: spreadsheet proficiency, strong analytical skills, print-shop operating experience and knowledge of the business. LDP became one more tool in their toolbox, and the corporate Lean Six Sigma program became the vehicle for delivering LDP to customers.</p><p>This solved several problems at once. There was a ready pool of trained people, a curriculum to join, and an existing way of selecting and running projects.</p><p>The training itself evolved. The first classes were intensive, highly technical five-day events built around a real case and a real data set, covering the tools from data collection through simulation to scheduling, and ending with presentations of trainees’ own shop recommendations. Trainees had to arrive with a project or site where they would apply LDP right away, and an experienced assessor coached them. As the software and interfaces improved, and as we screened candidates for the right skills, we cut the class from five days to three and a half. Black Belts were eventually offered a week of training.</p><p><strong>Lesson 3: Build the support structure early</strong></p><p>A solution that depends on its creators does not scale. Over time we built a package that contained everything an assessor needed to run a typical customer assessment, including standardized hardware and software. A support center provided first-tier help with setting up equipment and using the toolkit and second-tier help with problems using the application. It also handled installation, setup, testing and shipment of the package. A development team provided third-tier support, such as bug fixes and upgrades, working with the original researchers.</p><p>The structure kept the researchers involved without making them the first call for every question.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*m2l0iNTa33y2HBsQ8VWmzg@2x.jpeg" /></figure><p><em>Training flowed down the chain; support questions escalated down the tiers.</em></p><p><strong>Lesson 4: Tools lose their value when they are not used</strong></p><p>By the fifth phase, the value of the toolkit was widely recognized. But we also saw a quieter problem: when assessors did not use the tools on an ongoing basis, they lost their skills. Training is not a one-time event. It has to be sustained by regular practice, and the organization has to give people reasons and opportunities to keep using what they learned. That is why the final phase was devoted to analyzing the barriers and recommending next steps, not to launching something new.</p><p><strong>Lesson 5: The hardest barriers were cultural</strong></p><p>Some of the most difficult barriers had nothing to do with technology. Print-shop operators were skeptical that significant improvement was possible. Even when an assessment showed a clear opportunity, they were reluctant to move to a new layout and work process, particularly when the change had up-front costs, such as changing the shop layout. Some people feared for their jobs.</p><p><strong>What worked was a combination of things:</strong></p><p>•	<strong>Credible baselines. </strong>We established baseline metrics before making changes, and then monitored the same measures afterward, so that improvements were shown with data, not argued.</p><p>•	<strong>Coaching and mentoring.</strong> Experienced assessors stayed with new ones, sometimes shadowing every step.</p><p>•	<strong>Incentives where appropriate. </strong>In some cases, recognition and rewards tied to throughput and quality helped.</p><p><strong>What it added up to</strong></p><p>The results measured by the company bear out the approach. In the first 17 shops sampled, we saw labor savings of 20 percent and cycle-time reductions of 80 percent. In a later sample of more than 80 shops engaged through the Black Belt program, average cycle time fell by 50 percent and on-time performance rose by 11 percent. Xerox’s finance organization recorded a profit increase of $175 million across the customer value chain from 2003 to 2007, and cumulative profits above $250 million from 1999 to 2008. Implementations involved over 1,000 print-shop personnel in the United States, Canada, China and Thailand. In 2004 the teams received a corporate innovation award, and customers began asking for LDP by name.</p><p><strong>What I would tell any research team</strong></p><p>If I had to compress the experience into a list for a research group that wants its work to survive inside a large organization:</p><p>1.	<strong>Develop it where it will be used</strong>, with the people who will deliver it.</p><p>2.	<strong>Plan the delivery chain </strong>while you build the solution, not after.</p><p>3.	<strong>Simplify until the intended users</strong> can run it, even if that costs some sophistication.</p><p>4.	<strong>Attach it to something the organization</strong> already runs, such as a corporate improvement program.</p><p>5.	<strong>Build tiered support</strong> so the research team is the last tier, not the first.</p><p>6.	<strong>Keep the skills alive</strong> with practice, coaching and certification.</p><p>7.	<strong>Treat culture as a design problem</strong>, and measure before and after so results speak for themselves.</p><p>None of this is as interesting as the algorithm. But in my experience, it is what decides whether the algorithm ever matters.</p><p>This article draws on: Rai, S. et al. (2009), “Lean Document Production,” Interfaces 39(1), 69 — 90. The figures cited are those reported there.</p><p><em>The views expressed are my own and are not affiliated with or endorsed by any employer or organization.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a73bfe7f93a2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Takt Time for Shops Where Nothing Repeats]]></title>
            <link>https://medium.com/@sudhendu.rai/takt-time-for-shops-where-nothing-repeats-8e78abba73e5?source=rss-9fb5db03b1df------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e78abba73e5</guid>
            <category><![CDATA[capacity-planning]]></category>
            <category><![CDATA[process-improvement]]></category>
            <category><![CDATA[simulation]]></category>
            <category><![CDATA[operations-research]]></category>
            <category><![CDATA[lean-manufacturing]]></category>
            <dc:creator><![CDATA[Sudhendu Rai]]></dc:creator>
            <pubDate>Thu, 05 Nov 2026 01:06:46 GMT</pubDate>
            <atom:updated>2026-11-05T01:47:12.914Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Sudhendu Rai,Independent Researcher.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ixEU0D3s3s5QE3ZX8gxFRw.png" /><figcaption>Jobs of different sizes and due dates, stacked into one rising and falling curve. Its peak is a first estimate of the capacity a print shop needs.</figcaption></figure><p>Takt time is the simplest idea in Lean: available time divided by demand. It works beautifully when demand is level. A print shop is the opposite. Jobs arrive at random, each with its own size and its own due date. There is no single takt.</p><p>When we designed autonomous production cells for print shops in Lean Document Production, we still needed a first estimate of how much capacity each function should have. So we generalized takt.</p><h3>Two lines in a paper, and a lot of field use</h3><p>This idea has a small footprint in print. We presented it at the 2008 Franz Edelman competition and described it in a few lines of the <em>Interfaces</em> paper. No analysis, no simulation, just the statement of the idea and one example. Yet Xerox’s LDP consultants used it in many shop engagements. A tool took a shop’s job log and operating hours and produced the generalized takt rate, which gave a first estimate of the capacity each function needed before any simulation was run.</p><p>To my knowledge, work before 2008 dealt with single-stage processing. The LDP version handles functions in series, and the same fluid view also applies inside a cell. That is a lot of weight for a couple of lines, and it is why I have gone back to test the idea properly.</p><h3>The idea</h3><p>Give every job its own takt rate: its quantity divided by the time between arrival and due date. Then lay all those rates on the time axis and add them up. The result is a <em>takt profile</em>, a curve showing how fast the shop must be producing at each moment just to keep every job on schedule. Do it for each function (printing, cutting, finishing). Since small batches let finishing start as soon as printing starts, finishing gets the same time window as printing. That is an approximation, but a useful one. Set each function’s capacity at least equal to the peak of its profile, then refine by simulation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oy2BQ6TbtxmR3Nvgoa3GTQ.png" /><figcaption><em>Five jobs with their windows (top), and the stacked profile for printing and finishing. The peak of each profile is the starting capacity.</em></figcaption></figure><p>Consultants also used it as a diagnostic. Take a shop’s job log and operating hours, compute the generalized takt rate, and compare it with the installed capacity. If the two are close, there is little room to cut. If installed capacity is far above, there may be an opportunity, and simulation optimization can find out how much.</p><h3>Is it a known idea?</h3><p>Yes, in another field, though for a single stage. Computer scientists studying variable-speed processors defined the <em>Average Rate</em> heuristic in 1995: run each job at constant speed between release and deadline, and the processor speed is the sum of the active jobs’ speeds. That is the takt profile. They also solved the optimal version (the YDS algorithm), which gives the true minimum peak speed if every job must meet its deadline. So the profile’s peak is a safe capacity for the single-stage case, but not the minimum one.</p><h3>What I tested</h3><p>Before writing this up I wanted numbers, so I built a discrete-event simulation of a two-stage shop (print, then finish) and ran it over about a hundred scenarios. Job sizes were realistic: lognormal with moderate variation, plus a small/large mix. No extreme heavy tails. I added setups, availability losses, service noise, rush jobs, and a 95% on-time target. One modeling choice turned out to matter a great deal, so I ran two versions. In one, adding capacity also adds setup capacity, like adding machines. In the other, capacity is just a faster machine and setup time stays the same, so each stage has a ceiling on how many batches it can start per hour. Neither is a bound on a real shop, which could do worse than either, so read them as two contrasting assumptions.</p><p><strong>1. Is the same-window shortcut for finishing OK?</strong> Yes, under a condition I could measure. In these experiments, when the worst job’s batch transit time was under about 10% of that job’s lead time, the shortcut was essentially exact. Treat 10% as a guideline for similar shops, not a universal cutoff. By 20 to 40% it understated the capacity needed by a median of 14%, and beyond that by more. Without pipelining (finishing waits for the whole job) it can understate badly.</p><p><strong>2. Is the peak a good starting point?</strong> It is a feasible one. At peak-takt capacity the 95% target was met in all 96 scenarios when capacity scaled cleanly, and in 76 of 96 (79%) when it did not. But it is conservative. The best capacity I found sat at roughly 55 to 70% of the peak in the first model, and 53 to 115% in the second, where large shops need more. The eight hardest scenarios in the second model (the highest load with the smallest batches) needed about 3 to 7 times the peak, because setups eat most of the printing stage at any speed. Cost fell by a median of 41% (first model) and 37% (second) from the takt start. Treat these as “best found”, not proven optima, because they are optimistic. On 16 fresh job logs, the capacities chosen on eight logs averaged about 93.7% on time against the 95% target (93.2% in the second model), and only about half of the scenarios (51% in the first model, 42% in the second) actually met it. Most misses were small. I then scaled capacity up by a factor chosen on those fresh logs and tested it on a second, independent set: average on-time rose to about 96.7%, but roughly one scenario in five still missed 95%. That makes it a useful calibration step, not a service-level guarantee. Validate on fresh logs, and treat any margin as a starting point.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_p9a8yoUcehe1Ouw1GvZsA.png" /><figcaption>Best-found capacity relative to peak takt, by shop scale, under the two capacity models.</figcaption></figure><p>A narrower starting point saved little. Searching for the smallest feasible capacity took about 5 to 7 simulation evaluations from any sensible start, so a takt anchor saved one or two. The value of the takt rate is mostly in calibration and screening, not in speeding the search. I only tested a one-dimensional search, so that may change for richer searches.</p><p><strong>3. Is “installed capacity close to takt means little room” a reliable test?</strong> Partly. Comparing capacity with takt ranks shops well by reduction opportunity, measured against the simulation benchmark (the best capacity found, which as noted can itself be a little undersized). But the peak was not the best anchor. In both models the 95th-percentile of the takt profile ranked shops better than the peak did. When capacity scaled cleanly, the 95th-percentile and the optimal-fluid-floor ratios were the best screens by a wide margin. When setup time was fixed, every anchor did worse, because the hardest scenarios need capacity several times any of them, and plain average load ranked best. A shop sitting right at its peak takt still had a median 38% reduction opportunity relative to that benchmark in my runs, with a wide band. So a ratio near the peak does not mean capacity is tight.</p><p>One more correction to how I used to think about the floor: the optimal-fluid floor assumes every job meets its deadline. With a 95% target the shop can let the toughest jobs go late, and in 13 to 16% of scenarios the best capacity was below that floor. It is a reference level, not a lower limit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qPW3_GZSnBuNcLpaG4GqCg.png" /><figcaption>What a given capacity-to-takt ratio implies about the true opportunity.</figcaption></figure><h3>One more thing hides in the gap</h3><p>When installed capacity is well above takt, it is tempting to call it excess capacity. But dispatch rules matter. The LDP toolkit let users test a variety of scheduling policies against a design, and the Edelman paper lists the usual suspects for sequencing inside a cell: earliest due date, shortest processing time, least slack time, shortest remaining processing time. I compared four at the same capacity: earliest due date, shortest processing time, first-in-first-out, and a simple static priority (due date minus the job’s work), which is a crude stand-in for least slack time, not the full dynamic rule. Earliest due date, shortest processing time and the static priority landed within a couple of percent of each other. First-in-first-out did not: it needed a median 17 to 27% more capacity (depending on the capacity model) to hit the same 95% on-time target, and more when rush jobs were common. (Shortest processing time looks good on the fraction of jobs on time, but it does so by pushing big jobs back, and I did not measure that.) The ratio of capacity to takt cannot tell you whether you are looking at idle machines or poor sequencing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P5k-ONHFPj0WMslMVv7EOw.png" /><figcaption>Capacity needed under FIFO relative to earliest-due-date dispatch.</figcaption></figure><h3>Beyond print shops</h3><p>Nothing in the construction is specific to print. It needs three things: work that arrives at random with a quantity and a due date, several steps in sequence, and work that can move from one step to the next in small pieces, so that a downstream step can start soon after an upstream one does. When those hold, the work behaves like a fluid flowing through a series of pipes, and each step can be given the same arrival-to-due window and its own profile. The step’s capacity is then sized against the peak of that profile.</p><p>Many processes have this shape: document and mail processing, claims handling, reconciliation and other back-office pipelines, order fulfillment, lab testing, and job-shop manufacturing with transfer batches. The same logic applies inside a single cell or team, where the fluid view stands in for the detailed hand-offs between its stations. I have only tested the idea in the print-shop setting described above, so for another process the pipelining condition is the thing to check first. If hand-offs are large or slow, the same-window assumption fails, and the paper shows how quickly the error grows as batch transit time becomes a larger share of a job’s lead time.</p><h3>How I would use it now</h3><ul><li>Compute a ladder of levels from the job log: mean takt, 95th-percentile takt, the optimal fluid floor, and the peak, adjusted for setups and availability.</li><li>Treat the peak as a feasible start, not a bound, and expect the best capacity well below it.</li><li>Use the 95th-percentile or floor ratio as a triage screen, and calibrate it on comparable shops before turning it into a savings estimate.</li><li>Ask about dispatch policy before concluding the capacity is excess, and test the candidate policies in the optimization.</li><li>Check the chosen capacity on fresh job logs. In my runs, the capacity found on a few logs met the target on new logs only about half the time, and a calibrated margin still left about one scenario in five short.</li></ul><p>The takt profile did not turn out to be a precise answer. It turned out to be what we wanted it to be in the first place: a cheap, defensible place to start, from a job log and the shop calendar, that tells you where simulation should look next.</p><p><em>The full working paper, with the model, the simulation design and all results, is on ResearchGate.</em></p><p>Disclaimer: <em>The views expressed are my own and are not affiliated with or endorsed by any employer or organization.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e78abba73e5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Fastest Automation Was the Most Fragile]]></title>
            <link>https://medium.com/@sudhendu.rai/the-fastest-automation-was-the-most-fragile-be16f8081189?source=rss-9fb5db03b1df------2</link>
            <guid isPermaLink="false">https://medium.com/p/be16f8081189</guid>
            <category><![CDATA[data-quality]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[process-improvement]]></category>
            <category><![CDATA[fintech]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Sudhendu Rai]]></dc:creator>
            <pubDate>Tue, 27 Oct 2026 23:27:37 GMT</pubDate>
            <atom:updated>2026-10-28T00:02:54.572Z</atom:updated>
            <content:encoded><![CDATA[<h3>Human Effort Should Scale With Exceptions, Not Transaction Volume</h3><p>Why the fastest automation was also the most fragile, and where to put the human.</p><p>Oct 25, 2026</p><p>By Sudhendu Rai, Independent Researcher</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WKl1B055RjrZlscwL1I8hw.png" /></figure><p>One of the hardest design choices in automation is not how much to automate, but where to leave a person in the loop. This article compares two ways of automating a recurring task, with the manual process as the baseline. The task is fusing, processing, reconciling and analyzing data that arrives from many sources, each in its own semi-structured format, typically a spreadsheet. Versions of this task appear throughout financial services, wherever outside parties send data that must be brought together before anyone can rely on it.</p><h3>The problem</h3><p>On a regular cycle, data begins to arrive from many sources: counterparties, service providers, data providers and internal teams. Each sends semi-structured spreadsheets with its own layout, its own column names and its own meaning attached to them. The data has to be read, mapped to one standard format, fused with the data from every other source, checked against reference data for accuracy and not just format, reconciled, and loaded into an internal application where it is processed and analyzed. The window is short, and decisions downstream depend on the result. The details differ by function, but the structure is the same: many senders, one standard format, and a high cost for errors that reach the final data.</p><p>Three designs have been used for this work: the manual process and two automations. Each one answers the same question differently: where do errors get caught, and who is looking when they do?</p><h3>The baseline: Do it by hand</h3><p>In the manual process, a person opens each file, finds where each piece of data sits, maps it to the standard format, extracts it, and transforms it. Along the way they run checks, including a lookup against standard reference data to confirm the numbers are right, not just well formed.</p><p>This design has one real strength: a person is looking at every file, so judgment is applied everywhere. It also has two weaknesses. It is slow, and a tight window makes it slower still. And its quality depends on a tired person catching every problem on every file, which is inconsistent however skilled they are.</p><h3>Automation 1: A dedicated automation for every source</h3><p>The first automation built a dedicated automation for each input source. Each automation understood one source’s format, pulled the data from the right places, compared it automatically, ran some checks and cleaning, and loaded it into the internal format.</p><p>When the input matched what the automation was built to expect, this worked very well. The trouble came from inconsistency in what sources sent. For certain transactions, the extraction, validation and cleaning broke down, and erroneous data was loaded anyway. Someone then had to find it in the final dataset and correct it by hand.</p><p>This is a tightly coupled pipeline. Nothing sits between the source file and the system of record, so a mistake at the first step becomes a mistake in the final data. The error also loses its context on the way. Once a bad row has joined the good rows from every other source, finding it means searching the whole aggregate, not one file.</p><h3>Automation 2: A semantic layer and a staging buffer</h3><p>The third design changes two things at once.</p><p><strong>A semantic layer replaces the dedicated automations.</strong> Instead of code written for each source, incoming column headers are mapped through a set of aliases to standard fields. A new source, or a source that renames a column, does not need new code. When the tool meets a field it does not recognize, it does not guess. It asks the user to say what the field is, and the import continues.</p><p><strong>A staging buffer sits before the final load.</strong> The tool shows what the source sent beside the transformed, cleaned and validated version, row by row. Wherever the two disagree, the difference is flagged with a visible marker. The user can inspect and edit any row before importing, and only then does the data enter the final dataset.</p><p><strong>Validation is automatic, and a visible marker reports the outcome.</strong> While the data is staged, checks run on key fields. For example, each key identifier a source sends is checked against internal reference data. By hand, that means a spreadsheet lookup for every row. Automatically, it takes seconds. Where a source supplies a field in a non-standard form and the tool can look up or derive the standard form, it fills the value in. The marker then shows what happened, so the reviewer sees which values were confirmed, which were adapted, and which could not be resolved. Markers can be colors, icons, labels or notes. What matters is that each outcome looks different:</p><p>• <strong>Confirmed:</strong> the source’s value matches the reference data exactly.</p><p>• <strong>Adapted or changed:</strong> the source’s value differs from the validated value, or the tool mapped, converted or filled in a value. The user can see what changed.</p><p>• <strong>Unresolved:</strong> the source’s value could not be matched to the reference data, so the row needs manual follow-up.</p><p>The reviewer can skim the confirmed cells and spend their time on the adapted and unresolved ones. More marker types can be added the same way, for example for a column mapped through an alias, a unit conversion, or a figure outside its expected range. What matters is that each type of deviation or adaptation is visible and distinguishable, so attention goes where it is needed.</p><p><strong>New sources do not need a new dedicated automation.</strong> When a source is added, its data can be imported directly. Its headers are matched through the existing aliases, and any field the tool does not recognize is put to the user to identify during the import. Under the dedicated-automation design, every new source meant building another dedicated automation. Here the same tool keeps working, even when a source’s formatting or data is non-standard.</p><p>This design is mostly automated, but it adds a few manual steps compared with the dedicated-automation version. In exchange, it handles formats it has never seen, and errors are visible while they are still cheap to fix.</p><p>As far as I know, this approach was new for this problem in my organization, and I am not aware of it being used elsewhere for this kind of multi-source data work.</p><h3>The three pipelines</h3><p>In the first design a person touches every file. In the second, no one sees the data until it is in the aggregate. In the third, a person sees the exceptions before anything is committed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ydp0jm2hL93c7TqIY_IqSA.png" /><figcaption><em>The same task handled three ways. Blue marks where a person looks at the data; red marks where errors go unseen.</em></figcaption></figure><h3>Side by side</h3><p>The third design gives up some speed in exchange for contained failures and flexibility with new formats.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QJBDS5uLXX1y1rAjFjCIxA.png" /></figure><h3>The science behind it</h3><p>Four ideas explain why the third design behaves differently.</p><p><strong>1. Two kinds of coupling.</strong> In an earlier article on inline automation in production lines, I described availability coupling: when two machines are wired directly together, one stoppage idles the other, and a small buffer between them lets each keep working while the other is down (Gershwin, 1994). Data pipelines have a different coupling. A failing step does not stop. It passes a defect downstream, so I call this error coupling. The staging area is therefore less a queue than a checkpoint. It separates computing a result from committing it.</p><p><strong>2. Containment and search.</strong> Suppose, purely as an illustration, that a dedicated automation mishandles 2 percent of transactions. With nothing between the automation and the dataset, all of that 2 percent reaches the aggregate. If a checkpoint lets a reviewer catch 90 percent of them, the escapes fall to 0.2 percent. In general, escaped errors = e × (1 − d), where e is the error rate and d is the fraction caught at the checkpoint. These figures are not from this project. They only show that the checkpoint’s detection rate sets the outcome.</p><p>Where an error is found matters as much as whether it is found. At staging, the candidates are the rows of one file from one source, with the original beside the transformed version. In the aggregate, the candidates are every row from every source, and the original context is gone. The same defect costs far more to find.</p><p><strong>3. Requisite variety.</strong> Ashby (1956) argued that only variety can absorb variety. Differences between sources’ formats are a source of variety. The dedicated-automation design meets it by enumeration: one automation per format. Every new variation needs new code, and any variation nobody anticipated slips through. The semantic layer instead reduces variety at the boundary by mapping many names onto one standard vocabulary. It then routes what it cannot map to the one component able to absorb the unexpected, a person. The prompt for an unrecognized field is that routing.</p><p><strong>4. The human is a server.</strong> A reviewer has finite capacity and variable speed, so the staging area behaves like a queue and can become a bottleneck as volume grows. Reviewers can also drift toward approving everything when most rows are clean. Bainbridge (1983) observed that automation tends to leave people the hardest part of the task while reducing the practice and attention they need to do it. Visible markers on differences help by directing attention to the rows that need it, instead of asking for equal attention everywhere. The flags themselves behave like a classifier. A false alarm wastes attention, and a miss becomes an escape, so both can and should be measured.</p><h3>A simple model</h3><p>All three designs can be compared with one model, and every parameter in it can be measured.</p><p>Per cycle, a design is judged by three things: how many rows a person must touch, how likely a problem row is to be caught before import, and what it costs when one escapes into the final dataset. Let N be the rows per cycle and e the fraction carrying a problem, either a format variation or a data error. Let H be the human review time, D the probability that a problem row is caught before import, and κ the minutes needed to find and fix an error that has escaped into the aggregate. The total cost of a design per cycle, in minutes, is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/791/1*5t5X-HdGe0ob_1GAdt2tnw.png" /></figure><p>The reporting window adds a constraint: H must fit within the review time available. The three designs differ only in H and D:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hd-pHcK7Jf_p3y26J8k7xg.png" /></figure><p>Here tₘ is the minutes per row by hand, u is the share of problems that come from variations nobody anticipated, and dₐ and dᵤ are the dedicated automation’s detection rates for anticipated and unanticipated problems. For the staging design, ρ is flag recall (the share of real problems that get flagged), π is flag precision (the share of flagged rows that are real problems), and h is the reviewer’s catch rate on flagged rows. The number of flagged rows is F = N e ρ / π; in the tool, these are the rows with an adapted or unresolved marker. Review takes tᵣ minutes per flagged row, and K unmapped-field prompts take tₖ minutes each.</p><p><strong>The dedicated automation wins or loses on u.</strong> Its detection falls as the share of unanticipated variation grows. The semantic layer turns that variation into explicit prompts and flags. So staging should win when sources vary in ways no one planned for, and the dedicated automation should win when formats are stable and the variation is mostly known.</p><p><strong>False alarms are affordable.</strong> Staging has the lower total cost whenever its review time is less than the escape cost it avoids. Substituting F, and ignoring the prompts, this means precision must satisfy:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/490/1*mTvlYjtZOLWlexPDoBuxXQ.png" /></figure><p>The tolerable false-alarm rate is set by how much more an escape costs than a row review. The more expensive an escape, the lower the precision at which reviewing flags still pays.</p><p><strong>Where to stop reviewing.</strong> Rank rows by their estimated probability p of being a problem. Reviewing a row pays off when p × h × κ &gt; tᵣ. If problems concentrate in a small share of rows, which is an assumption to test, the cumulative errors caught rise steeply and then flatten as more rows are reviewed. That is the same shape as the buffer curve in my earlier article: a small amount of slack, here a small amount of review, captures most of the gain.</p><p><strong>An illustration with invented numbers.</strong> These values are chosen to show how the model works. They are not measurements. Take N = 2,000 rows, e = 2 percent (40 problem rows), tₘ = 2 minutes, κ = 45 minutes, tᵣ = 1 minute, five prompts at 3 minutes each, dₕ = 0.85, u = 0.5, dₐ = 0.9, dᵤ = 0.1, ρ = 0.9, π = 0.4 and h = 0.95.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tS9BIsECH-MQInta3DXljw.png" /></figure><p>Change one assumption and the ranking can flip. If only 10 percent of problems come from unanticipated variation (u = 0.1), the dedicated automation’s total falls to 324 minutes and beats staging. With these values, the break-even is about u = 0.13. The model’s value is that it names the conditions under which each design wins.</p><p>The model is simple on purpose. It assumes fixed rates, independent errors and no learning, although in practice the alias set grows each cycle, which should lower u over time. Each parameter can be estimated from the tool’s own logs: ρ and π from the flags and the user’s edits, h and the escapes from errors found after import, tᵣ and tₖ from review times, u from the fields that had to be mapped by hand, and κ from the time spent fixing escapes.</p><h3>Where this leaves the human</h3><p>Automation is often judged by how many human steps it removes. A better question is where it leaves the human, and what it shows them there. In the production lines I wrote about earlier, the answer was a small buffer between machines. Here the same principle takes a different form: a visible checkpoint between the source and the system of record.</p><p>The value comes in three ways: it lowers risk by containing errors before they reach the final data, it scales to new sources without new dedicated automations, and it offers a pattern that can be reused in other automation work.</p><p>The common lesson is to design for how things fail. Wiring steps together makes a demonstration. Understanding where errors enter, how far they travel, and who is positioned to stop them makes a system.</p><h3>References</h3><p>Ashby, W. R. (1956). <em>An introduction to cybernetics</em>. Chapman &amp; Hall.</p><p>Bainbridge, L. (1983). Ironies of automation. <em>Automatica</em>, 19(6), 775–779.</p><p>Gershwin, S. B. (1994). <em>Manufacturing systems engineering</em>.</p><p><em>Disclaimer: The views expressed in this article are my own, based on my personal experience and independent research. They are not affiliated with, endorsed by, or representative of any current or former employer or any organization mentioned. Process details are described in general terms.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be16f8081189" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sort or Slice? Two Ways to Tame Heavy-Tailed Work]]></title>
            <link>https://medium.com/@sudhendu.rai/sort-or-slice-two-ways-to-tame-heavy-tailed-work-55ece6705723?source=rss-9fb5db03b1df------2</link>
            <guid isPermaLink="false">https://medium.com/p/55ece6705723</guid>
            <category><![CDATA[lean-manufacturing]]></category>
            <category><![CDATA[process-optimization]]></category>
            <category><![CDATA[queuing-theory]]></category>
            <category><![CDATA[scheduling]]></category>
            <category><![CDATA[operations-research]]></category>
            <dc:creator><![CDATA[Sudhendu Rai]]></dc:creator>
            <pubDate>Sun, 25 Oct 2026 22:37:42 GMT</pubDate>
            <atom:updated>2026-10-25T22:37:42.112Z</atom:updated>
            <content:encoded><![CDATA[<p>Oct 25, 2026</p><p><em>By Sudhendu Rai, Independent Researcher</em></p><figure><img alt="Two glass boxes. In the left one, a huge block sits at the front of a line of small pebbles, blocking them. In the right one, the block is cut into slices with small pebbles passing between them." src="https://cdn-images-1.medium.com/max/1024/1*YxMg-KZjCtMBELRs9Mv-jg.png" /><figcaption>One giant job blocks the line. Cut into slices, it lets small jobs through. <em>Illustration generated with AI.</em></figcaption></figure><h3>Why small jobs wait</h3><p>In a shop with heavy-tailed work, a few giant jobs decide how long everyone else waits. In the model below, sorting jobs by size cuts the average slowdown from about 370 to about 22. Slicing the giants into small pieces, and letting short jobs slip in between the pieces, cuts it to somewhere between 2 and 8, as long as a slice is cheap to set up.</p><p>Heavy-tailed means most jobs are small and a handful are enormous: a one-page flyer next to a hundred-thousand-piece mailing. In the model used here, job sizes run from 1 to 10,000, and the biggest 1 percent of jobs carry half of all the work. Exponential sizes with the same average would put only about 6 percent of the work in that top 1 percent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*FW6I9X3X7VAT8BgbBVdJPA.png" /></figure><p><em>Figure 1. In a heavy-tailed mix, the largest 1 percent of jobs carry half of all work.</em></p><p>When one of those giants starts, every small job behind it waits, and the wait is enormous next to the small job’s own size. This article looks at two ways to stop that. One is to sort: send small and large jobs to different machines. The other is to slice: cut big jobs into pieces so that small jobs can get in between them.</p><p>Sorting is the subject of an MIT master’s thesis by John Nham (2008) that I co-supervised with Professor John Tsitsiklis. Slicing is the idea from my earlier article on bricks and pebbles, applied to a queue instead of a pipeline. I tested both in the same simulator.</p><h3>What the thesis studied</h3><p>The thesis studies the smallest system that still shows the problem: two machines, jobs arriving at random, and a dispatcher that must send each job to one machine the moment it arrives. Each machine serves its queue in arrival order. Jobs are never split or moved.</p><p>The score is slowdown: the time a job waits divided by its own size. A ten-minute wait is a disaster for a one-minute job and nothing for a ten-hour job, and slowdown captures that.</p><p>Dispatchers come in two kinds. A size-blind dispatcher ignores how big the arriving job is. Random assignment is one. Least-work-first, which sends the job to the machine with less work left, is another. A size-aware dispatcher uses the size. The simplest is size-splitting: jobs below a cut-off go to machine 0, jobs above it go to machine 1, and the cut-off is set so both machines carry equal load.</p><p>The thesis uses two sets of job sizes. A toy set has just two sizes, 1 and x, with equal load from each. A bounded Pareto set has sizes from 1 up to a largest size z, with a tail like the one in Figure 1.</p><h3>Why a heavy tail hurts</h3><p>For one machine serving jobs in arrival order, queueing theory gives the average wait directly (the Pollaczek–Khinchin formula, used in Chapter 2 of the thesis). Here ρ is the load, X is a job’s size, and E[X²] is the average of the squared sizes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*32i9wqiODgAV6UtmRGbwcA.png" /></figure><p>The squared size is the trouble. A few giant jobs dominate the average of the squares. For this job mix, E[X²] is about the largest job z, while the average size E[X] is only about ln z. Doubling the largest job from 10,000 to 20,000 nearly doubles the wait, yet the average job grows only from 9.2 to 9.9.</p><p>Worse, in arrival order the wait does not depend on the job’s own size. A job of size 1 waits as long as a job of size 1,000, so its slowdown is a thousand times worse.</p><h3>Lever one: sort by size</h3><p>The thesis proves what sorting buys. In the toy mix, size-splitting holds slowdown at ρ / (2(1 − ρ)), which is 2 at 80 percent load however large x gets. Least-work-first grows in proportion to x, and the thesis proves that no size-blind policy can do better than that order. For bounded Pareto sizes the thesis finds these growth rates in the largest job size z:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2hM6l3_tQXWaWlJjDOGiVA.png" /></figure><p>The gap between size-blind and size-aware policies widens like the square root of the largest job.</p><p>I rebuilt the simulation to check. It reproduces the thesis’s Table 3.1 for least-work-first at 80 percent load: 4.36 against 4.4 at x = 10, and 40.9 against 42 at x = 100. Size-splitting matches the exact formulas. Figure 2 shows all three policies as the tail gets heavier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*SXTOnS0zMXVNp7FjBvhDYQ.png" /></figure><p><em>Figure 2. Two machines at 80 percent load. Random and size-splitting are exact; least-work-first is simulated, with 95 percent error bars.</em></p><p>With a largest job of 10,000, random assignment gives a slowdown of about 1,090, least-work-first about 400 (370 to 430 across my runs, since heavy-tailed averages converge slowly), and size-splitting about 22.</p><p>Sorting has limits. You must know a job’s size when it arrives. The loads must balance, or one machine overloads, as the thesis notes. And sorting leaves the tail inside each machine: in my runs, jobs of size 10 or less still averaged a slowdown near 24, stuck behind jobs up to size 100 on their machine. The biggest unit of work in the system is untouched.</p><h3>Lever two: slice into pieces</h3><p>Slicing attacks the biggest unit of work directly. Cut a big job into pieces of at most size c, and the longest a small job can be stuck behind one piece falls from 10,000 to c. The thesis lists this under future work: split large jobs, accepting a split cost that may be small in printing. I tested it with the same two machines.</p><p>Every job above size c is cut into near-equal pieces no larger than c. Every piece after the first carries an extra setup of s, in the same units as job size, where the smallest job is 1. Pieces go to the machine with less work. I tried two release rules.</p><p>• <strong>All pieces at once.</strong> The pieces join the queue when the job arrives.</p><p>• <strong>One piece at a time.</strong> A piece joins the queue only when the previous piece of the same job has finished.</p><p>The first rule does nothing. With c = 10 and s = 0.05, slowdown is about 500, worse than the 370 without slicing, because the queue still holds all of the giant’s work. The second rule changes the picture, because between two pieces of a giant, short jobs get through.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*h2SL6pR-oFVXZBre2E2yhw.png" /></figure><p><em>Figure 3. Release one piece at a time, two machines at 80 percent load, job sizes 1 to 10,000. Stars mark the best slice size for each setup cost.</em></p><p>Two forces set the shape. Bigger slices mean short jobs wait behind longer pieces, and in the figure slowdown climbs roughly in step with c once c passes 5. Smaller slices mean more setups, which add load. For a job mix like this one, the added load is roughly</p><p>where ρ is the original load, s the setup per extra piece, c the slice size and z the largest job. With s = 0.2 and c = 3, the load rises from 80 to 84 percent. As it nears 100 percent the queue blows up, which is why each curve turns upward on the left.</p><p>In a finer sweep, the best slice size was 1.5, 2, 2, 3, 5 and 10 as the setup cost rose from 0.02 to 0.05, 0.1, 0.2, 0.5 and 1. That is roughly ten times the setup cost, but never below the size of the smallest job. At the best slice size, slowdown was between 2 and 8 for setup costs up to half the smallest job, and 14 when the setup cost equalled the smallest job, against 22 for sorting and about 370 with neither.</p><p>Slicing also did not hurt the giants. With s = 0.2 and c = 3, jobs above size 100 had a slowdown of about 2.4 (they finished in about 3.4 times their own size), against about 20 for the same jobs under sorting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*eWKj11NXO1vGnf9ZXqmUiA.png" /></figure><p><em>Figure 4. The lead holds from 50 to 90 percent load. Largest job 10,000.</em></p><p>A cap works better than a proportional cut. When I split every job above the average into a fixed number of equal pieces, slowdown fell only slowly: 288 with 2 pieces, 185 with 4, 67 with 16 and 32 with 64. The pieces stay proportional to the job, so the biggest piece is still 10,000 divided by the piece count, and the slowdown of the jobs above the average rose from 60 to 121 as the piece count went from 2 to 64. Any rule that shrinks big jobs by a percentage behaves this way. What bounds the damage is an absolute cap on piece size.</p><p>One caution about the rules of the game. Slicing with one-at-a-time release lets short jobs overtake a giant between pieces and lets pieces move between machines. The thesis model forbids both, which is why its lower bound on size-blind policies does not apply here. The price is the setup on every extra piece, and the work must be cuttable in the first place: pieces that can run on their own, at low cost. That is the same condition as in bricks and pebbles.</p><h3>Sort or slice, and how this meets bricks and pebbles</h3><p>The two levers have different requirements. Sorting needs the size at arrival and balanced loads, and costs nothing per job. Slicing needs work that can be cut into independent pieces at a modest setup cost, and it works only if pieces are released gradually. Where both hold, slicing did better in my runs, and it gave the biggest jobs a much better deal than sorting did. Where size is known but cutting is expensive, sorting is the safer lever. I did not test the two together, so I make no claim about the combination.</p><p>There is a second reason to slice, and it connects to the bricks-and-pebbles article. In a multi-step line, cutting one job into batches lets the steps overlap, and the best batch size balances setup cost against fill-and-drain gaps. That optimum grows with the square root of job size, so for a giant job it can be large. The queue asks for something different: a cap on piece size, about ten times the setup cost in these runs, so that short jobs are never stuck behind a big piece for long.</p><p>The two answers need not agree. A sensible rule to test is to slice at the smaller of the two. When the cap is far below the pipeline optimum, the giant job gives up some of its own pipeline efficiency, and that is the price of keeping the queue fair for everyone else. Both numbers can be computed before choosing.</p><p>The same logic applies to digital work. A large batch of underwriting requests, a big data load, or a long document review is a giant job. Cut into lots with a controlled release, it lets the many small requests flow through the same reviewers and systems without waiting for the giant to finish.</p><h3>Did optimal batching already hedge the tail?</h3><p>Here is a question I could not leave alone. In the lean document production cells, batch sizes came from the optimal batching rule of the earlier article, and the number of batches in flight was held by WIP limits. That is slicing with controlled release. So I ran the rule itself as the slice size in the queue model: b* = √(N · s / (P − p)), with a setup s of 0.2 per extra batch and the other steps adding half the bottleneck’s work per unit, as in the print-and-bind example.</p><p>The tail hedge was real, and it grew with the tail. Average slowdown, without slicing and then with b* slicing, by largest job size:</p><p>• 100: about 9.5 and 11.9. Slightly worse, because the setups cost more than they save.</p><p>• 300: about 22 and 18.</p><p>• 1,000: about 62 and 29.</p><p>• 3,000: about 158 and 46.</p><p>• 10,000: about 470 and 80.</p><p>Across other setup costs and line balances at 10,000, b* slicing gave 58 to 141 in all but one setting, against 370 to 470 without slicing. The exception was a large setup of 1.0 with a balanced line, where it gave no gain. It does not match a cap chosen for the queue, which reached 2 to 8, because b* grows with the square root of job size and a giant job still gets large batches.</p><p>This fits what I saw in the shops. Most print shops never saw a job like a two-million-card service-bureau run. Their largest jobs were far smaller, with an occasional large one. In that range the batching rule was not a cure for a heavy tail, but it was a hedge against one, working alongside its main purpose of overlapping the steps within a job. The Edelman paper evaluates batching one job at a time. The effect here appears only when jobs share a shop, and to my reading it is an additional benefit that the single-job analysis does not measure.</p><p>Two cautions. This is a simulated queue with the other steps reduced to a single ratio, not shop data, so I call it a mechanism consistent with the shop experience rather than a measured result. And the benefit depended on controlled release: without the WIP limits, cutting a giant into batches that all enter the queue at once did nothing in my runs.</p><h3>What to take away</h3><p>Heavy-tailed work makes a few giants responsible for most of the waiting. Two levers help, and they work differently.</p><p>• Sorting by size is proven in the thesis and reproduced here: at a largest job of 10,000 it cut slowdown from about 400 (least-work-first) and 1,090 (random) to about 22.</p><p>• Slicing with one-piece-at-a-time release did better still in simulation, between 2 and 8 at the best slice size for setup costs up to half the smallest job, because it shrinks the biggest unit of work rather than sorting around it.</p><p>• Cutting a job into pieces that all enter the queue at once does nothing, and cutting into a fixed number of pieces helps slowly. A cap on piece size, released gradually, is what works. Using the optimal batch size of the earlier article as the slice size also helped, by about 2 times at a largest job of 1,000 and about 6 times at 10,000, which suggests batching with WIP control was hedging the tail in the shops.</p><p>• The best slice size is roughly ten times the setup cost, never below the smallest job. That is the bricks-and-pebbles trade-off, now set by queueing instead of by pipeline gaps.</p><p>The limits are real. This is two machines, one processing step, one tail shape (bounded Pareto with index 1, largest job 10,000), random arrivals, and a setup cost that is the same for every slice. Slowdown averages in heavy-tailed systems converge slowly, so I used 4 to 6 runs of 300,000 to 1,000,000 jobs and quote round numbers. Slicing with gradual release resembles time-sharing in the scheduling literature, and I did not compare it with policies that favor the shortest remaining work. I also did not test heavy-tailed arrivals, more machines, several processing steps, or the combination of sorting and slicing. The thesis names several of these as future work. Treat the numbers as a demonstration of the mechanism, not a forecast for any particular shop.</p><h3>Sources</h3><p>• Nham, J. 2008. Size-Independent vs. Size-Dependent Policies in Scheduling Heavy-Tailed Distributions. Master of Engineering thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology. Supervised by J. N. Tsitsiklis and S. Rai.</p><p>• Harchol-Balter, M., Crovella, M., Murta, C. 1999. On choosing a task assignment policy for a distributed server system. <em>Journal of Parallel and Distributed Computing</em> 59, 204–228.</p><p>• Rai, S., Duke, C. B., Lowe, V., Quan-Trotter, C., Scheermesser, T. 2009. LDP Lean Document Production: O.R.-Enhanced Productivity Improvements for the Printing Industry. <em>Interfaces</em> 39(1), 69–90. <a href="https://doi.org/10.1287/inte.1080.0413">doi:10.1287/inte.1080.0413</a>. The batching algorithm is in the appendix, pp. 87–89.</p><p>All simulation results are my own runs of a model written for this article, validated against the thesis’s Table 3.1 and against exact formulas for random assignment and size-splitting.</p><p><em>Disclaimer: The views expressed in this article are my own, based on my personal experience and independent research. They are not affiliated with, endorsed by, or representative of any current or former employer or any organization mentioned.</em></p><p><a href="https://medium.com/tag/operations-research?source=post_page---footer_tags--7acbb90069c2-----------------------------------------">Operations Research</a></p><p><a href="https://medium.com/tag/lean-manufacturing?source=post_page---footer_tags--7acbb90069c2-----------------------------------------">Lean Manufacturing</a></p><p><a href="https://medium.com/tag/scheduling?source=post_page---footer_tags--7acbb90069c2-----------------------------------------">Scheduling</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=55ece6705723" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Bricks, Pebbles, and the Hidden Geometry of Capacity]]></title>
            <link>https://medium.com/@sudhendu.rai/bricks-pebbles-and-the-hidden-geometry-of-capacity-7acbb90069c2?source=rss-9fb5db03b1df------2</link>
            <guid isPermaLink="false">https://medium.com/p/7acbb90069c2</guid>
            <category><![CDATA[scheduling]]></category>
            <category><![CDATA[operations-research]]></category>
            <category><![CDATA[lean-manufacturing]]></category>
            <category><![CDATA[supply-chain]]></category>
            <category><![CDATA[process-optimization]]></category>
            <dc:creator><![CDATA[Sudhendu Rai]]></dc:creator>
            <pubDate>Sun, 25 Oct 2026 22:03:21 GMT</pubDate>
            <atom:updated>2026-10-25T22:03:21.064Z</atom:updated>
            <content:encoded><![CDATA[<p>Oct 25, 2026</p><p>By Sudhendu Rai, Independent Researcher</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_6aH8zx4C840Y5-_PUmUzA.png" /><figcaption>Same box, same material. Loose bricks leave gaps. Small pebbles fill them, but each one needs a little packing around it. <em>Illustration generated with AI.</em></figcaption></figure><h3>The puzzle: busy machines, late jobs</h3><p>A print shop can look fully loaded and still deliver jobs late, because capacity is lost in the gaps between steps, not inside them.</p><p>In our Lean Document Production work at Xerox, most jobs passed through several processing steps before they were packaged and shipped. The natural way to run such a line is job by job: finish all of Step 1, move the whole job to Step 2, and so on. Every machine is busy at some point, yet each downstream step sits idle until the upstream step has completed the entire job. The larger the job, the longer everything downstream waits.</p><p>Our answer was to split larger jobs into smaller batches and let them flow through the steps as a pipeline, so downstream stages start working while upstream stages are still going. That pipelining was a key part of how jobs were processed inside the production cells.</p><p>It raises two questions. How small should the batches be? And is there a way to explain to someone outside operations why this packs more work into the same capacity?</p><p>This is the batch-splitting idea inside the lean document production solution we described in our 2009 <em>Interfaces</em> paper for the Franz Edelman Award competition. The paper gives the batching algorithm in its appendix. This article is about what that algorithm does not spell out: an intuition for why it works, a closed-form answer for the simple case, and a prediction for busy shops. It also reaches beyond print: any work that moves through stages in order, including digital workflows, follows the same arithmetic.</p><h3>Capacity as a box of bricks and pebbles</h3><p>Picture your capacity as a box. One side is the line of processing steps and the other side is time, so the volume is all the machine time you have available.</p><p>Now picture each job as a brick whose size is the processing time it needs. A brick cannot be in every step at once, because the job has to move through the steps in order. Packed as bricks, jobs leave gaps in the box, and those gaps are capacity you paid for and cannot use.</p><p>Break the brick into pebbles and the box packs far better. Small pieces slide into the corners, and downstream steps get work almost as soon as upstream steps start.</p><p>There is a catch. Every pebble has to be wrapped in packing material before it goes in the box. In a shop, that wrapping is the setup and transfer time you pay each time a batch starts at a step. Make the pebbles too small and the packing material takes over the box. Make them too large and you are back to bricks.</p><p>Bricks waste space in gaps. Pebbles waste space in wrapping. The best pebble sits somewhere between, and the next two sections show where.</p><h3>The model: one job on one line</h3><p>The best batch size comes from three quantities: the bottleneck’s setup time, the size of the job, and how much non-bottleneck work the line has to fill and drain.</p><p>Start with the simplest case. The workflow is a linear line of stages, and a single job of N units flows through it alone. Stage i costs a setup and transfer time sᵢ each time a batch starts, plus a processing time pᵢ per unit. With batch size b, one batch takes sᵢ + b·pᵢ at stage i. The bottleneck stage, call it k, is the one where a batch takes longest.</p><p>The first batch has to travel the whole line. After that, batches leave at the bottleneck’s pace. Adding it up, with S as the sum of all setups and P as the sum of all per-unit times:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vx2sp5CCmyjWkXb5sLXz-w.png" /></figure><p>The first two terms do not depend on b. They are the ideal: the whole job running at the bottleneck’s speed. The last two terms are the cost of imperfect packing, and they pull in opposite directions. The term b·(P − pₖ) grows with batch size, because bigger batches take longer to fill and drain the line. The term N·sₖ / b shrinks with batch size, because fewer batches mean fewer setups at the bottleneck.</p><p>Setting the slope to zero gives the best batch size:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/451/1*fTONdt4GDLsK0aGw2ZUheA.png" /></figure><p>A larger job or a costlier bottleneck setup favors bigger batches. More work in the stages around the bottleneck favors smaller ones. It is the same square-root structure as the classic economic order quantity, with a different cost on each side.</p><h3>From formula to shop floor: the batching algorithm in the Edelman paper</h3><p>The model above is the same one that sits in the appendix of the Edelman paper, where it becomes a practical algorithm for choosing batch sizes.</p><p>The paper describes a job that passes through n operations in series, each with a processing time per item and a setup time each time a new batch starts. The total time is the time for the first batch to cross the line, plus the remaining batches released at the pace of the slowest stage’s batch time. The paper writes that slowest stage as a maximum over all stages, so it does not have to be named in advance.</p><p>The paper finds the best batch size by enumeration. List the batch sizes that divide the job evenly, compute the total time for each, and take the smallest. That is exact for a serial line, and it keeps working when the bottleneck moves from one stage to another as the batch size changes. The closed form earlier in this article applies when one stage stays the bottleneck across the range of batch sizes. In that case it gives the answer without a search, and it shows what drives the answer.</p><p>For assembly operations, where several flows merge, the paper traces the critical path through the network. That path can change with batch size, because two stations can swap places as the slowest one, so the total time is re-evaluated whenever the critical path changes.</p><p>The paper’s own example makes the formula concrete. A tape-bound book needs printing, at 1 minute per book, and then binding, at 0.5 minutes per book. Unloading a printed batch and restarting the printer costs 2 minutes, and loading a batch into the binder costs 1.5 minutes. Take a job of 100 books, a number I chose for illustration. Printing is the bottleneck at every batch size, so the closed form applies, and the best batch is √(100 × 2 / 0.5) = 20 books.</p><p>• One batch of 100 (the whole job): 153.5 minutes</p><p>• Two batches of 50: 130.5 minutes</p><p>• Four batches of 25: 122.0 minutes</p><p>• Five batches of 20: 121.5 minutes</p><p>• Ten batches of 10: 126.5 minutes</p><p>• Twenty batches of 5: 144.0 minutes</p><p>The whole job as one brick takes 153.5 minutes. Batches of 20 finish in 121.5 minutes, about 21 percent sooner, and enumerating the divisors of 100 picks the same size as the formula. Batches of 5 are already 22.5 minutes slower than batches of 20, because the setups have started to crowd the box.</p><p>Figure 1 shows the whole curve, with two other setup times at the bottleneck for comparison. A larger setup moves the best batch size to the right, and every curve has a flat bottom. For the book example, any batch size from 14 to 28 books finishes within 1.5 minutes of the best.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*uq2EXN8xi4ZhuP-evuSvjw.png" /></figure><p><em>Figure 1. Total time to finish a 100-book job as batch size changes, for three setup times at the bottleneck. Stars mark the best batch size from the formula. Dots mark the batch sizes that divide the job evenly in the book example.</em></p><p>What the paper does not spell out is why this works and how the answer should change with load. The next sections take those up.</p><h3>The best pebble: wasted space equals packing material</h3><p>At the best batch size, the space lost to gaps exactly equals the space lost to packing material.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*xC2eZnWVu4oaqiA1SU1kdA.png" /></figure><p><em>Figure 2. Bricks and pebbles in space and time. Stages run down, time runs across, and the dashed box is the capacity available.</em></p><p>In that picture, stages run down one side and time runs across. The gaps are the idle triangles at the start and end of the run, where downstream stages wait for work or finish early. Their area is roughly b·(P − pₖ), so they grow as the batches grow. The packing material is the setup shell around each batch, with total area roughly (N / b)·sₖ, so it shrinks as the batches grow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*XgHkb_DEm2kMm77xoPsjqg.png" /></figure><p>At the optimum, each of the two areas equals the square root of N·sₖ·(P − pₖ), so the total overhead is exactly twice that. This gives the lay version in one sentence: the right pebble size is where the packing material you add equals the gap you remove.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*qtJzI_02hHgTQwDMmNz41Q.png" /></figure><p><em>Figure 3. For the book example, the gaps grow with batch size, the setup shell shrinks, and their sum is smallest where the two are equal.</em></p><p>The optimum is also forgiving. Because the two terms trade off smoothly, a batch one and a half times the best size, or two thirds of it, raises the overhead by about 8 percent, and a batch twice or half the best size raises it by about 25 percent. You need the right neighborhood, not the exact number.</p><h3>A loaded shop changes the answer</h3><p>In a busy shop the best pebble is larger than the single-job formula says, because setups are paid out of capacity that every other job is also waiting for.</p><p>The single-job model has no queues. A real shop has many jobs competing for the same stages, and setups now do two kinds of damage. Each unit of work effectively costs p + s / b of machine time, so the utilization of a stage rises to roughly λ·(p + s / b), where λ is the rate at which units arrive. Smaller batches push utilization up. Queueing delay climbs steeply as utilization approaches 100 percent, roughly in proportion to 1 / (1 − utilization), so a small increase in setup load near saturation costs far more waiting than the same increase in a quiet shop.</p><p>That suggests a prediction: the best batch size should rise with shop utilization. In an empty shop the shell is cheap and small pebbles win. In a loaded shop the shell is expensive and larger pebbles win.</p><p>There is a second effect the single-job picture cannot show. In a loaded shop, pebbles from one job fill the gaps left by other jobs. That is where much of the capacity gain comes from, and it is a different mechanism from the faster finish of a single job.</p><p>The prediction is easy to test. Run a simulation or use shop data with batch size varied at several utilization levels, find the batch size that minimizes cycle time at each level, and see whether the minimum moves to the right as the load goes up.</p><h3>Beyond the print shop: orchestrating digital work</h3><p>The same geometry applies wherever work moves through steps in order, including digital processes that orchestrate several steps.</p><p>Take a batch of underwriting requests that arrives each morning and must pass through steps such as intake, data enrichment, risk assessment, pricing and review. Working the whole batch one step at a time is the brick: risk assessment sits idle until intake has cleared every request, and reviewers wait until all the pricing is done. Cutting the batch into meaningful lots, say by product or complexity, and letting each lot move on as soon as it is ready is the pebble. The lots can then be distributed across the available people or automated workers, so every step has something to work on early.</p><p>The packing material is there too. Forming a lot, routing it and picking it up at each step takes effort. Lots that are too small drown the process in handoffs, and lots that are too large bring back the idle gaps and delay the first decisions. As in the shop, the best lot size should rise when the operation is busy, because handoffs use the same capacity as the work.</p><p>In the print shop, a controller split jobs into lots and sent them to cells. In a digital process, the orchestration layer plays that part. Claims intake, loan applications, invoice processing and reconciliations all have the same shape.</p><h3>What to take away</h3><p>Split large jobs into batches, but not into the smallest pieces you can make, because both extremes waste capacity. The same holds for any multi-stage work, from underwriting queues to other digital workflows.</p><p>• Batching works by removing the idle gaps between steps. Setup and transfer time is the price of removing them.</p><p>• The best batch size depends on the bottleneck’s setup time, the size of the job, and the processing time in the stages around the bottleneck.</p><p>• The best size is forgiving. Landing within a factor of two is usually good enough.</p><p>• Expect larger batches to be better as the shop gets busier.</p><p>The model has limits. The closed form assumes a linear line, a bottleneck that stays the same stage as batch size changes, and a processing batch that equals the transfer batch. Enumeration relaxes the bottleneck assumption for a serial line, and the critical-path version in the paper extends it to assembly operations. The interaction of many jobs is not captured by either. The load effect in particular is a prediction to confirm, not a result.</p><p>The next step is to test it: simulate a shop with several jobs, vary the batch size at several utilization levels, and compare the best sizes with the formula.</p><h3>Sources</h3><p>• Rai, S., Duke, C. B., Lowe, V., Quan-Trotter, C., Scheermesser, T. 2009. LDP Lean Document Production: O.R.-Enhanced Productivity Improvements for the Printing Industry. <em>Interfaces</em> 39(1), 69–90. <a href="https://doi.org/10.1287/inte.1080.0413">doi:10.1287/inte.1080.0413</a>. The batching algorithm is in the appendix, pp. 87–89.</p><p>• Rai, S., Maitra, S. C. Method for determining optimal batch sizes for processing print jobs in a printing environment. US patent 6,805,502, issued 2004.</p><p>• Lin, J., Jacobs, T. W., Rai, S. System and method for determining an optimal batch size for a print job. US patent publication 20070092323, 2007.</p><p><em>Disclaimer: The views expressed in this article are my own, based on my personal experience and independent research. They are not affiliated with, endorsed by, or representative of any current or former employer or any organization mentioned.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7acbb90069c2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When Automation Makes Things Worse: The Small Buffer That Fixes It]]></title>
            <link>https://medium.com/@sudhendu.rai/when-automation-makes-things-worse-the-small-buffer-that-fixes-it-d893237fdc83?source=rss-9fb5db03b1df------2</link>
            <guid isPermaLink="false">https://medium.com/p/d893237fdc83</guid>
            <category><![CDATA[operations-management]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[process-improvement]]></category>
            <category><![CDATA[lean]]></category>
            <category><![CDATA[manufacturing]]></category>
            <dc:creator><![CDATA[Sudhendu Rai]]></dc:creator>
            <pubDate>Sat, 24 Oct 2026 15:44:08 GMT</pubDate>
            <atom:updated>2026-10-24T15:44:08.269Z</atom:updated>
            <content:encoded><![CDATA[<p>Oct 24, 2026</p><p>By Sudhendu Rai, Independent Researcher</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k4WPZ-35_Xc0WeAc70AT_A.png" /><figcaption>Coupled versus decoupled: with a small buffer between them, one machine can keep working while the other is stopped.</figcaption></figure><p>Wire two failure-prone machines directly together and you will often get a worse system than if you had left them apart. Put a small buffer between them and productivity jumps. I first saw this in 1999, while studying how production operations really run, and then watched it repeat twice more.</p><p>The three cases came from a document production shop, a credit card manufacturer and a book-publishing startup. The equipment, vendors and businesses were all different. The mechanism, the fix and the result were the same each time.</p><p>This is the story of those three cases, why the fix works, and the question I now ask whenever someone shows me an impressive inline automation line.</p><h3>Case 1: The best-run shop in the country</h3><p>In 1999 I was a researcher at Xerox in Webster, New York. A Xerox Document Technology Center sat very close to my lab, and it was considered one of the best-run print shops of its kind in the country. I got permission to spend time there. I wanted to understand how our equipment was used in real customer environments, and I was teaching myself manufacturing systems engineering and looking for places to apply it.</p><p>Over a few months I got to know the shop manager. One day he showed me something he had built: a printer coupled directly to an inline shrink wrapper. Documents were printed, fed into the wrapper, shrink-wrapped automatically, and dropped into a bin as finished, ship-ready packages. On the face of it, it looked extremely efficient.</p><p>He was very unhappy with it. He had invested considerable money and time, working with the finishing company and with the Xerox printer group, and the line delivered only about 30 percent productivity. Faults and jams kept interrupting it. Everyone involved had worked hard on the problems, but even after several rounds of fixes the line was not reaching the productivity he had hoped for.</p><p>At the time I was reading a book on manufacturing systems engineering by Dr. Stanley Gershwin of MIT, with whom I had partnered. One of its central themes is the effect of buffers between machines in a line, and how to size them. Looking at the shop’s line through that lens, the problem was clear. The failure modes of the printer and the wrapper were reinforcing each other. If the printer went down, the wrapper sat idle. If the wrapper went down, the printer sat idle. The uptime of the whole line was far lower than either machine’s alone.</p><p>I suggested that he decouple the two machines and put a small buffer between them. If the wrapper went down, the printer could keep printing and fill the buffer. If the printer went down, the wrapper could keep pulling from the buffer and keep working.</p><p>He was reluctant, and I understood why. He had invested heavily in the coupled design, and the teams involved were still working to resolve the issues within it. It took several conversations, and an explanation of how buffers work, before he agreed.</p><p>The moment he decoupled the machines, his productivity shot up. A small amount of buffer between two machines that fail at random, and take a random time to repair, beat the tightly coupled line. It was my first lesson in what I would come to call intelligent automation, as opposed to blind automation that simply couples things together.</p><h3>Case 2: Three machines, three vendors, one line</h3><p>In 2003 I was working with a large credit card manufacturer. They had connected a Xerox printer, a card embosser and an inserter into a single inline system. The printer produced the carrier document, the embosser produced the card, and the inserter put them together in an envelope, ready to mail.</p><p>They had the same problem. If anything went down, the whole line went down. Productivity was lower than planned and the business case for the automation was under pressure. It was also expensive: automating three complex machines to work as one required special-purpose engineering. Integrating machines from three different companies into one seamless system is a hard engineering problem.</p><p>I recognized the pattern immediately. I suggested they decouple the machines and put small buffers between them, and I told them about what I had seen at the Webster shop. They took a while to come around, but when they did it, their productivity went up.</p><h3>Case 3: A publishing startup in Florida</h3><p>The third case was a book-publishing startup in Florida, run by a group of young founders. Xerox had sold them inline book production units: paper went in one end, finished books came out the other, with the cutter, the binder and everything else in a single line. They had bought several units against a specific business case.</p><p>The founders were not getting the productivity they had expected, and the business case was under pressure.</p><p>I had gone there to implement lean document production, not to look at the inline units. But when I saw them, I made the same comment I had made twice before: decouple them and your productivity will go up. I showed them how adding a small buffer increases productivity dramatically.</p><p>They acknowledged the insight and said they would take steps to address it in the future across all of their systems.</p><h3>Why it works</h3><p>An inline line has no slack. When two machines are coupled directly, the line runs only while both machines run. A stoppage on either one starves or blocks the other, so the failures of the two machines add up instead of averaging out.</p><p>Here is an illustrative example, not data from any of these shops. Suppose each machine is available 85 percent of the time. Coupled with no buffer, the line runs only when both are up, and its output lands in the low 70s as a percentage of nominal capacity. The best the pair could ever do, with unlimited buffering and perfect decoupling, is the 85 percent of the slower machine. Most of that gap of roughly 11 to 13 points is lost to nothing more than the two machines’ random failures lining up badly.</p><p>A buffer breaks that dependence. While one machine is down, the other keeps working against the buffer, drawing it down or filling it up, and the line is only affected if the outage outlasts the buffer. Because breakdowns are random and repairs take a random amount of time, the buffer absorbs most of the short outages and leaves only the long ones.</p><p>This is the territory of Stanley Gershwin’s <em>Manufacturing Systems Engineering</em>, which analyzes exactly this kind of line: unreliable machines separated by finite buffers, and how throughput depends on buffer size. The mathematics is rigorous, but the practical message is simple. The right buffer size depends on how long the machines take to repair, and a little buffer buys a lot.</p><h3>The curve that matters</h3><p>When our team presented Lean Document Production® at the 2008 INFORMS Franz Edelman competition (Rai et al., 2008), I made this argument in a single slide. It shows a printer feeding a finisher with a buffer between them, and below it a curve of system efficiency against buffer size.</p><p>The shape is the whole message. At zero buffer, the fully coupled case, efficiency is low. It climbs steeply with the first small amount of buffer, then flattens almost immediately. Most of the benefit comes from the first few units. Everything beyond that is extra floor space and extra work in process.</p><p>That plateau is also the answer to the most common objection. People hear “decouple” and picture a large staging area, more handling and more inventory. The curve says otherwise: you need only a small buffer, sized to the repair behavior of the machines on either side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TXI6SlPhUJ6f4feLXl717g.png" /></figure><p><em>Efficiency versus buffer size for two failure-prone machines. Illustrative shape only, based on the argument I presented at the 2008 INFORMS Franz Edelman competition.</em></p><h3>Questions to ask before you buy an inline line</h3><p>After these three cases, I started raising the issue every time a print shop owner described an inline system they were considering. These are the questions I would put to any inline automation proposal:</p><p>1. <strong>How often does each machine stop, and for how long?</strong> Frequency and repair time both matter, and the repair time sets how much buffer you need.</p><p>2. <strong>What happens to the other machines when one stops?</strong> In a fully coupled line, the answer is that they all stop.</p><p>3. <strong>Who owns the failures of the combined system?</strong> With equipment from several vendors, someone has to be accountable for the performance of the line as a whole, not just each machine.</p><p>4. <strong>Can a small buffer go between the machines?</strong> Even a modest one changes the economics. Ask what it would cost before accepting that the line must be seamless.</p><p>5. <strong>What does the business case look like at the line’s real uptime, not the nameplate?</strong> If it only works at 100 percent coupled availability, it will not work.</p><h3>When inline is still the right call</h3><p>Inline coupling is not always a mistake. Machines that are very reliable and quick to repair lose little to coupling. Some products cannot be buffered at all, such as items that are fragile, hot, or tightly controlled. In those cases the answer is to make the coupled machines more reliable, not to add a buffer.</p><p>But in the shops I worked with, the inline line looked like the smart, efficient choice, and it cost the owners real money. Print is a highly competitive business with thin margins that keep getting squeezed, and the shops could not afford to be wrong. I was able to help several of them avoid that mistake before they made it.</p><p>The broader lesson is about what automation is for. Joining machines together is easy to do and easy to admire. Making the whole system produce is harder, and it starts with understanding how the pieces fail. Automation that respects the randomness of real machines is more intelligent than automation that ignores it.</p><h3>References</h3><p>Gershwin, S. B. (1994). <em>Manufacturing systems engineering</em>.</p><p>Rai, S., Duke, C. B., Quan-Trotter, C., Lowe, V., &amp; Scheermesser, T. (2008). <em>LDP Lean Document Production®: Dramatic productivity improvements for the printing industry</em> [Presentation]. INFORMS Franz Edelman Award competition, Baltimore, MD. Presenters: S. Rai and W. A. Steenburgh.</p><p><em>Disclaimer: The views expressed in this article are my own, based on my personal experience and independent research. They are not affiliated with, endorsed by, or representative of any current or former employer or any organization mentioned. Customer examples are described in general terms.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d893237fdc83" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[We Were Doing Process Mining in 2002 Without Calling It That]]></title>
            <link>https://medium.com/@sudhendu.rai/we-were-doing-process-mining-in-2002-without-calling-it-that-144e22ff7607?source=rss-9fb5db03b1df------2</link>
            <guid isPermaLink="false">https://medium.com/p/144e22ff7607</guid>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[process-mining]]></category>
            <category><![CDATA[process-improvement]]></category>
            <category><![CDATA[operations-management]]></category>
            <category><![CDATA[innovation]]></category>
            <dc:creator><![CDATA[Sudhendu Rai]]></dc:creator>
            <pubDate>Fri, 23 Oct 2026 12:12:25 GMT</pubDate>
            <atom:updated>2026-10-23T12:12:25.309Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How barcode scanners, Palm handhelds, and a forward-looking print shop manager led to event log analysis years before the term “process mining” reached the shop floor.</em></p><p>By Sudhendu Rai, Independent Researcher</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/836/1*k8LC5lXbZblLlqybKwbBxw@2x.jpeg" /></figure><p><em>Scan by scan, a print shop floor became a stream of event data.</em></p><p>In 2002, I walked into a print shop in Dallas, Texas. Xerox ran it, and it was called the Document Technology Center. I had already helped several other centers around the country improve their productivity, and the services management team asked me to see whether I could do the same here.</p><p>I started the way I always did: collecting data. I pulled job tickets, analyzed them, and built simulation models. Then I found something unusual. The shop manager was very computer savvy. He had been keeping his own records in Access databases and had even built a software solution on top of them. He tracked when jobs came in, when they were completed, and a few steps in between. I was impressed.</p><p><strong>First insight: capture more timestamps</strong></p><p>Looking at his data, I realized a few more data points would give us a much better view of the shop. I suggested he record a timestamp at every meaningful point in the workflow: when a job arrives, when it starts on a machine, when it completes, and at each step as it moves through the shop.</p><p>He was happy to try. We started manually, with operators writing start and finish times on paper, and the numbers were then keyed into his Access database. Even that crude process made the data far richer, and it let both of us do much more analysis.</p><p><strong>Second insight: stop writing it down</strong></p><p>While that was running, I wondered whether we could automate the collection itself. Palm OS handhelds with wireless barcode scanners were reaching the market. You could scan a barcode and the data went straight to a server. A Seattle company, Wavelink, made the software that supported them.</p><p>I went back to my research team, asked to buy a few scanners, and wrote the data collection software myself using Wavelink. Then we took it to the shop and made one more change. We assigned barcodes not just to jobs but to machines and operators as well. Operators scanned themselves, the machine, and the job ticket. Every scan was an event with a who, a what, and a when.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/477/1*pFmL2QxLHv0FUJZAOfg73g@2x.jpeg" /></figure><p><em>The handheld data-capture screen used in the shop. Each scan recorded the job, operator, machine, event, and timestamp.</em></p><p><strong>What the data made possible</strong></p><p>After a few weeks, we had a rich event stream from the floor. We combined it with the information already in the Access database and built a detailed picture of operations. From those timestamps I built analytics for:</p><p>•	Utilization</p><p>•	Cycle time</p><p>•	Work in process (WIP)</p><p>•	Process cycle efficiency</p><p>The same data also fed simulation models of the shop. The analytics and the models came from a single source of truth: the real behavior of the shop, recorded as it happened.</p><p>This work became part of the Lean Document Production (LDP) Toolkit, which was deployed across hundreds of Xerox shops. It was also the basis of our entry in the 2008 INFORMS Franz Edelman competition, later published in Interfaces. A video of the system in use, shown at the competition, is here: <a href="https://youtu.be/AQXJ2Rx-q3I?is=3A6uTOMwlbTRkFs2">watch on YouTube</a>.</p><p><strong>What we had actually built</strong></p><p>We called it event log analysis. That was simply the name for it then. Only later did I realize we had independently built what the field now calls process mining: reconstructing and analyzing operations from timestamped event logs of cases, activities, resources, and times.</p><p>There was one difference in emphasis. In a print shop the workflow is largely understood in advance, so we weren’t using the logs to discover the process. We used them to measure operational efficiency and to drive simulation. Process mining today often emphasizes discovery. Our use was closer to performance analytics combined with a simulation-based digital twin, and I’d argue that is where much of the practical value lies.</p><p>The name stuck with us, too. Years later, I was still publishing under it. There are patents covering this work, along with several publications on using event data in new ways, listed below.</p><p><strong>The lesson</strong></p><p>Innovation rarely arrives with a label. We didn’t set out to invent a field. A curious shop manager had good data, and I had a modeling mindset and a hunch that more granular timestamps would reveal things. We kept asking two questions: what could we capture next, and how could we capture it with less human effort?</p><p>If you are sitting on operational data and wondering whether it’s good enough, start by asking what event is missing and what it would cost to record it. The method you need may already exist under a name you haven’t learned yet. Learn the name when you can, but don’t wait for it to start.</p><p><strong>References</strong></p><p>•	Rai, S., Duke, C. B., Lowe, V., Quan-Trotter, C., &amp; Scheermesser, T. (2009). LDP lean document production: OR-enhanced productivity improvements for the printing industry. Interfaces, 39(1), 69–90.</p><p>•	Rai, S., &amp; Daniels, M. (2015). An event-log analysis and simulation-based approach for quantifying sustainability metrics in production facilities. 2015 Winter Simulation Conference (WSC), pp. 1033–1043. IEEE.</p><p>•	Rai, S., Hu, B., &amp; Ettam, R. K. (2015). Modeling customer demand in print service environments using bootstrapping. 2015 Winter Simulation Conference (WSC), pp. 598–608. IEEE.</p><p>•	Rai, S., &amp; Viassolo, D. E. (2003). A lean document production controller for printshop management. 42nd IEEE Conference on Decision and Control.</p><p>•	Rai, S. (2018). Process Wind Tunnel for improving insurance business processes. INFORMS Business Analytics Conference, Baltimore, MD.</p><p>•	Rai, S. (2018). Process Wind Tunnel for improving business processes. 2018 International Conference on Process Mining.</p><p><strong>Disclaimer</strong>: <em>This article reflects my personal recollections and views as an independent researcher. It is not affiliated with, endorsed by, or representative of my current employer or any organization I have worked for. References to past work are for historical and educational purposes only.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=144e22ff7607" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>