What Changed When the Outcome Didn’t?
What stable results can and can’t tell us after organizational change
A team gets smaller, but the work appears to keep moving at roughly the same pace. Six months later, the volume is about the same, urgent requests are still being handled on time at about the same rate, and the employees using the service are just as satisfied as they were before.
I can understand why we’d look at that and say the reduction worked, and after looking into it more closely, I might come to exactly the same conclusion. What I don’t know from those numbers alone is whether the team is producing the same result under roughly the same conditions, or whether something about the work changed after the staffing reduction.
That’s the part I keep coming back to, because “performance stayed stable” and “the team successfully absorbed the reduction” sound very similar, but they aren’t quite the same statement. The first describes what happened to the outcome, while the second starts explaining why it happened, and I think there’s more room between those two things than we usually acknowledge. That distinction becomes especially important when the explanation starts influencing what the organization decides to do next.
The team, people, and figures in this article are illustrative composites, not findings from a single organization.
So, What Else Changed?
For a team that went from 100 employees to 85, I’d start by looking at what was happening before the reduction and what changed alongside it. Had automation already reduced the manual workload, was demand declining, or had the organization redesigned its processes before deciding it needed fewer people? The sequence matters because the staffing reduction may have followed a genuine improvement in efficiency rather than created one, and I’d approach the analysis differently depending on what actually happened.
The difficulty is that several changes can happen together, which is perfectly reasonable when you’re running a business and considerably less convenient when you’re trying to isolate the impact of a particular decision. A team gets reorganized, new technology is introduced, responsibilities move around, service expectations change, and a vendor’s scope expands, sometimes within the same quarter. Six months later, Analytics gets asked which decision produced the result, even though the organization made several decisions that could’ve contributed to it. There may be a way to separate their effects, but I’d want to understand how the changes were implemented before deciding that a simple before-and-after comparison could answer the question.
Even identifying when the staffing reduction began can be more complicated than I’d expect from something with an effective date in Workday. The reorganization might officially take effect on April 1, but employees could’ve learned about it in January, managers may have stopped backfilling positions in February, and responsibilities might’ve started shifting in March because everyone already knew what was coming. By the time April arrives, the team may have spent several months adjusting to a change that our data still treat as a future event. An effective date is useful for reporting, but it doesn’t necessarily tell me when employees started responding to a decision or when the operation began functioning differently.
I’d also want to look at the composition of the work, because two months can each contain 10,000 employee requests while requiring very different amounts of time, expertise, and attention. A new self-service tool might be resolving straightforward questions that previously needed a person, while a vendor has taken over one of the more complicated case types. The internal team could still be receiving roughly the same number of requests, but those requests might now involve different approval steps, require more judgment, or come from parts of the business with different service expectations. Even the definition of an on-time response could’ve changed if the organization adjusted when its service clock starts or stops.
I wouldn’t consider any of those possibilities evidence that the staffing reduction failed. In fact, if automation or process redesign eliminated enough manual work to make the smaller team sustainable, I’d want the analysis to recognize that. But it would change how I’d describe the result, particularly if someone wanted to use it as evidence that the employees who remained had successfully taken on everything their former colleagues used to do.
There’s also a difference between an unchanged overall result and unchanged performance across every part of the service. If roughly 94 percent of urgent requests were handled within the promised response window before the reduction and roughly 94 percent still are, I’d be comfortable describing that aggregate as stable. But straightforward urgent cases could be closing faster while complicated ones take longer, or one business unit could’ve improved enough to offset deterioration elsewhere. I’d want to examine the underlying distribution before deciding what the overall percentage meant, especially if the organization was using it to evaluate whether the new operating model was working.
The possibility of genuine efficiency is important here because I don’t want to approach a favorable outcome as though there’s automatically something suspicious about it. If the work became easier, the process improved, and the team no longer needed the same staffing level, the straightforward explanation may be correct. But if the work remains reasonably comparable and the service results still look stable, I’d become much more interested in how the team is producing those results and whether anything changed in the parts of the operation that aren’t represented by the headline measures.
The Work Behind the Result
One of the first things I’d separate is incoming demand from completed work, because those numbers can behave very differently while a prominent service measure stays relatively unchanged. In our composite team, lower-priority requests might’ve been arriving at around 500 per month before the staffing reduction, with roughly 480 being completed during the same period. That queue was already accumulating about 20 additional requests each month, but if monthly completions later fell to around 390 while incoming demand remained steady, the accumulation would accelerate to approximately 110 requests per month.
If the team really was still completing roughly the same total number of requests each month, I’d also want to understand where those 90 fewer lower-priority completions were being offset. Perhaps the team was resolving more straightforward requests, or perhaps it had deliberately shifted capacity toward urgent work. Either explanation could be reasonable, but the lower-priority backlog would still be growing much faster than before, and the overall completion total wouldn’t tell me whether that was intentional.
I’d be particularly interested in what those lower-priority requests actually involved, because the label doesn’t necessarily tell me much about their consequences. Some might be routine administrative questions that can comfortably wait another week, while others might involve an employee who needs several things resolved before they can begin a new role or a manager waiting on information to complete a time-sensitive process. If leadership and the team knowingly decided to prioritize urgent requests while accepting longer waits elsewhere, the data may be confirming an intentional tradeoff. If nobody realized how quickly unfinished work was accumulating, we’d have a different conversation.
Cases per employee would probably appear somewhere in that conversation too, and this is where an impressive-looking productivity figure deserves a little scrutiny. If a team goes from 100 full-time equivalents to 85 while completing exactly the same number of cases, completed cases per FTE increase by approximately 17.6 percent through the change in headcount alone. I could calculate that before examining a single employee’s workload, which is why I wouldn’t use the resulting percentage as evidence that people were putting in 17.6 percent more effort.
That doesn’t make cases per FTE a bad operating measure. I’d still want to know whether output increased, declined, or remained stable relative to staffing, but I’d also want to understand whether the cases were comparable and how much manual effort they required. A faster process, better automation, improved employee proficiency, and a heavier individual workload could all contribute to higher output per employee, but they wouldn’t mean the same thing for the organization or the people working in it.
I’d also be curious about where complicated work was going, particularly in a team where escalations and specialist assignments are recorded consistently enough to examine over time. If the same three senior employees handled around 28 percent of difficult escalations before the reduction and six months later were handling half of them, I’d want to understand whether that concentration was intentional. The organization might’ve decided to route specialized work to its most experienced people because doing so improved quality, or those three employees may have gradually become the default contacts for anything the standard process couldn’t resolve.
Get Whitney Hosea’s stories in your inbox
Join Medium for free to get updates from this writer.
There are reasonable arguments for specialization, especially when certain cases genuinely require expertise that isn’t distributed evenly across a team. What I’d be interested in is whether the operating model accounted for that concentration, whether other employees were still developing the necessary expertise, and what would happen if one of those senior employees became unavailable. Assignment records could help establish whether the distribution changed, but I’d probably need more context to understand why it changed and whether the arrangement was sustainable.
The harder part is that some of the work I’d be interested in may never have been recorded consistently enough to reconstruct. An employee who spends twenty minutes helping three coworkers understand a confusing process may prevent several requests from entering the queue, but the case-management system isn’t necessarily recording that conversation. A process guide may still be available even though nobody has meaningfully updated it in months, and mentoring can gradually disappear because the person responsible for it keeps getting pulled into urgent work. Nobody has to make a formal decision to discontinue those activities for them to become less frequent.
There is no abandoned_work = TRUE field waiting politely in Snowflake, which would admittedly make that part of the investigation considerably easier. What I’d have instead might be document histories, project records, training materials, and whatever people can tell me about how their responsibilities changed. Some of those records may offer useful evidence, while others may be too incomplete to support much more than a question I’d want to investigate further.
This is where qualitative evidence becomes particularly useful, because I’d be interested in what people describe when I ask them to walk through how the work gets done now compared with how it used to get done. I wouldn’t treat their accounts as automatically more accurate than the operational records, since people’s experiences differ and their recollections can be imperfect, but I wouldn’t dismiss them simply because they aren’t already structured into fields either. If several employees independently describe complicated requests increasingly going to the same two people, I could examine whatever assignment history exists to see whether that pattern appears there too.
I’d be equally interested in cases where the evidence challenges the interpretation I was starting to form. The operational data might suggest that the remaining employees are handling more work, while interviews reveal that a new process eliminated several unnecessary steps and made the cases substantially easier to resolve. If I also found fewer escalations, less rework, and a healthier backlog, I’d have good reasons to reconsider whether the staffing reduction created the pressure I initially thought it might have. I’d rather discover that my first explanation was incomplete than spend the entire analysis finding increasingly sophisticated ways to defend it.
When the Work Doesn’t Stay Inside the Team
The investigation could also move beyond the systems People Analytics normally owns, because the work may have changed across departments without appearing as a formal transfer anywhere. Managers might now be answering questions employees previously directed to the support team, another HR function may have picked up certain tasks, or employees may be resolving more issues through a vendor or self-service platform. The original team’s performance could be accurately reported while representing a smaller portion of the overall service than it did before.
I wouldn’t automatically interpret higher activity in another department as evidence that work was transferred there, though. That function may be responding to its own changes in demand, and the vendor’s higher volume could be the result of a broader contract rather than the staffing reduction we’re studying. I’d need some combination of process documentation, routing history, vendor records, and conversations across the affected teams to understand whether responsibilities actually shifted. Even then, an informal change that nobody documented at the time might be difficult to quantify six months later.
That uncertainty is particularly interesting to me because employees may have changed the process before anyone formally updated the operating model. Someone may have started answering questions directly rather than routing them through the usual queue, another person may have created a workaround that eliminated a recurring approval delay, or the team may have redistributed responsibilities as vacancies remained unfilled. By the time I measure the result, I’m looking at an operation that has already had months to respond to whatever changed, and those responses may be part of the reason performance looks the way it does.
One possible statistical structure for that situation is inconsistent mediation, where a direct effect and an indirect effect operate in opposite directions. In this case, a staffing reduction could put downward pressure on performance while changes in how employees organize and perform their work offset some of that pressure, leaving the overall result relatively stable. It’s a useful concept for thinking through one possible explanation, although a flat performance measure wouldn’t establish that the mechanism actually occurred. Demonstrating it would require appropriately timed measurements, a defensible measure of the proposed employee response, a credible causal design, and serious consideration of the other changes happening inside the organization.
The relationships could become more complicated once employees start affecting one another’s work, too. If one senior employee begins taking most of the difficult escalations, the remaining team members may receive a different mix of cases, junior employees may get fewer opportunities to develop that expertise, and managers may spend less time troubleshooting. Those changes can affect individual and team-level outcomes simultaneously, which makes it difficult to treat each employee’s response as an independent event.
I probably wouldn’t lead with inconsistent mediation in a leadership meeting unless the analysis genuinely called for it, but the concept is useful for thinking about one way a stable result could emerge from changing organizational conditions. I’d still want the statistical reasoning to serve the operational question, particularly if the people doing the work could help me identify a simpler explanation that the available data hadn’t captured.
Jane, one of the senior employees in our composite team, offers another way to think about the timing. If she leaves several months after the staffing reduction and complicated requests subsequently start taking longer, her departure may have caused the immediate deterioration, but I’d still be interested in whether the team’s reliance on her had been increasing beforehand. Assignment histories, if available, might show that she gradually took on a larger share of difficult cases, while interviews or process records could reveal whether other employees had increasingly relied on her knowledge of poorly documented exceptions.
Of course, I could look into Jane’s history and find that nothing particularly unusual happened before she resigned. Her responsibilities may have been entirely consistent with her role, the team may have functioned normally whenever she was unavailable, and she may simply have been an excellent employee whose departure created a temporary disruption. I’d consider that a perfectly credible explanation, and I wouldn’t want to manufacture a story about organizational dependency just because it made the timing more interesting.
What I’d find useful about examining the period before Jane left is the opportunity to distinguish between a sudden loss of expertise and a dependency that developed over time, provided the necessary evidence exists. The departure itself would tell me when performance began to deteriorate, but I’d need to understand the earlier distribution of work before deciding whether the organizational conditions that contributed to that deterioration had been developing for months.
The Explanation Someone Will Use Next
After working through the different possibilities, I could end up agreeing almost completely with the original interpretation of the staffing reduction. If automation had removed unnecessary work, the remaining cases were reasonably comparable, service quality had improved, unfinished work was declining, and employees consistently described a more efficient process, I’d have a much stronger basis for saying the organization had successfully redesigned the operation. I wouldn’t keep searching for evidence of hidden strain when the available evidence supported genuine improvement.
I could also find that the urgent-service result stayed stable because the team made deliberate decisions about which work to prioritize, and that would lead to a different but equally useful explanation. Leadership might be comfortable with the tradeoff once it understood what was accumulating elsewhere, or it might decide the additional delays created risks the original service measures hadn’t captured. Either way, we’d be discussing an identifiable operating decision rather than treating the unchanged percentage as proof that every part of the service had been unaffected.
Where the distinction becomes especially important is in what leadership decides to do with the result afterward. If the organization concludes that the original team had more capacity than it needed, another department might reasonably be asked whether it could operate with a similar reduction. But if the first team’s performance remained stable because it had already automated a substantial portion of its work, redesigned its processes, or moved certain responsibilities to a vendor, the second team may not have any of those conditions in place.
The first reduction could’ve been entirely successful while providing very little evidence that repeating the same percentage elsewhere would produce the same outcome. I’d be much more comfortable discussing which parts of the original operating change contributed to its success than treating the staffing reduction itself as the thing another team should replicate. That distinction could affect everything from investment in automation to implementation timing, cross-training, service expectations, and whether the next department is realistically prepared to operate with fewer people.
There may also be situations where I can’t get to a clean explanation because too many things changed together, important work wasn’t recorded, or the available evidence can’t distinguish between competing possibilities. I’d still want to explain what I’d examined and which interpretations remained plausible, rather than continuing to add proxies until the analysis appeared more conclusive than it really was. A company can make reasonable operational decisions under uncertainty, and I think People Analytics should be able to communicate that uncertainty without turning it into an excuse for avoiding a recommendation.
What interests me about all of this is how naturally we move from describing an outcome to explaining an organizational decision, particularly when the result is reassuring and the explanation makes intuitive sense. I can see why a leader would interpret stable performance after a staffing reduction as evidence that the new operating model worked, and I may ultimately agree. But I’d want to understand whether the result depended on better processes, a different mix of work, deliberate prioritization, employee adaptation, or some combination of those things before treating that interpretation as established.
I don’t think every unchanged KPI warrants months of investigation, especially when the business decision is relatively small or the explanation is already well supported. But when an organization makes a meaningful change and uses the resulting performance measures to evaluate whether it worked, I’d want the analysis to be useful beyond confirming what happened to the metric. If we’re going to conclude that a team handled a staffing reduction well, I’d want us to understand enough about the conditions that made it possible to know what we’re actually recommending when someone inevitably asks whether another team could do the same.
