UAT Beaker Upgrades
UAT Scripts for Beaker Upgrades: What I Actually Test and Why
Every time Epic ships a Beaker upgrade, somebody on your lab informatics team has to write the UAT scripts. If you are new to that job, you probably picked up a template from your predecessor and started modifying it, and the whole thing felt a little arbitrary. Why these tests. Why in this order. Why only these result components.
This post is about the UAT scripts I actually write, in the order I write them, and the reasoning behind the choices. I have run UAT for Beaker upgrades at four health systems, and the scripts I use now look very different from the ones I started with.
Test the Interfaces First, Not the Workflow
The first mistake I made as a new analyst was writing UAT scripts that started with the clinical workflow. Place the order, collect the specimen, run the test, result the test, verify the result in ClinDoc. End to end.
That is a reasonable test, but it is the wrong first test.
The right first test is the interface. HL7 ORU, HL7 OML, HL7 MFN, HL7 ORL. Every lab interface the hospital has, pointing in and out of Beaker. For each interface, you build a focused script that sends a single message, checks that Beaker consumes or emits it correctly, and verifies the acknowledgment comes back.
The reason you test interfaces first is that interface failures are the hardest to diagnose during a go-live. If an orderable procedure is wrong, a tech will tell you within an hour. If an HL7 interface is dropping messages, you will not know until a provider calls to ask where the result is for a patient from three days ago. By then you are already behind.
My interface scripts are tiny, maybe 20 to 30 lines each, and I run them before any workflow script. If the interface tests pass, I move on. If any one of them fails, I escalate before anybody else touches the environment.
Test the Master File Dependencies
The second thing I test is the master file dependency chain.
Beaker master files reference each other in ways that are not always obvious. An orderable procedure references a result component. A result component references a LOINC code and a unit. A unit references a scale. A specimen source references a container type. A container type references a draw priority.
When Epic ships an upgrade, any one of those dependency links can be the thing that broke. If you only test the top level (the orderable), you will miss the broken link one level down.
My master file dependency scripts walk the chain explicitly. For each critical orderable, I verify that the result components exist, the LOINC codes resolve, the units are correct, the specimen sources are valid, the container types are valid, and the reference ranges are populated and age-stratified where appropriate. This is tedious. It is also the single most valuable test suite I run.
I caught a bug at Mayo during a 2023 upgrade where the reference range for pediatric creatinine had reverted to the adult reference range after the upgrade. The orderable looked fine. The result component looked fine. The LOINC was fine. The unit was fine. The reference range was the broken link. If I had only tested the orderable I would have shipped a Beaker build that under-flagged elevated creatinine in children.
Test the Reflex and Cascade Logic
The third test suite covers reflex testing and cascade orders. This is where Beaker gets interesting and where upgrades most often break things.
A reflex test is a lab order that is automatically generated based on the result of another test. For example, a positive hepatitis C antibody automatically reflexes to a hepatitis C RNA PCR. A critical troponin automatically generates a repeat troponin in two hours. The reflex rule is defined in Beaker and it depends on the result component, the result value, and often a context like patient location or ordering provider type.
Upgrades to Beaker regularly change the way reflex rules are evaluated. Not always in ways Epic announces. I have seen reflex rules silently stop firing because the result component the rule was tied to had its value coding changed in an upgrade.
Get Cynthiankemumeh’s stories in your inbox
Join Medium for free to get updates from this writer.
My reflex testing script covers every active reflex rule in the lab. For each one, I generate a test result that should trigger the reflex, verify the reflex fires, and verify the reflexed order routes correctly. If a reflex should not fire (because the result is in a normal range), I verify that it does not fire. Both directions matter.
Test the Downtime Procedures
The fourth test is downtime, and most analysts skip this. Do not skip it.
Every hospital has a Beaker downtime procedure. Usually it involves paper requisitions, manual result entry after recovery, and a specific set of printers that must be available when Chronicles is down. The downtime procedure gets written once, taped to a wall, and forgotten about until the first unplanned downtime.
Every upgrade is an opportunity to verify that the downtime procedure still works. The downtime printers often have IP addresses that change during infrastructure upgrades. The paper requisition forms often reference orderables that have been renamed. The result entry workflow after recovery often depends on a specific user role that may have been reconfigured.
My downtime script runs through the full procedure in a sandbox environment, including the recovery steps. If anything is broken, we fix it before go-live, not during the first real downtime.
I ran this test at the Nashville medical center and discovered that the downtime recovery workflow was pointing at an orderable code that had been deprecated two upgrades ago. The procedure document was out of date. If we had hit a real downtime before the next upgrade, we would have been entering recovery results against a dead orderable.
Test the Edge Cases That Have Burned You Before
The last section of my UAT scripts is personal. It is the list of specific edge cases that have burned me or somebody I know in a previous go-live.
Some examples of what lives in this list.
Hemolysis, icterus, and lipemia (HIL) indices. Does Beaker still flag HIL correctly after the upgrade. Does the HIL comment get appended to the result correctly.
Autoverification. For labs that run autoverification on routine results, does the autoverification logic still fire. Does it still respect the critical value escape path (critical values should never autoverify).
Blood Bank crossmatch logic. For labs that use Beaker Blood Bank, this gets its own full test pass every time. No shortcuts.
Microbiology sensitivities and antibiograms. Does the antibiogram still populate correctly. Do new sensitivity breakpoints get applied.
Specimen rejection workflow. Does the specimen rejection reason set still include all the active rejection reasons. Is the recollection order generated correctly.
Each item on this list exists because somebody got burned by it, including me. Keep the list. Add to it after every upgrade. Share it with the next analyst who inherits your build.
The Documentation That Matters
The last thing I will say about UAT is that the test documentation is as valuable as the tests themselves.
Every script I run gets a result row. Pass, fail, or note. If a test fails, I document the specific failure, the master file involved, the person who made the fix, and the retest result. All of this goes into the upgrade run book.
The reason this matters is that six months later, when something subtle breaks in production, you are going to need to figure out whether the subtle thing is a new bug or a known behavior change from the last upgrade. The only way to answer that question quickly is to look at the upgrade documentation and see whether the behavior was tested.
If the behavior was tested and passed, the subtle thing is a new bug. If the behavior was not tested, you are flying blind.
I have been burned both ways. The run book is cheap insurance. Write it even when nobody asks for it. Future you will thank present you.
