apx-testkit — We asked for a second user. One showed up.

We asked for a second user. One showed up.

Our last update ended with a challenge: find a second independent team, point apx-testkit at a real Oracle APEX application, and see where it breaks. It was the only way to surface our next wave of bugs, since we had already squeezed every last insight out of staring at our own codebase.

A few weeks later, someone took us up on it. Here is what happened, alongside everything else we shipped in the interim.

The bug report we were waiting for

A user exported their own app — using Sample Reporting via SQLcl 26.2.1 on Windows — and ran it straight through apx-testgen. It failed instantly on every single page with a clear, uncompromising error:

page '1' is missing a valid integer 'page:' property.

Our reference fixtures all had an interior page: N line inside every page N ( ... ) block. Every fixture we had ever written did. But the user’s real SQLcl export didn't—and as it turns out, neither does any real Oracle-generated export.

We pulled the exact file from Oracle’s own oracle/apex GitHub repository to verify. The user was entirely right: page 1 ( is followed immediately by name: ..., with no interior property to be found. Our EBNF grammar marked that property as "required," but it only belonged to a group that real-world exports omit because the page number is already right there on the opening line. Our parser had been looking at the wrong field all along; it just never encountered a real export that didn't carry our own redundant, hand-crafted fixtures.

We fixed it, verified the patch against a full 42-page export end-to-end, and immediately unmasked a second bug right behind it: the same app lacked a runtime { } block in application.apx, and our parser was treating its absence as fatal. Same root cause, same fix—an EBNF-documented group that real-world exports simply leave out.

The user re-ran the suite, confirmed both fixes, and surfaced a third issue on the way out: a generated page title asserting the literal string &APP_NAME. instead of the app's actual name because we weren't resolving APEX's static substitution tokens. Fixed and verified against the real export, too.

Get Satwik’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Three genuine bugs from one real second user in about a week. This is the exact feedback loop we needed.

Everything else since the last post

• The generic runtime escape hatch is gone. We used to expose a raw callRegionMethod(region, method, args) to invoke anything on apex.region(), whether it was verified or not. It’s now package-internal. Every public class exposes exclusively named, evidence-backed methods.
• Region resolution stopped being a guess. Our previous rule about exported IDs occasionally failing to match runtime IDs used to mean falling back to a static assumption when uncertain. resolveRegion() now tests each evidence-backed candidate live against the actual running page and explicitly reports which one succeeded.
• A structured verification registry. All the discipline of verifying and re-verifying from our last post has evolved into a machine-checkable ledger. It consists of one JSON file mapped to a strict schema across five evidence levels (verified, documented, observed, unverified, unsupported), wired directly into CI so the support matrix can never silently drift.
• Flow Map (apx-flow). A deterministic navigation graph built entirely from the typed AST without requiring a live app. It reads every branch, region action, column link, and button target in the export to produce a clear graph showing which pages reach others, complete with confidence levels and cited evidence per edge.
• A unified CI dashboard (apx-report). This bundles coverage, regression diffs, and parser warnings into a single self-contained HTML artifact—pure composition of data the other tools already compute.

The newest addition: apx-onboard

Oracle introduced two distinct ways to build APEX apps with AI this year: the App Builder’s “Create App Using Generative AI,” and spec-driven development via 26.1, where an agent converts a functional spec and your schema into an importable Markdown blueprint. In both cases, Oracle positions the resulting APEXlang export as the reviewable, Git-friendly source of truth for the AI’s output.

That happens to be the exact layer we sit on top of, so we built the front door for it.

apx-onboard is a single orchestration command—also exposed as our seventh MCP tool, onboard_generated_apex_app. It takes a freshly generated export and runs it through everything we have: manifest and version validation, deep inspection, an optional diff against a prior export, the flow map, documentation generation, and full Playwright test creation—all bundled into one comprehensive onboarding report. The report highlights precisely what remains unmodeled, what is unsafe to auto-route, and what still requires human eyes on a live system, derived entirely from actual execution rather than a canned checklist.

SQLcl’s own apex validate utility is built right in as an opt-in feature (off by default). If you request it and the validation fails to run, the entire process fails loudly—we don't bury a skipped validation in a report and call it a day.

This isn’t apx-testkit morphing into yet another AI app generator. It’s the exact opposite bet: let Oracle's native tools build the app, and serve as the deterministic safety net that checks its work before anyone dares trust it in production.

What’s still open

Two primary blockers stood between us and generated CRUD tests: a lack of a typed primary-key field, and the absence of a real page to observe a create-and-read-back round trip. The first is officially resolved — ApexProcess.target is now a strongly typed field rather than a raw cross-reference between disparate AST nodes. The second remains a matter of finding the right credentialed app to test against.

Calendar and Map regions remain right where they were last time: we understand their exported structure, but we still haven’t exercised either against a live application.

The ask remains unchanged. If something you just read is the exact feature you’ve been waiting for, or if you have a real export that breaks in a way ours never has — send it our way. It’s the fastest path to getting the next item on this list checked off.

previous story — https://medium.com/@jsatwik/under-the-hood-honestly-apx-testkits-architecture-and-what-s-still-open-ae02669f31f0