Background
Functional testing of enterprise systems — payments, approvals, configuration back-offices — has long depended on manual work: writing cases from requirement documents is slow and incomplete, execution is click-by-click and easy to skip, and results are usually a bare pass/fail with no evidence. The hardest class of defect to catch systematically is "documentation versus implementation": the system reaches a legitimate state, just not the one the process document expects. Script-based automation is expensive to maintain and largely has to be rewritten for each new system under test.
What we built
- Materials to test cases: upload requirement documents, process definitions, prototypes and the client's own case template; the platform parses them into a previewable, editable intermediate representation (flows, nodes, roles, actions, state machine, control inventory), then generates structured cases per business chain. Chinese or English, each case carrying an automation level and a link back to its source, with anything unsupported marked as inferred.
- Execution in real browsers: agents drive real browsers from the cases, with an isolated session per role and concurrency across environments — serial within one business instance, parallel across chains, with an account pool leasing credentials per role.
- Rules first, model as fallback: toast text, element state, API error codes and entity state are judged by deterministic rules; only natural-language assertions go to a model. Verdicts are three-state — pass, fail, needs human review — so uncertainty is never dressed up as certainty.
- Evidence throughout: before/after screenshots, page text and accessibility-tree snapshots, API response summaries, console errors and session recordings, with credentials redacted from every artefact and log.
- Watchable and interruptible: parallel swim lanes show execution live; runs can be paused, steps skipped, and execution resumed after edits; manual steps wait for input.
- New system, no code changes: the roles, action vocabulary, state labels, locator preferences and waiting strategy of a system under test live in a single "system profile" — switching systems means switching that profile.
- Cost and safety under control: tokens and spend are metered per model call with budget cut-offs; a host allow-list blocks out-of-scope navigation, high-risk steps require pre-approval, and test data carries a shared prefix.
Technical approach
| Layer | Choice |
|---|---|
| Execution | Playwright driving real browsers, one context per role |
| Generation & judgement | Multiple models routed by tier (generation / judgement / re-judgement / vision) |
| Orchestration | Chain-level concurrency, blocking propagation, checkpoint resume |
| Delivery | Public SaaS; control plane and execution nodes both in the cloud, nodes routed by network label |
| Reporting | Three-state counts, state-machine coverage, node × action matrix, defect de-duplication, export to the client's xlsx template |
Results
| Item | Detail |
|---|---|
| First system under test | B2B cross-border payment platform (client name confidential) |
| Case generation | Generated per process domain in chains, up to 400 cases per run |
| Verdict model | Three-state result plus a human-review loop; failures are merged into defects automatically |
| Defect categories | Functional, doc-vs-implementation, UI data, layout, colour, permissions, stability |
| Coverage and regression-time comparison | Not disclosed |
Authorization
The product, methodology and technical approach are public; the first client's name, business data and test results are withheld under a confidentiality agreement.