Phase 2: Test Design Agent

This phase is where Testonaut stops being a generic test-case generator and starts behaving like a repository-aware QA architect. It does not only read the Jira story: when a target repository is available, it scans the existing Cucumber feature files and reuses the team's existing Gherkin step vocabulary so it does not create near-duplicate steps that later require duplicate step definitions.

Inputs and outputs

Where the repository connection belongs

Repository discovery starts from information gathered in Phase 1, but repository-aware step reuse belongs to Phase 2 because it directly changes how the Test Architect writes scenarios. There are two ways the design phase gets a repository:

If no repository is resolved, Testonaut still designs tests, but it cannot reuse existing step phrasings. The CLI makes that visible with a message telling the user to pass --repo or configure TESTONAUT_REPO_ROOTS.

Key capability: repository-aware Gherkin step reuse

Most AI test generators can write plausible Cucumber scenarios, but they often create subtly different phrasings for behavior that the repository already supports. That causes duplicate Gherkin steps and duplicate step-definition code:

Humans understand those as equivalent. Cucumber does not — if the generated wording is different, the implementation phase may need new glue code even though equivalent step definitions already exist. Testonaut handles this by scanning the target repository before scenario authoring.

What the scanner reads

The step library scanner walks the repository and reads .feature files, ignoring generated or irrelevant folders (node_modules, .git, .agents, dist, out, build, coverage, bin, obj, target). It collects:

Step normalization

The scanner normalizes value-like literals so structurally identical steps collapse into one reusable template:

This is what lets the prompt say "reuse this step shape, substituting the right data," instead of treating every literal value as a separate step.

How step reuse reaches the LLM

When a step library exists, the Gherkin prompt includes a dedicated section, grouped by keyword (Given / When / Then), instructing the model to reuse those exact phrasings and only write a new step when none matches. This is not a soft hint hidden in a paragraph — it is an explicit authoring constraint, because avoiding duplicate Cucumber glue code is a first-class design requirement.

Existing test de-duplication

Repository scenario de-duplication

While scanning .feature files, Testonaut also collects scenario and scenario-outline titles. Those titles are placed in the prompt with the instruction: do not author a scenario that means the same thing; if an acceptance criterion is already covered by an existing repo scenario, put that AC id in acsCoveredByExisting instead of writing a duplicate scenario.

Linked Xray test de-duplication

The Researcher includes already-linked Xray tests in ContextPack.xrayTestCases, and Phase 2 adds them to the existing-test prompt section. During Xray upload, Testonaut performs another safety check: it normalizes scenario titles and skips tests whose title already exists among linked tests or among scenarios created in the same run — catching case, punctuation, or unicode differences that would otherwise create near-duplicates.

Two-pass design flow

The Test Architect uses multiple bounded design passes rather than one big prompt.

Pass 1: acceptance criteria extraction

The first LLM pass extracts atomic acceptance criteria from the ContextPack. Each AC is assigned an id such as AC-1, written in business language, and can point back to its source — the description, a comment, or a Confluence page. After AC extraction, the evidence graph is rebuilt so requirement nodes appear in the graph.

Pass 2: scenario authoring

In Gherkin mode, the author receives the existing repository step vocabulary and is instructed to reuse it. In plain-English mode, step-library reuse is not relevant, but existing test de-duplication still matters.

Scenario contract

Each generated scenario carries structured fields used by gates, reporting, and implementation:

Tag hygiene

Tags are deliberately kept simple — the agent does not use tags for data that belongs in structured fields:

The Jira key is added at the feature level automatically. @smoke is derived from risk: high. AC ids, risk, and polarity live in structured fields, which keeps coverage checks machine-readable.

Coverage gates

The Coverage Auditor checks the generated design before it is accepted:

If AC coverage fails and the gate is required, the run stops with a clear error. If the gate is warn, the gap is recorded as an open question and warning.

Derived requirements and adversarial critique

The literal acceptance criteria are the floor, not always the whole test strategy. Phase 2 can also mine implied requirements from the full context — missing but evidence-backed behaviors such as:

• error and rejection paths;
• boundary values;
• missing or maximal data;
• permissions and security expectations;
• state transitions and concurrency;
• notable usability or performance constraints.

After the initial scenario design, an adversarial critique pass asks: "what did this design miss?" It can add genuinely missing scenarios, record coverage gaps, or raise open questions. It is bounded, evidence-grounded, and not allowed to duplicate already drafted scenarios.

Xray upload

When upload is enabled, Testonaut applies the TestDesignPack to Xray:

• Gherkin scenarios become Xray Cucumber tests; plain-English scenarios become Xray Manual tests.
• Tests are linked back to the story using the configured or resolved Jira link type.
• Required Jira fields can be inherited or supplied through xray.testFields / --test-fields.
• Titles are normalized to avoid duplicate creation across reruns.
• Upload can run as --dry-run or require --yes / policy approval.

Failure behavior and guardrails

Configuration knobs