Phase 4: Feedback & Learning

This phase is intentionally conservative. Reviewer text is treated as feedback, not as unchecked system instructions. Testonaut can fix the PR or propose a learning rule, but the durable knowledge update is always raised through a separate human-reviewed PR.

What this phase owns

PR responder command

testonaut respond-to-pr --repo <path> --pr <id> --issue <KEY> [options]

The command resolves the VCS adapter from the repository remote — Azure DevOps and GitHub are supported through the shared adapter interface, used to list PR threads, post replies, and raise learning proposal PRs.

Reviewer commands

@testonaut

Use when a reviewer wants Testonaut to react directly on the implementation PR:

@testonaut use the existing LoginPage helper instead of creating a new helper.
@testonaut why did you add a separate scenario for the cancellation case?
@testonaut thanks, looks good.

The responder classifies the comment into one of four intents:

@update-knowledge

Use when the reviewer wants a project-specific rule captured for the target repository:

@update-knowledge Prefer existing page-object helpers before adding new Playwright locators.

This creates or updates a Markdown file under .agents/guidelines/<category>.md in the target repository. The change is raised as a separate PR — it is not merged automatically and does not silently modify the implementation branch.

@update-agent

Use when the reviewer wants to propose a general Testonaut behavior change that should apply beyond the current project:

@update-agent When a repository has Cucumber step definitions, always inspect them before writing new step text.

This targets the Testonaut agent repository (proposed-guidelines/<category>.md), not the project repository. The runtime must know where the Testonaut checkout lives via TESTONAUT_AGENT_REPO. If the variable is missing, the responder refuses @update-agent cleanly and posts a clear reply.

Feedback-scoped fix flow

When a comment is classified as fix, Testonaut does not rerun the whole implementation phase. It creates a narrow repair prompt that includes the Jira issue key, the target repository root, the exact reviewer feedback, the reviewer name when available, the verify command, and the files changed by the original implementation. The prompt enforces the feedback boundary:

• make the smallest change that addresses the review comment;
• do not refactor unrelated code;
• do not modify product code;
• treat .feature files as the source of truth unless the reviewer explicitly asks to change the test design;
• reuse existing helpers, fixtures, page objects, and selector conventions;
• do not weaken assertions, skip scenarios, or hide failures to make verification pass.

The reply summarizes what changed, which files changed, whether verification passed, and the verification exit code. If verification still fails, Testonaut includes enough output context for the reviewer to understand what remains.

Verification after a fix

The verification command comes from the selected stack in .agents/testonaut.config.json, unless overridden by --verify. Phase 4 validates the same path that humans and CI care about — if the project has a Docker-backed test stack from Phase 3, the responder can reuse that configured verify command when applying PR feedback.

Learning and Markdown guideline updates

Learning is separated from immediate PR repair. A reviewer can ask Testonaut to capture a durable rule, but Testonaut only proposes that rule through a reviewable Markdown change. The classifier decides which file should receive the proposed rule:

Project-level rules live under .agents/guidelines/; agent-level rules under proposed-guidelines/. If a guideline file does not exist, Testonaut creates it with a heading and provenance note. If the same proposed rule is already present, it skips the duplicate edit.

Why learning is proposed through PRs

Reviewer feedback can contain useful rules, but it is still untrusted input. Phase 4 uses a review-first learning model:

• no automatic merge into the project guidelines;
• no automatic merge into Testonaut's own packaged guidelines;
• no hidden edits to the implementation branch;
• each rule is visible as Markdown;
• every proposal has provenance back to PR feedback;
• humans decide whether the rule is correct, broadly applicable, and safe.

This keeps learning useful without letting a single PR comment permanently change agent behavior without review.

Loop safety

Audit and observability

The audit path is under the local out directory for the responder run, and configured redaction patterns are applied before writing audit details.

Operational requirements

• a supported VCS remote — currently Azure DevOps or GitHub;
• a token for the VCS provider, such as AZURE_DEVOPS_PAT or GITHUB_TOKEN;
• a valid .agents/testonaut.config.json with PR settings and at least one stack verify command;
• local credentials or configured push auth if Testonaut needs to push proposal branches;
• TESTONAUT_AGENT_REPO when using @update-agent.

{
  "pr": {
    "mention": "@testonaut",
    "maxIterationsPerPr": 5
  }
}