How to Integrate Testonaut with a Jira Project

Integration overview

The most important part is the automation rule. The rule sends a JSON payload to the Testonaut runner pipeline, and that payload tells Testonaut which ticket to process, which mode to run, and which git repository to use.

What you need before creating the rule

Step 1: Prepare the target repository

Add the Testonaut configuration under the target repo at .agents/testonaut.config.json. The Quick Start covers testonaut init, the readiness check, and the minimal and Docker-backed stack configs in full.

Step 2: Choose the Testonaut runner pipeline

The pipeline definitions accept these parameters:

Note the pipeline id — you need it for the Jira web request URL:

https://dev.azure.com/afs-dev/riverty-test-automation/_apis/pipelines/2571/runs?api-version=7.1

Step 3: Prepare the git automation credentials

There are two different credentials involved. Keeping them separate makes troubleshooting easier.

The pipeline clones the target repository using git -c http.extraHeader so the token does not appear in the remote URL or logs. Clone URL shapes:

# Azure DevOps
https://dev.azure.com/<org>/<project>/_git/<repo>

# GitHub
https://github.com/<owner>/<repo>.git

The runner configures git identity before committing:

git config --global user.email "testonaut-agent@riverty.com"
git config --global user.name "Testonaut Agent"

Step 4: Encode the automation PAT for Jira

Jira Automation must call Azure DevOps with Basic auth. Azure DevOps expects an empty username and the PAT as the password, so encode the string :<PAT> (note the leading colon):

# PowerShell
"Basic " + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":PASTE_YOUR_PAT"))

# Bash
printf ':%s' 'PASTE_YOUR_PAT' | base64 | tr -d '\n'

Store the complete value as a Jira Automation secret, for example AZDO_BASIC, and reference it in the rule's Authorization header.

Step 5: Create the Jira Automation rule

In Jira: Project settings → Automation → Create rule. Recommended rule shape:

• Trigger: Manual.
• Condition: issue type is Story, Task.
• Add field: field name Repo, variable name targetRepo, optionally with a default repository value.
• Action: Send web request.

Step 6: Configure the Send Web Request action

Step 7: Add the JSON request body

Design only

Use this when the transition means "create or update test design in Xray."

{
  "templateParameters": {
    "mode": "design",
    "issueKey": "{{issue.key}}",
    "targetRepo": "https://dev.azure.com/<org>/<project>/_git/<repo>",
    "reporter": "{{initiator.accountId}}",
    "deriveRequirements": "true",
    "adversarialCritique": "true",
    "riskCoverage": "warn"
  }
}

targetRepo is optional for design, but keep it when possible — with a repo, Testonaut can inspect existing Cucumber features and avoid generating duplicate Gherkin steps.

Implement only

Use this after the test design is approved and Xray tests are linked to the story.

{
  "templateParameters": {
    "mode": "implement",
    "issueKey": "{{issue.key}}",
    "targetRepo": "https://dev.azure.com/<org>/<project>/_git/<repo>"
  }
}

Design and implement in one run

{
  "templateParameters": {
    "mode": "both",
    "issueKey": "{{issue.key}}",
    "targetRepo": "https://dev.azure.com/<org>/<project>/_git/<repo>",
    "reporter": "{{initiator.accountId}}",
    "deriveRequirements": "true",
    "adversarialCritique": "true",
    "riskCoverage": "warn"
  }
}

Coverage analysis

Use this for scheduled or manual checks comparing story acceptance criteria against linked Xray tests.

{
  "templateParameters": {
    "mode": "coverage",
    "issueKey": "{{issue.key}}",
    "targetRepo": "https://dev.azure.com/<org>/<project>/_git/<repo>",
    "reporter": "{{initiator.accountId}}"
  }
}

Step 8: Add project-mandatory Xray fields

Some Jira/Xray projects require extra fields when creating a Test issue. Add them through testFields. Because Azure pipeline parameters are strings, testFields must be an escaped JSON string:

{
  "templateParameters": {
    "mode": "design",
    "issueKey": "{{issue.key}}",
    "targetRepo": "https://dev.azure.com/<org>/<project>/_git/<repo>",
    "testFields": "{\"customfield_15714\": 170}",
    "reporter": "{{initiator.accountId}}"
  }
}

How to find the field:

• Find the project's Xray Test/Test Case issue type id.
• Query Jira create metadata for that project and issue type.
• Look for required fields and their allowed values.
• Add the write value to testFields.

If Jira returns an error like "operation value must be a number," the field id is probably correct but the value shape is wrong.

Step 9: Recommended rule set

For most Jira projects, create separate rules instead of one rule doing everything:

This separation keeps the workflow predictable. The implementation rule does not need artifacts from the design rule because it reconstructs the test design from the Xray tests linked to the Jira story.

Step 10: Verify the integration

• Trigger the rule on a test story and check the Jira Automation Audit Log.
• Confirm the web request returned 200 or 201 from Azure DevOps.
• Open Azure DevOps and confirm a new pipeline run started with the expected mode, issueKey, and targetRepo.
• For design, confirm Xray tests are created and linked to the story as is tested by.
• For implement or both, confirm a branch and PR were created in the target repo.
• Open Langfuse and confirm a session exists for jira:<ISSUE-KEY>.
• Download the testonaut-out pipeline artifact and inspect context.json, test-design.json, implementation.json, audit-log.jsonl, and cost-ledger.jsonl.

Troubleshooting

Final checklist

Before enabling the rule for the whole project, confirm:

• .agents/testonaut.config.json exists in the target repo.
• The runner pipeline exists and has a known pipeline id.
• testonaut-secrets contains Jira, Xray, LLM, VCS, and Langfuse variables.
• Jira Automation has the encoded Azure DevOps Basic auth secret.
• The automation rule uses POST and Content-Type: application/json.
• The request body includes mode, issueKey, and the correct targetRepo for implementation runs.
• Required Xray fields are provided in testFields if the project needs them.
• The rule trigger cannot fire repeatedly for the same design event.
• A test run creates linked Xray tests and, for implementation, a PR in the target repo.