JUnit XML to Aqua Integration

JUnit XML to Aqua Integration Overview

Purpose Automate the flow from local/unit test execution → JUnit XML generation → ingestion into Aqua as grouped test executions (by Aqua Test Scenario) with pass/fail outcomes, enabling CI/CD visibility and governnce.

High-Level Flow

1. Developer writes/maintains tests (CodeceptJS / Playwright) using naming conventions embedding Aqua IDs.

2. Tests run (locally or in GitLab CI) producing a JUnit XML file at output/result.xml.

3. GitLab CI captures the XML as an artifact and (optionally) a JUnit report.

4. A publishing stage runs an importer that:

   1. Authenticates against Aqua ( AQUA_URL , AQUA_USER , AQUA_PASSWORD ).

   2. Parses the JUnit XML.

   3. Extracts Aqua Test Scenario IDs (TS…) and Test Case IDs (TC…).

   4. Groups cases by Scenario ID.

   5. Performs one REST API call per Scenario to create executions & results. Exits with appropriate status code so the pipeline can pass/fail.

Test Authoring Conventions

Use IDs inside test suite / test names so they can be parsed:

• Scenario (suite/classname/title) prefix:

• Test case (individual test) prefix: TC (e.g., TC1132533 - Load the GitHub homepage )

Regex patterns:

• Scenario ID: /(TS\d+)/

• Case ID: /(TC\d+)/

If an ID is missing, treat that test as invalid (policy: fail import or skip—configurable, recommended fail for data integrity).

Generating JUnit XML (CodeceptJS + mocha-junitreporter)

Typical script (already configured):

npm ci
npx playwright install --with-deps
npm run test # Produces output/result.xml

Ensure codecept.conf.ts contains a mocha reporter configuration pointing to output/result.xml (e.g. via mocha-junit-reporter ).

GitLab CI/CD Pipeline Structure ( .gitlab-ci.yml )

Stages (recommended):

stages:
  - test
  - upload

Test stage example:

execute_tests:
  stage: test
  image: node:20
  script:
    - npm ci
    - npx playwright install --with-deps
    - npm run test
  artifacts:
    when: always
    expire_in: 1 week
    paths:
      - output/
    reports:
      junit: output/result.xml

Upload/import stage example:

upload_to_aqua:
  stage: upload
  image: mcr.microsoft.com/powershell
  dependencies: [execute_tests]
  script:
    - pwsh ./upload.ps1

(Or use a .NET image / custom importer binary if needed.)

Environment Variables (CI/CD Secrets)

Set in GitLab → Settings → CI/CD → Variables (masked & protected as appropriate):

• AQUA_URL (e.g., https://workshop.aqua-cloud.io/aquaWebNG )

• AQUA_USER

• AQUA_PASSWORD

• Optionally REPORT_FILE (default ./output/result.xml )

Importer Responsibilities (Detailed)

For each run:

1. Read JUnit XML ( REPORT_FILE ).

2. Authenticate: POST token endpoint -> receive bearer token.

3. Parse <testsuite> / <testcase> nodes.

4. For each <testcase>:

   1. Extract Scenario ID (look in classname or parent suite name text).

   2. Extract Case ID (look in name ).

   3. Determine status:

      1. Passed: absence of <failure> and <error>.

      2. Failed: presence of <failure> or <error> (capture message + stack if present).

5. Group test cases by Scenario ID (all with same TS… together).

6. Build JSON payload per Scenario containing all included case executions.

7. POST to Aqua Project / Execution endpoint (single call per Scenario).

8. Track any failed POSTs.

9. Exit with appropriate code (see below).

Exit Codes (Suggested Mapping)

Code- Meaning

• 0 —Success (all scenarios posted)

• 2 — Authentication failure

• 3 — Network / endpoint unreachable

• 4 — Partial failure (some scenarios failed)

• 5 — Invalid or unreadable XML

• 6 — Missing required IDs / validation error

• 7 — Unexpected internal exception

CI will fail the job if exit code != 0 (default GitLab behavior), thus potentially failing the pipeline.

JUnit XML Example

<testsuite name="CodeceptJS" tests="2" failures="0">
  <testcase classname="TS1132748 - GitHub" name="TC1132533 - Load the GitHub homepage" time="2.131" />
  <testcase classname="TS1132748 - GitHub" name="TC1132534 - Load the Google homepage" time="1.842" />
</testsuite>

Troubleshooting

Empty result.xml

Symptom: Empty result.xml

Likely Cause: Tests didn’t execute or reporter misconfigured

Action: Re-run with --verbose, check reporter config

401 Unauthorized

Symptom: 401 Unauthorized

Likely Cause: Bad credentials or URL

Action: Verify env vars, token endpoint path

Missing IDs error

Symptom: Missing IDs error

Likely Cause: Test names not following convention

Action: Rename tests to include TS / TC prefixes

Partial posting

Symptom: Partial posting

Likely Cause: Network/API intermittent

Action: Retry logic or mark exit code 4

Pipeline green but no data in aqua

Symptom: Pipeline green but no data in aqua

Likely Cause: Import stage skipped

Action: Ensure stage not when: manual and dependencies correct

Summary

This setup enforces traceability between automated test execution and Aqua-managed test scenarios/cases by embedding IDs in test names, producing standard JUnit XML, and transforming that into structured Aqua executions inside the CI/CD pipeline, with robust exit codes controlling pipeline outcome.