Conversation
mrge from upstream
Implement a new `preliminary-testing` supervisor workflow that automates the evaluation of preliminary testing (gating and CI checks) for RHEL Jira issues. Previously, QE engineers had to manually verify test results and set the Preliminary Testing field — this agent handles it automatically. The workflow uses an AI agent (BeeAI ToolCallingAgent) to analyze test results from two sources: 1. GreenWave gating status — fetches and interprets the HTML page from gating-status.osci.redhat.com for the build NVR when available. 2. OSCI results in MR comments — discovers linked merge requests via the Jira dev-status API, then fetches MR notes from GitLab to find OSCI "Results for pipeline ..." comments. The workflow gracefully degrades when only one source is available (e.g. no build NVR set yet, or no linked MRs). Entry conditions: issue must be In Progress, Preliminary Testing not already Pass, and at least one of Fixed in Build NVR or linked PRs. Outcomes: - Tests passed + Test Coverage set → sets Preliminary Testing = Pass - Tests passed + Test Coverage missing → flags for human attention - Tests failed/not running/error → flags for human attention - Tests running/pending → reschedules for later New files: - supervisor/preliminary_testing_handler.py — main workflow handler - supervisor/preliminary_testing_analyst.py — AI agent for analysis - supervisor/tools/fetch_greenwave.py — BeeAI tool for GreenWave HTML - supervisor/tools/fetch_gitlab_mr_notes.py — BeeAI tool for MR notes Modified files: - supervisor/jira_utils.py — add set_preliminary_testing() and get_issue_pull_requests() (Jira dev-status API) - supervisor/main.py — add preliminary-testing CLI command - Makefile — add preliminary-testing make target - README-supervisor.md — document the new workflow Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new 'preliminary-testing' workflow. This workflow leverages an AI agent to analyze Jira issues, GreenWave gating status, and GitLab merge request comments to determine if a build has passed preliminary testing. The changes include adding a new make preliminary-testing target, updating the README with detailed documentation for this workflow, and implementing new Python modules for the handler, the AI analyst, and tools to fetch data from Jira, GreenWave, and GitLab. A review comment suggests improving the ValueError message in preliminary_testing_handler.py for better debuggability by including the specific state value.
| raise ValueError( | ||
| f"Unknown testing state: {analysis.state}" | ||
| ) |
There was a problem hiding this comment.
The ValueError raised in the match statement should include the specific state value in the error message to improve debuggability.
| raise ValueError( | |
| f"Unknown testing state: {analysis.state}" | |
| ) | |
| raise ValueError( | |
| f"Unknown testing state: {analysis.state!r}" | |
| ) |
1 participant
TODO:
packit/packit.dev.Fixes
Related to
Merge before/after
RELEASE NOTES BEGIN
Packit now supports automatic ordering of ☕ after all checks pass.
RELEASE NOTES END