diff --git a/.github/actions/overture-projection/README.md b/.github/actions/overture-projection/README.md new file mode 100644 index 0000000..9d9d592 --- /dev/null +++ b/.github/actions/overture-projection/README.md @@ -0,0 +1,149 @@ +# Overture PRojection + +Posts an AI-generated code review comment on a pull request. Skills drive what the model looks for — each skill is a `SKILL.md` file that provides focused review instructions for a particular concern. + +Supports [GitHub Models](https://docs.github.com/en/github-models) (default) and [Anthropic](https://docs.anthropic.com/en/docs/about-claude/models) as model providers. + +## How it works + +1. **Load skills** — sparse-checkouts `omf-devex/skills/`, parses frontmatter, filters to `pr-reviewer` surface. Raw content is stored; nothing is fetched yet. +2. **Fetch PR diff** — title, body, branch refs, closing issues (GraphQL), and changed file patches up to `max-diff-chars`. +3. **Select skills** — a fast/cheap model reads skill descriptions and changed file paths, picks which optional skills apply, and logs its reasoning. `always-skills` bypass this step entirely. +4. **Fetch context files** — only for selected skills; fetched in parallel via the GitHub App token, compressed, and capped per file at `max-context-file-chars` (defaults to 10% of the input token budget). +5. **Post review** — builds system prompt from selected skills + context, trims the diff to the remaining token budget, calls the review model, and posts or updates a PR comment. + +## Recipes + +### GitHub Copilot (default) + +No extra secrets needed beyond the standard workflow token. + +```yaml +permissions: + contents: read + pull-requests: write + issues: read + models: read + +steps: + - uses: OvertureMaps/workflows/.github/actions/overture-projection@030d1cf86ff0013daa6f41ba0073cf048ec2d494 # reusable-PRojection-workflow + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + app-private-key: ${{ secrets.OVERTURE_PROJECTION_APP_PEM }} +``` + +**Automatic defaults** (GitHub Models gpt-4.1, 8,000 token context window): + +| Input | Auto default | +| --- | --- | +| `model` | `gpt-4.1` | +| `selection-model` | `gpt-4.1-mini` | +| `max-input-tokens` | `6200` (= 8,000 − 1 500 output − 300 margin) | +| `max-output-tokens` | `1500` | + +### Anthropic + +Add `ANTHROPIC_API_KEY` as a repo or org secret. All current Claude models have a 200k token context window. + +```yaml +permissions: + contents: read + pull-requests: write + issues: read + +steps: + - uses: OvertureMaps/workflows/.github/actions/overture-projection@030d1cf86ff0013daa6f41ba0073cf048ec2d494 # reusable-PRojection-workflow + with: + model-provider: anthropic + model: claude-opus-4-6 + anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} + app-private-key: ${{ secrets.OVERTURE_PROJECTION_APP_PEM }} +``` + +Token limits (`max-input-tokens`, `max-output-tokens`) default automatically to the right values for the provider — you only need to set them if you're using a model with a non-standard context window. + +**Automatic defaults** (Anthropic 200k context window): + +| Input | Auto default | +| --- | --- | +| `model` | `claude-opus-4-6` | +| `selection-model` | `claude-haiku-4-6` | +| `max-input-tokens` | `190000` (= 200k − 4,096 output − ~6,000 margin) | +| `max-output-tokens` | `4096` | + +Note: `models: read` permission is not required when using Anthropic. + +## Inputs + +### Provider + +| Input | Default | Description | +| --- | --- | --- | +| `model-provider` | `github-models` | `github-models` or `anthropic` | +| `model` | _(default)_ | Model ID for the review. Defaults to `gpt-4.1` (github-models) or `claude-opus-4-6` (anthropic) | +| `selection-model` | _(default)_ | Model ID for skill selection. Defaults to `gpt-4.1-mini` (github-models) or `claude-haiku-4-6` (anthropic) | +| `max-input-tokens` | _(default)_ | Max input tokens. Defaults to `6200` (github-models) or `190000` (anthropic). Override only for non-standard context windows | +| `max-output-tokens` | _(default)_ | Max tokens the model may generate. Defaults to `1500` (github-models) or `4096` (anthropic) | +| `github-token` | `github.token` | Token with `pull-requests:write`, `models:read`, and read access to `omf-devex`. Not used for model calls when `model-provider` is `anthropic` | +| `anthropic-api-key` | _(empty)_ | Anthropic API key. Required when `model-provider` is `anthropic` | + +### Auth + +| Input | Default | Description | +| --- | --- | --- | +| `app-id` | `Iv23liBMB2dC9UQJ5pHL` | Overture PRojection GitHub App Client ID | +| `app-private-key` | _(empty)_ | GitHub App private key (`secrets.OVERTURE_PROJECTION_APP_PEM`). Used to generate an installation token for cross-repo context file reads. Falls back to `github-token` if omitted | + +### Behaviour + +| Input | Default | Description | +| --- | --- | --- | +| `always-skills` | `pr-review` | Comma-separated skill names included on every run, bypassing model selection | +| `devex-ref` | `main` | Git ref of `omf-devex` to load skills from | +| `max-files` | `20` | Maximum number of changed files to fetch from the GitHub API | +| `max-diff-chars` | `100000` | Fetch ceiling for diff content. The actual amount sent to the model is computed dynamically based on the remaining token budget after skills and metadata | +| `max-context-file-chars` | _(default)_ | Hard cap per individual skill context file (the cross-repo files declared via `context-files:` in skill frontmatter — not the overall prompt context). Defaults to 10% of the input token budget (~2 500 chars for github-models, ~76 000 for anthropic). Set this to enforce a tighter ceiling regardless of token budget | +| `comment-mode` | `update` | `update` edits the existing comment in place; `new` posts a fresh PR review each run | +| `comment-tag` | `overture-projection` | Hidden HTML marker used to identify the managed comment in `update` mode | +| `pr-number` | _(event)_ | PR number to review. Required for `workflow_dispatch` triggers | +| `repository` | _(current repo)_ | Target repository in `owner/repo` format | +| `dry-run` | `false` | Print the review body to the log without posting it | + +## Token budget + +The action computes the diff budget dynamically at review time: + +``` +diff budget = (max-input-tokens × 4 chars/token) − system prompt chars − user prompt preamble chars +``` + +Files are included whole (never truncated mid-diff); once the budget is exhausted, remaining files are listed in the review with a recommendation to split the PR. + +`max-input-tokens` and `max-output-tokens` default automatically based on the provider (see `scripts/lib/defaults.js`). You only need to set them explicitly when using a model with a non-standard context window: + +| Provider | `max-input-tokens` | `max-output-tokens` | Basis | +| --- | --- | --- | --- | +| `github-models` | `6200` | `1500` | 8,000 context − 1,500 output − 300 margin | +| `anthropic` | `190000` | `4096` | 200k context − 4,096 output − ~6,000 margin | + +## Skills + +Skills live in `omf-devex/skills//SKILL.md`. The folder name is the skill ID — it must match the `name` frontmatter field and is what you pass to `always-skills`. + +Only skills with `surfaces: [pr-reviewer]` (or no `surfaces` field) are loaded. Skills tagged `surfaces: [agent]` are filtered out before the selection model sees them. + +- `always-skills` bypass selection and are always included in the system prompt. +- All other `pr-reviewer` skills are passed to the selection model with their `description`; the model picks which are relevant to the PR. +- `context-files` are fetched after selection, so only selected skills pay the network cost. + +For full frontmatter field reference and authoring guidance see the [omf-devex README](../../../../README.md#skills). + +## Required workflow permissions + +```yaml +permissions: + contents: read # checkout + pull-requests: write # post/update review comment + issues: read # closingIssuesReferences GraphQL query + models: read # GitHub Models API (not needed for anthropic provider) +``` diff --git a/.github/actions/overture-projection/action.yml b/.github/actions/overture-projection/action.yml new file mode 100644 index 0000000..2799c95 --- /dev/null +++ b/.github/actions/overture-projection/action.yml @@ -0,0 +1,330 @@ +--- +name: Overture PRojection +description: > + Posts an AI-generated code review comment on a pull request. Discovers + skills from OvertureMaps/omf-devex (including any context files they + reference in other org repos), selects which apply via a fast model, + then posts a structured review. Skills are fetched, selected, and applied + as discrete steps for maintainability. + +inputs: + always-skills: + description: > + Comma-separated skill names that are always included regardless of + model selection. Defaults to 'pr-review'. + required: false + default: pr-review + selection-model: + description: >- + Model ID used for skill selection (fast/cheap). When not set, defaults to + gpt-4.1-mini for github-models or claude-haiku-4-6 for anthropic. + required: false + default: "" + devex-ref: + description: Git ref of OvertureMaps/omf-devex to fetch skills from + required: false + default: main + model: + description: >- + Model ID used for the review. When not set, defaults automatically by + provider: gpt-4.1 (github-models) or claude-opus-4-6 (anthropic). + Mirror of GITHUB_MODELS_DEFAULT_MODEL / ANTHROPIC_DEFAULT_MODEL in defaults.js. + required: false + default: "" + model-provider: + description: >- + Model provider. 'github-models' (default) uses the GitHub Models endpoint + with the github-token. 'anthropic' calls the Anthropic Messages API + directly — set anthropic-api-key and use a Claude model ID (e.g. + claude-opus-4-6). + required: false + default: "github-models" + anthropic-api-key: + description: >- + Anthropic API key. Required when model-provider is 'anthropic'. + Pass as a secret: secrets.ANTHROPIC_API_KEY. + required: false + default: "" + max-output-tokens: + description: >- + Maximum tokens the model may generate in its response. Also subtracted + from the context window when computing how much room is left for the diff. + Increase for very long reviews; decrease to save cost. + When not set, defaults automatically by provider: 1500 (github-models) or + 4096 (anthropic). Mirror of GITHUB_MODELS_MAX_OUTPUT_TOKENS / + ANTHROPIC_MAX_OUTPUT_TOKENS in defaults.js. + required: false + default: "" + max-input-tokens: + description: >- + Maximum tokens available for the full input prompt (system + user). + When not set, defaults automatically by provider: 6200 (github-models, + = 8000 context − 1500 output − 300 margin) or 190000 (anthropic, + = 200k context − 4096 output − ~6000 margin). Override only if using a + model with a different context window. Mirror of + GITHUB_MODELS_MAX_INPUT_TOKENS / ANTHROPIC_MAX_INPUT_TOKENS in defaults.js. + required: false + default: "" + github-token: + description: > + GitHub token. Must have pull-requests:write on the target repo (the + workflow repo by default, or the repository input if set), + models:read for GitHub Models API access, and read access to + OvertureMaps/omf-devex. Defaults to the workflow token. + required: false + default: "" + app-id: + description: > + GitHub App Client ID used to generate an installation token for reading + cross-repo context files referenced in skill frontmatter. + Defaults to the Overture PRojection app. + required: false + default: "Iv23liBMB2dC9UQJ5pHL" # https://github.com/organizations/OvertureMaps/settings/apps/overture-projection + app-private-key: + description: > + GitHub App private key used to generate an installation token for + cross-repo context file reads. Pass secrets.OVERTURE_PROJECTION_APP_PEM. + When omitted, falls back to github-token (cross-repo reads of private + repos will not work in that case). + required: false + default: "" + max-files: + description: Maximum number of changed files to include in the diff sent to the model + required: false + default: "20" + max-diff-chars: + description: >- + Fetch ceiling: maximum total characters of patch content to pull from the + GitHub API. Files are fetched in full (never truncated mid-diff); once + this ceiling is hit, remaining files are not fetched at all. The actual + context-window budget is computed dynamically in post-review based on the + real system-prompt size, so this is only a fetch guard against pulling + unbounded diffs from large PRs. + required: false + default: "100000" + max-context-file-chars: + description: >- + Hard cap (in characters) on each individual skill context file — the + cross-repo Markdown files declared via `context-files:` in skill + frontmatter. This does NOT affect the overall prompt context size. + When not set, the per-file limit is computed dynamically as 10% of the + input token budget (e.g. ~2 500 chars for github-models, ~76 000 chars + for anthropic). Set this to enforce a tighter ceiling regardless of the + token budget, e.g. to keep context files brief for cost reasons. + required: false + default: "" + ignore-files: + description: >- + Newline-separated list of filename patterns to exclude from the diff sent + to the model. Supports * wildcards, matched against the full file path and + the basename. Set to an empty string to disable all exclusions. + required: false + default: | + package-lock.json + yarn.lock + pnpm-lock.yaml + poetry.lock + Pipfile.lock + Gemfile.lock + composer.lock + Cargo.lock + go.sum + *.lock + *.snap + comment-mode: + description: >- + Controls how the review is posted. 'update' posts a single issue comment + the first time, then edits it in place on subsequent runs — keeps the PR + timeline clean (default). 'new' posts a fresh PR review on every run. + required: false + default: "update" + comment-tag: + description: >- + Unique identifier embedded as a hidden HTML comment () used + to find and update the existing comment when comment-mode is 'update'. + Change this if you run multiple AI review actions on the same PR to + prevent them from overwriting each other. Defaults to 'overture-projection'. + required: false + default: "overture-projection" + pr-number: + description: >- + Pull request number to review. When not set, falls back to + context.payload.pull_request.number (available on pull_request events). + Required when triggering via workflow_dispatch. + required: false + default: "" + repository: + description: >- + Repository containing the PR to review, in owner/repo format. + Defaults to the repository the workflow is running in. + When set, github-token must have pull-requests:write on the target repo. + required: false + default: "" + dry-run: + description: >- + If true, print the review body to the log instead of posting a PR comment. + Useful for local testing with act. + required: false + default: "false" + +runs: + using: composite + steps: + + # -- Step 0a: Mask secrets so they never appear in logs -------------------- + # zizmor: ignore[template-injection] -- intentional; ${{ inputs[...] }} expansion + # is required here because ::add-mask:: must see the literal value to register it. + # Using env vars would defeat the purpose — the mask command itself is what sets them. + - name: Mask secrets + shell: bash + # zizmor: ignore[template-injection] -- intentional; must see literal value to mask it + run: | + if [ -n "${{ inputs['app-private-key'] }}" ]; then + echo "::add-mask::${{ inputs['app-private-key'] }}" + fi + if [ -n "${{ inputs['anthropic-api-key'] }}" ]; then + echo "::add-mask::${{ inputs['anthropic-api-key'] }}" + fi + if [ -n "${{ inputs['github-token'] }}" ]; then + echo "::add-mask::${{ inputs['github-token'] }}" + fi + + # -- Step 0b: Resolve target repo name (bare, no owner prefix) ------------- + # 'repository' input is owner/repo; the create-github-app-token 'repositories' + # input requires bare repo names. Falls back to the current repo. + - name: Resolve target repo name + id: target_repo + shell: bash + # zizmor: ignore[template-injection] -- must read owner/repo from caller; no env var alternative + run: | + repo="${{ inputs['repository'] != '' && inputs['repository'] || github.repository }}" + echo "name=${repo##*/}" >> "$GITHUB_OUTPUT" + + # -- Step 0c: Generate an installation token for cross-repo context reads -- + # Scoped to exactly two repos: omf-devex (skills + context files) and the + # target PR repo (posting the review). Avoids the default org-wide token. + - name: Generate context token + id: context-token + if: ${{ inputs['app-private-key'] != '' }} + uses: actions/create-github-app-token@f8d387b68d61c58ab83c6c016672934102569859 # v3.0.0 + with: + app-id: ${{ inputs['app-id'] }} + private-key: ${{ inputs['app-private-key'] }} + owner: ${{ github.repository_owner }} + repositories: | + omf-devex + operating-procedures + ${{ steps.target_repo.outputs.name }} + + # -- Step 1a: Checkout skills from omf-devex -------------------------------- + # Sparse-checks out only the skills/ tree from omf-devex at a fixed ref. + - name: Checkout skills + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + repository: OvertureMaps/omf-devex + ref: ${{ inputs['devex-ref'] }} + sparse-checkout: skills + sparse-checkout-cone-mode: true + path: .omf-devex-skills + persist-credentials: false + token: ${{ steps.context-token.outputs.token || inputs['github-token'] != '' && inputs['github-token'] || github.token }} + + # -- Step 1b: Load skills from disk ---------------------------------------- + # Reads SKILL.md files, parses frontmatter, filters to pr-reviewer surface. + # Stores raw content + context-file refs only — no fetching yet. + # Context files are fetched in Step 3b, after selection, so only selected + # skills pay the network cost. + - name: Load skills + id: fetch-skills + uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0 + env: + SKILLS_DIR: ${{ format('{0}/.omf-devex-skills/skills', github.workspace) }} + with: + github-token: ${{ steps.context-token.outputs.token || inputs['github-token'] != '' && inputs['github-token'] || github.token }} + # zizmor: ignore[template-injection] -- standard GitHub Actions pattern for requiring local scripts + script: | + const script = require('${{ github.action_path }}/scripts/load-skills.js') + await script({ github, context, core }) + + # -- Step 2: Fetch PR metadata and diff ------------------------------------ + # Retrieves PR title, body, changed files, and patches in one pass. + # Writes pr.json to RUNNER_TEMP. Outputs changed-paths for the next step. + - name: Fetch PR diff + id: fetch-diff + uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0 + env: + MAX_FILES: ${{ inputs['max-files'] }} + MAX_DIFF_CHARS: ${{ inputs['max-diff-chars'] }} + IGNORE_FILES: ${{ inputs['ignore-files'] }} + PR_NUMBER: ${{ inputs['pr-number'] }} + REPOSITORY: ${{ inputs['repository'] }} + with: + github-token: ${{ steps.context-token.outputs.token || inputs['github-token'] != '' && inputs['github-token'] || github.token }} + # zizmor: ignore[template-injection] -- standard GitHub Actions pattern + script: | + const script = require('${{ github.action_path }}/scripts/fetch-diff.js') + await script({ github, context, core }) + + # -- Step 3: Select applicable skills -------------------------------------- + # Uses a fast/cheap model to decide which optional skills apply, based on + # the PR context and each skill's frontmatter description. Always-skills + # bypass this step. Outputs selected skill names as a JSON array string. + - name: Select skills + id: select-skills + uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0 + env: + AI_TOKEN: ${{ inputs['model-provider'] == 'anthropic' && inputs['anthropic-api-key'] || inputs['github-token'] != '' && inputs['github-token'] || github.token }} + MODEL_PROVIDER: ${{ inputs['model-provider'] }} + ALWAYS_SKILLS: ${{ inputs['always-skills'] }} + SELECTION_MODEL_ID: ${{ inputs['selection-model'] }} + CHANGED_PATHS: ${{ steps.fetch-diff.outputs.changed-paths }} + with: + # zizmor: ignore[template-injection] -- standard pattern + script: | + const script = require('${{ github.action_path }}/scripts/select-skills.js') + await script({ github, context, core }) + + # -- Step 3b: Fetch context files for selected skills only ----------------- + # Now that we know which skills were selected, fetch only their context-files. + # Skipped entirely if no selected skill has context-files declared. + # Per-file char limit is computed dynamically from MAX_INPUT_TOKENS (10% of + # the token budget), optionally capped by MAX_CONTEXT_FILE_CHARS_OVERRIDE. + - name: Fetch context files + id: fetch-context + uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0 + env: + CONTEXT_TOKEN: ${{ steps.context-token.outputs.token || inputs['github-token'] != '' && inputs['github-token'] || github.token }} + SELECTED_SKILLS: ${{ steps.select-skills.outputs.selected }} + MODEL_PROVIDER: ${{ inputs['model-provider'] }} + MAX_INPUT_TOKENS: ${{ inputs['max-input-tokens'] }} + MAX_CONTEXT_FILE_CHARS_OVERRIDE: ${{ inputs['max-context-file-chars'] }} + with: + github-token: ${{ steps.context-token.outputs.token || inputs['github-token'] != '' && inputs['github-token'] || github.token }} + # zizmor: ignore[template-injection] -- standard pattern + script: | + const script = require('${{ github.action_path }}/scripts/fetch-context.js') + await script({ github, context, core }) + + # -- Step 4: Compose prompt and post review -------------------------------- + # Reads skills.json and pr.json, builds the system prompt from selected + # skills (in order), calls the review model, and posts a PR review comment. + - name: Post review + uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0 + env: + AI_TOKEN: ${{ inputs['model-provider'] == 'anthropic' && inputs['anthropic-api-key'] || inputs['github-token'] != '' && inputs['github-token'] || github.token }} + MODEL_ID: ${{ inputs.model }} + MODEL_PROVIDER: ${{ inputs['model-provider'] }} + MAX_OUTPUT_TOKENS: ${{ inputs['max-output-tokens'] }} + MAX_INPUT_TOKENS: ${{ inputs['max-input-tokens'] }} + SELECTED_SKILLS: ${{ steps.select-skills.outputs.selected }} + COMMENT_MODE: ${{ inputs['comment-mode'] }} + COMMENT_TAG: ${{ inputs['comment-tag'] }} + DRY_RUN: ${{ inputs['dry-run'] }} + PR_NUMBER: ${{ inputs['pr-number'] }} + REPOSITORY: ${{ inputs['repository'] }} + with: + github-token: ${{ steps.context-token.outputs.token || inputs['github-token'] != '' && inputs['github-token'] || github.token }} + # zizmor: ignore[template-injection] -- standard pattern + script: | + const script = require('${{ github.action_path }}/scripts/post-review.js') + await script({ github, context, core }) diff --git a/.github/actions/overture-projection/scripts/fetch-context.js b/.github/actions/overture-projection/scripts/fetch-context.js new file mode 100644 index 0000000..40b436e --- /dev/null +++ b/.github/actions/overture-projection/scripts/fetch-context.js @@ -0,0 +1,120 @@ +/** + * @file fetch-context.js + * @description Step 3b — Fetch context files for selected skills only. + * + * Skills can declare `context-files` in their frontmatter — cross-repo + * Markdown files that give the review model additional grounding. This step + * fetches those files via the GitHub Contents API, compresses them, and + * truncates each one to a per-file character budget before storage. + * + * The per-file budget is computed dynamically as 10% of the input token budget + * (converted to chars at 4 chars/token), so context files scale with the + * model's context window. An optional hard cap (MAX_CONTEXT_FILE_CHARS_OVERRIDE) + * lets callers enforce a tighter ceiling regardless of the token budget. + * + * This does NOT affect the overall prompt context size — only individual + * skill context files declared via `context-files:` in skill frontmatter. + * + * Fetching is deferred to this step so that only skills that survived model + * selection incur the network cost. All fetches run in parallel. + * + * Env vars consumed: + * CONTEXT_TOKEN — installation token (or fallback github-token) for API calls + * SELECTED_SKILLS — JSON array of selected skill names from Step 3 + * MODEL_PROVIDER — 'github-models' (default) | 'anthropic' + * MAX_INPUT_TOKENS — input token budget (used to compute per-file char limit) + * MAX_CONTEXT_FILE_CHARS_OVERRIDE — optional hard cap in chars per context file + * RUNNER_TEMP — standard Actions temp dir for inter-step artefacts + * + * Outputs written: + * $RUNNER_TEMP/ai-review-context.json — Record + * + * @param {import('@actions/github-script').AsyncFunctionArguments} args + */ +module.exports = async ({ core }) => { + const fs = require('fs'); + const path = require('path'); + const { parseContextRef, processContextFile, contextFileCharBudget, groupBySkill, MAX_CONTEXT_FILE_CHARS } = require('./lib/context'); + const { getProviderDefaults, DEFAULT_PROVIDER } = require('./lib/defaults'); + + const provider = process.env.MODEL_PROVIDER || DEFAULT_PROVIDER; + const maxInputTokens = parseInt(process.env.MAX_INPUT_TOKENS) || getProviderDefaults(provider).maxInputTokens; + const override = parseInt(process.env.MAX_CONTEXT_FILE_CHARS_OVERRIDE) || 0; + const perFileLimit = contextFileCharBudget(maxInputTokens, override); + + core.info( + `📎 Context file char limit: ${perFileLimit} chars per file` + + (override > 0 ? ` (dynamic budget capped at ${override})` : ` (10% of ${maxInputTokens} input tokens)`) + ); + + const skills = JSON.parse(fs.readFileSync(path.join(process.env.RUNNER_TEMP, 'ai-review-skills.json'), 'utf-8')); + const selectedNames = new Set(JSON.parse(process.env.SELECTED_SKILLS || '[]')); + const selectedSkills = skills.filter(s => selectedNames.has(s.name)); + + /** + * @typedef {Object} ContextRef + * @property {string} skillName - The skill that declared this context-file ref. + * @property {string} ref - The raw `owner/repo:path` ref string. + */ + + /** @type {ContextRef[]} Flat list of all context-file refs across selected skills. */ + const allRefs = selectedSkills.flatMap(s => + (s.contextFiles || []).map(ref => ({ skillName: s.name, ref })) + ); + + if (allRefs.length === 0) { + core.info('⏭️ No context files to fetch for selected skills'); + fs.writeFileSync(path.join(process.env.RUNNER_TEMP, 'ai-review-context.json'), JSON.stringify({})); + return; + } + + /** @type {(ContextEntry|null)[]} */ + const fetchedContext = await Promise.all( + allRefs.map(async ({ skillName, ref }) => { + const parsed = parseContextRef(ref); + if (!parsed) { + core.warning(`⚠️ Skill '${skillName}': bad context-file ref '${ref}' (expected owner/repo:path)`); + return null; + } + const { owner: cfOwner, repo: cfRepo, filePath } = parsed; + try { + core.info(` ↓ [${skillName}] fetching ${ref}`); + const apiUrl = `https://api.github.com/repos/${cfOwner}/${cfRepo}/contents/${filePath}`; + const apiResp = await fetch(apiUrl, { + headers: { + 'Authorization': `Bearer ${process.env.CONTEXT_TOKEN}`, + 'Accept': 'application/vnd.github+json', + }, + }); + if (!apiResp.ok) throw new Error(`HTTP ${apiResp.status}`); + + const data = await apiResp.json(); + const { content, truncated } = processContextFile(data.content, perFileLimit); + + if (truncated) { + core.warning( + ` ✂️ [${skillName}] ${ref} was truncated to ${perFileLimit} chars` + + ` — consider trimming this context file to only the sections relevant to the skill` + ); + } + return { skillName, ref, content }; + } catch (err) { + core.warning(`❌ [${skillName}] could not fetch ${ref} — ${err.message}`); + return null; + } + }) + ); + + const contextBySkill = groupBySkill(fetchedContext); + + core.startGroup(`📎 Context files fetched`); + for (const [skillName, entries] of Object.entries(contextBySkill)) { + for (const e of entries) core.info(` ✅ [${skillName}] ${e.ref} (${e.content.length} chars)`); + } + core.endGroup(); + + fs.writeFileSync( + path.join(process.env.RUNNER_TEMP, 'ai-review-context.json'), + JSON.stringify(contextBySkill) + ); +}; diff --git a/.github/actions/overture-projection/scripts/fetch-diff.js b/.github/actions/overture-projection/scripts/fetch-diff.js new file mode 100644 index 0000000..681acd2 --- /dev/null +++ b/.github/actions/overture-projection/scripts/fetch-diff.js @@ -0,0 +1,115 @@ +/** + * @file fetch-diff.js + * @description Step 2 — Fetch PR metadata and diff. + * + * Makes four parallel GitHub API calls to collect everything downstream steps + * need about the pull request: + * 1. PR metadata (title, body, branch refs, author association, file count) + * 2. Changed files with patches (up to MAX_FILES) + * 3. Closing-issue references via GraphQL (best-effort; silently degraded if + * the token lacks issues:read) + * 4. Repository licence (best-effort; null if unavailable) + * + * Changed files are filtered against IGNORE_FILES patterns, then whole files + * are dropped once the total diff character budget is exhausted (in API order; + * later files may still be included if they are smaller than remaining budget). + * No patch is ever truncated mid-diff; the model only sees complete patches. + * + * Env vars consumed: + * MAX_FILES — max number of changed files to include (default 20) + * MAX_DIFF_CHARS — fetch ceiling: max chars of patch content to pull from GitHub API (default 100000) + * IGNORE_FILES — newline-separated glob patterns for files to skip + * PR_NUMBER — PR number override (falls back to event payload) + * REPOSITORY — target repo in owner/repo format (falls back to context) + * RUNNER_TEMP — standard Actions temp dir for inter-step artefacts + * + * Outputs written: + * $RUNNER_TEMP/ai-review-diff.json — PRData + * + * Step outputs set: + * changed-paths — newline-separated list of included file paths + * + * @param {import('@actions/github-script').AsyncFunctionArguments} args + */ +module.exports = async ({ github, context, core }) => { + const fs = require('fs'); + const path = require('path'); + const { buildIgnorePatterns, isIgnored, applyFileBudget } = require('./lib/diff'); + const { resolveRepo, resolvePrNumber } = require('./lib/github'); + const { DEFAULT_MAX_DIFF_CHARS } = require('./lib/defaults'); + + const maxFiles = parseInt(process.env.MAX_FILES) || 20; + const fetchCeiling = parseInt(process.env.MAX_DIFF_CHARS) || DEFAULT_MAX_DIFF_CHARS; + + const ignorePatterns = buildIgnorePatterns(process.env.IGNORE_FILES); + const { owner, repo } = resolveRepo(process.env.REPOSITORY, context.repo); + const prNumber = resolvePrNumber(context.payload.pull_request?.number, process.env.PR_NUMBER); + if (!prNumber) { + core.setFailed('No PR number available: set the pr-number input or trigger via a pull_request event.'); + return; + } + + const [prResp, filesResp, issuesResp, licenseResp] = await Promise.all([ + github.rest.pulls.get({ owner, repo, pull_number: prNumber }), + github.rest.pulls.listFiles({ owner, repo, pull_number: prNumber, per_page: maxFiles }), + + // GraphQL for closing-issue refs — best-effort, requires issues:read + github.graphql(` + query($owner: String!, $repo: String!, $number: Int!) { + repository(owner: $owner, name: $repo) { + pullRequest(number: $number) { + closingIssuesReferences(first: 10) { + nodes { number title url } + } + } + } + }`, { owner, repo, number: prNumber } + ).catch(err => { + core.warning(`⚠️ GraphQL closingIssuesReferences failed (check issues:read permission): ${err.message}`); + return null; + }), + + // Repo licence — best-effort, null if repo has no licence file + github.rest.licenses.getForRepo({ owner, repo }).catch(() => null), + ]); + + const linkedIssues = issuesResp?.repository?.pullRequest?.closingIssuesReferences?.nodes ?? []; + const includedFiles = filesResp.data.filter(f => !isIgnored(f, ignorePatterns)); + const ignoredFiles = filesResp.data.filter(f => isIgnored(f, ignorePatterns)); + + if (ignoredFiles.length > 0) { + core.info(`⏭️ Ignored ${ignoredFiles.length} file(s): ${ignoredFiles.map(f => f.filename).join(', ')}`); + } + + const { included: files, skipped: budgetSkippedFiles } = applyFileBudget(includedFiles, fetchCeiling); + core.info(`📐 Fetch ceiling: ${fetchCeiling} chars — fetched ${files.length} file(s), ceiling-skipped ${budgetSkippedFiles.length} file(s)`); + if (budgetSkippedFiles.length > 0) { + core.info(`⚠️ Ceiling-skipped (not fetched): ${budgetSkippedFiles.map(f => f.filename).join(', ')}`); + } + + const prData = { + number: prResp.data.number, + title: prResp.data.title, + body: prResp.data.body?.trim() || '', + totalFiles: prResp.data.changed_files, + headRef: prResp.data.head.ref, + baseRef: prResp.data.base.ref, + authorAssociation: prResp.data.author_association ?? null, + linkedIssues, + repoLicense: licenseResp?.data?.license?.spdx_id ?? null, + files, + budgetSkippedFiles: budgetSkippedFiles.map(f => f.filename), + }; + + fs.writeFileSync( + path.join(process.env.RUNNER_TEMP, 'ai-review-diff.json'), + JSON.stringify(prData) + ); + core.setOutput('changed-paths', files.map(f => f.filename).join('\n')); + core.info( + `📂 Fetched diff: ${files.length} of ${prData.totalFiles} file(s)` + + `${prData.repoLicense ? ` | license: ${prData.repoLicense}` : ''}` + + ` | linked issues: ${prData.linkedIssues.length}` + + ` | author: ${prData.authorAssociation ?? 'unknown'}` + ); +}; diff --git a/.github/actions/overture-projection/scripts/lib/__tests__/context.test.js b/.github/actions/overture-projection/scripts/lib/__tests__/context.test.js new file mode 100644 index 0000000..ee06bb7 --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/__tests__/context.test.js @@ -0,0 +1,224 @@ +'use strict'; + +const { describe, it } = require('node:test'); +const assert = require('node:assert/strict'); +const { parseContextRef, processContextFile, contextFileCharBudget, groupBySkill, MAX_CONTEXT_FILE_CHARS } = require('../context'); + +// --------------------------------------------------------------------------- +// parseContextRef +// --------------------------------------------------------------------------- + +describe('parseContextRef', () => { + it('parses a valid owner/repo:path ref', () => { + const result = parseContextRef('OvertureMaps/schema:docs/overview.md'); + assert.deepEqual(result, { owner: 'OvertureMaps', repo: 'schema', filePath: 'docs/overview.md' }); + }); + + it('parses a ref with a nested file path', () => { + const result = parseContextRef('acme-org/my-repo:a/b/c/file.md'); + assert.deepEqual(result, { owner: 'acme-org', repo: 'my-repo', filePath: 'a/b/c/file.md' }); + }); + + it('parses a ref with a file at the repo root', () => { + const result = parseContextRef('owner/repo:README.md'); + assert.deepEqual(result, { owner: 'owner', repo: 'repo', filePath: 'README.md' }); + }); + + it('returns null when the colon separator is missing', () => { + assert.equal(parseContextRef('OvertureMaps/schema/docs/overview.md'), null); + }); + + it('returns null when the repo portion has no slash (missing owner)', () => { + assert.equal(parseContextRef('schema:docs/overview.md'), null); + }); + + it('returns null for an empty string', () => { + assert.equal(parseContextRef(''), null); + }); + + it('returns null when owner is empty', () => { + assert.equal(parseContextRef('/repo:path.md'), null); + }); + + it('returns null when repo is empty', () => { + assert.equal(parseContextRef('owner/:path.md'), null); + }); + + it('returns null when filePath is empty', () => { + assert.equal(parseContextRef('owner/repo:'), null); + }); + + it('uses the first colon as the separator (filePath may contain colons)', () => { + const result = parseContextRef('owner/repo:path/to:file.md'); + assert.deepEqual(result, { owner: 'owner', repo: 'repo', filePath: 'path/to:file.md' }); + }); +}); + +// --------------------------------------------------------------------------- +// processContextFile +// --------------------------------------------------------------------------- + +/** Encode a string to base64 the same way GitHub's API does. */ +function toBase64(str) { + return Buffer.from(str, 'utf-8').toString('base64'); +} + +describe('processContextFile', () => { + it('decodes base64 content and returns it', () => { + const { content } = processContextFile(toBase64('Hello world')); + assert.equal(content, 'Hello world'); + }); + + it('strips YAML frontmatter via compressMarkdown', () => { + const raw = '---\nname: foo\n---\n# Body\n\nContent here.'; + const { content } = processContextFile(toBase64(raw)); + assert.ok(!content.includes('name: foo')); + assert.match(content, /Content here\./); + }); + + it('strips HTML comments via compressMarkdown', () => { + const raw = '# Title\n\n\n\nBody.'; + const { content } = processContextFile(toBase64(raw)); + assert.ok(!content.includes('hidden')); + }); + + it('returns truncated: false when content is within the budget', () => { + const { truncated } = processContextFile(toBase64('Short content.')); + assert.equal(truncated, false); + }); + + it('returns truncated: false when content exactly equals maxChars', () => { + const raw = 'x'.repeat(100); + const { content, truncated } = processContextFile(toBase64(raw), 100); + assert.equal(content.length, 100); + assert.equal(truncated, false); + }); + + it('truncates content that exceeds maxChars and sets truncated: true', () => { + const raw = 'x'.repeat(200); + const { content, truncated } = processContextFile(toBase64(raw), 100); + assert.equal(truncated, true); + assert.ok(content.startsWith('x'.repeat(100))); + }); + + it('appends a truncation notice when content is cut', () => { + const raw = 'x'.repeat(200); + const { content } = processContextFile(toBase64(raw), 100); + assert.match(content, /truncated by the review system/); + }); + + it('uses MAX_CONTEXT_FILE_CHARS as the default budget', () => { + // Content just under the default limit — should not be truncated + const raw = 'x'.repeat(MAX_CONTEXT_FILE_CHARS); + const { truncated } = processContextFile(toBase64(raw)); + assert.equal(truncated, false); + }); + + it('truncates when content exceeds MAX_CONTEXT_FILE_CHARS by default', () => { + const raw = 'x'.repeat(MAX_CONTEXT_FILE_CHARS + 1); + const { truncated } = processContextFile(toBase64(raw)); + assert.equal(truncated, true); + }); + + it('handles multi-line content with blank-line collapsing', () => { + const raw = 'Line one\n\n\n\nLine two'; + const { content } = processContextFile(toBase64(raw)); + assert.equal(content, 'Line one\n\nLine two'); + }); +}); + +// --------------------------------------------------------------------------- +// groupBySkill +// --------------------------------------------------------------------------- + +describe('groupBySkill', () => { + it('groups entries by skillName', () => { + const entries = [ + { skillName: 'a', ref: 'o/r:f1', content: 'c1' }, + { skillName: 'b', ref: 'o/r:f2', content: 'c2' }, + { skillName: 'a', ref: 'o/r:f3', content: 'c3' }, + ]; + const result = groupBySkill(entries); + assert.equal(result['a'].length, 2); + assert.equal(result['b'].length, 1); + }); + + it('filters out null entries', () => { + const entries = [ + { skillName: 'a', ref: 'o/r:f', content: 'c' }, + null, + null, + ]; + const result = groupBySkill(entries); + assert.equal(result['a'].length, 1); + assert.equal(Object.keys(result).length, 1); + }); + + it('returns an empty object for an all-null array', () => { + assert.deepEqual(groupBySkill([null, null]), {}); + }); + + it('returns an empty object for an empty array', () => { + assert.deepEqual(groupBySkill([]), {}); + }); + + it('preserves ref and content on grouped entries', () => { + const entry = { skillName: 'a', ref: 'owner/repo:path.md', content: 'text' }; + const result = groupBySkill([entry]); + assert.deepEqual(result['a'][0], entry); + }); + + it('skills with all-null fetches are absent from the result', () => { + const result = groupBySkill([null]); + assert.ok(!('a' in result)); + }); +}); + +// --------------------------------------------------------------------------- +// MAX_CONTEXT_FILE_CHARS +// --------------------------------------------------------------------------- + +describe('MAX_CONTEXT_FILE_CHARS', () => { + it('is a positive number', () => { + assert.ok(typeof MAX_CONTEXT_FILE_CHARS === 'number' && MAX_CONTEXT_FILE_CHARS > 0); + }); +}); + +// --------------------------------------------------------------------------- +// contextFileCharBudget +// --------------------------------------------------------------------------- + +describe('contextFileCharBudget', () => { + it('returns 10% of maxInputTokens × 4 chars/token', () => { + assert.equal(contextFileCharBudget(6200), Math.floor(6200 * 4 * 0.1)); + assert.equal(contextFileCharBudget(190000), Math.floor(190000 * 4 * 0.1)); + }); + + it('returns a larger budget for large token windows (anthropic)', () => { + assert.ok(contextFileCharBudget(190000) > contextFileCharBudget(6200)); + }); + + it('applies the override cap when it is smaller than the dynamic budget', () => { + // dynamic = 190000 * 4 * 0.1 = 76000; cap = 20000 + assert.equal(contextFileCharBudget(190000, 20000), 20000); + }); + + it('does not apply the cap when it exceeds the dynamic budget', () => { + // dynamic = 6200 * 4 * 0.1 = 2480; cap = 99999 + assert.equal(contextFileCharBudget(6200, 99999), Math.floor(6200 * 4 * 0.1)); + }); + + it('ignores a zero override (no cap)', () => { + assert.equal(contextFileCharBudget(6200, 0), Math.floor(6200 * 4 * 0.1)); + }); + + it('ignores a falsy override (no cap)', () => { + assert.equal(contextFileCharBudget(6200, undefined), Math.floor(6200 * 4 * 0.1)); + }); + + it('returns a positive integer', () => { + const result = contextFileCharBudget(6200); + assert.ok(Number.isInteger(result)); + assert.ok(result > 0); + }); +}); diff --git a/.github/actions/overture-projection/scripts/lib/__tests__/defaults.test.js b/.github/actions/overture-projection/scripts/lib/__tests__/defaults.test.js new file mode 100644 index 0000000..65bdd78 --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/__tests__/defaults.test.js @@ -0,0 +1,151 @@ +'use strict'; + +const { describe, it } = require('node:test'); +const assert = require('node:assert/strict'); + +const { + DEFAULT_PROVIDER, + GITHUB_MODELS_DEFAULT_MODEL, + GITHUB_MODELS_DEFAULT_SELECTION, + ANTHROPIC_DEFAULT_MODEL, + ANTHROPIC_DEFAULT_SELECTION, + GITHUB_MODELS_MAX_INPUT_TOKENS, + GITHUB_MODELS_MAX_OUTPUT_TOKENS, + ANTHROPIC_MAX_INPUT_TOKENS, + ANTHROPIC_MAX_OUTPUT_TOKENS, + DEFAULT_MAX_DIFF_CHARS, + DEFAULT_MAX_CONTEXT_FILE_CHARS, + getProviderDefaults, +} = require('../defaults'); + +describe('defaults', () => { + // ── Provider ─────────────────────────────────────────────────────────────── + + it('DEFAULT_PROVIDER is a non-empty string', () => { + assert.equal(typeof DEFAULT_PROVIDER, 'string'); + assert.ok(DEFAULT_PROVIDER.length > 0); + }); + + it('DEFAULT_PROVIDER matches the github-models getProviderDefaults key', () => { + const d = getProviderDefaults(DEFAULT_PROVIDER); + assert.equal(d.model, GITHUB_MODELS_DEFAULT_MODEL); + }); + + // ── Model IDs ────────────────────────────────────────────────────────────── + + it('exports GITHUB_MODELS_DEFAULT_MODEL as a non-empty string', () => { + assert.equal(typeof GITHUB_MODELS_DEFAULT_MODEL, 'string'); + assert.ok(GITHUB_MODELS_DEFAULT_MODEL.length > 0); + }); + + it('exports GITHUB_MODELS_DEFAULT_SELECTION as a non-empty string', () => { + assert.equal(typeof GITHUB_MODELS_DEFAULT_SELECTION, 'string'); + assert.ok(GITHUB_MODELS_DEFAULT_SELECTION.length > 0); + }); + + it('exports ANTHROPIC_DEFAULT_MODEL as a non-empty string', () => { + assert.equal(typeof ANTHROPIC_DEFAULT_MODEL, 'string'); + assert.ok(ANTHROPIC_DEFAULT_MODEL.length > 0); + }); + + it('exports ANTHROPIC_DEFAULT_SELECTION as a non-empty string', () => { + assert.equal(typeof ANTHROPIC_DEFAULT_SELECTION, 'string'); + assert.ok(ANTHROPIC_DEFAULT_SELECTION.length > 0); + }); + + it('GitHub Models defaults are distinct from Anthropic defaults', () => { + assert.notEqual(GITHUB_MODELS_DEFAULT_MODEL, ANTHROPIC_DEFAULT_MODEL); + assert.notEqual(GITHUB_MODELS_DEFAULT_SELECTION, ANTHROPIC_DEFAULT_SELECTION); + }); + + it('review model and selection model are distinct within each provider', () => { + assert.notEqual(GITHUB_MODELS_DEFAULT_MODEL, GITHUB_MODELS_DEFAULT_SELECTION); + assert.notEqual(ANTHROPIC_DEFAULT_MODEL, ANTHROPIC_DEFAULT_SELECTION); + }); + + // ── Token / budget limits ────────────────────────────────────────────────── + + it('GITHUB_MODELS_MAX_INPUT_TOKENS is a positive integer', () => { + assert.equal(typeof GITHUB_MODELS_MAX_INPUT_TOKENS, 'number'); + assert.ok(Number.isInteger(GITHUB_MODELS_MAX_INPUT_TOKENS)); + assert.ok(GITHUB_MODELS_MAX_INPUT_TOKENS > 0); + }); + + it('GITHUB_MODELS_MAX_OUTPUT_TOKENS is a positive integer', () => { + assert.equal(typeof GITHUB_MODELS_MAX_OUTPUT_TOKENS, 'number'); + assert.ok(Number.isInteger(GITHUB_MODELS_MAX_OUTPUT_TOKENS)); + assert.ok(GITHUB_MODELS_MAX_OUTPUT_TOKENS > 0); + }); + + it('ANTHROPIC_MAX_INPUT_TOKENS is a positive integer', () => { + assert.equal(typeof ANTHROPIC_MAX_INPUT_TOKENS, 'number'); + assert.ok(Number.isInteger(ANTHROPIC_MAX_INPUT_TOKENS)); + assert.ok(ANTHROPIC_MAX_INPUT_TOKENS > 0); + }); + + it('ANTHROPIC_MAX_OUTPUT_TOKENS is a positive integer', () => { + assert.equal(typeof ANTHROPIC_MAX_OUTPUT_TOKENS, 'number'); + assert.ok(Number.isInteger(ANTHROPIC_MAX_OUTPUT_TOKENS)); + assert.ok(ANTHROPIC_MAX_OUTPUT_TOKENS > 0); + }); + + it('Anthropic has a much larger input token budget than GitHub Models', () => { + assert.ok(ANTHROPIC_MAX_INPUT_TOKENS > GITHUB_MODELS_MAX_INPUT_TOKENS * 10); + }); + + it('DEFAULT_MAX_DIFF_CHARS is a positive integer', () => { + assert.equal(typeof DEFAULT_MAX_DIFF_CHARS, 'number'); + assert.ok(Number.isInteger(DEFAULT_MAX_DIFF_CHARS)); + assert.ok(DEFAULT_MAX_DIFF_CHARS > 0); + }); + + it('DEFAULT_MAX_CONTEXT_FILE_CHARS is a positive integer', () => { + assert.equal(typeof DEFAULT_MAX_CONTEXT_FILE_CHARS, 'number'); + assert.ok(Number.isInteger(DEFAULT_MAX_CONTEXT_FILE_CHARS)); + assert.ok(DEFAULT_MAX_CONTEXT_FILE_CHARS > 0); + }); + + it('DEFAULT_MAX_DIFF_CHARS is larger than DEFAULT_MAX_CONTEXT_FILE_CHARS', () => { + assert.ok(DEFAULT_MAX_DIFF_CHARS > DEFAULT_MAX_CONTEXT_FILE_CHARS); + }); + + // ── getProviderDefaults ──────────────────────────────────────────────────── + + it('returns github-models defaults for "github-models"', () => { + const d = getProviderDefaults('github-models'); + assert.equal(d.model, GITHUB_MODELS_DEFAULT_MODEL); + assert.equal(d.selectionModel, GITHUB_MODELS_DEFAULT_SELECTION); + assert.equal(d.maxInputTokens, GITHUB_MODELS_MAX_INPUT_TOKENS); + assert.equal(d.maxOutputTokens, GITHUB_MODELS_MAX_OUTPUT_TOKENS); + }); + + it('returns anthropic defaults for "anthropic"', () => { + const d = getProviderDefaults('anthropic'); + assert.equal(d.model, ANTHROPIC_DEFAULT_MODEL); + assert.equal(d.selectionModel, ANTHROPIC_DEFAULT_SELECTION); + assert.equal(d.maxInputTokens, ANTHROPIC_MAX_INPUT_TOKENS); + assert.equal(d.maxOutputTokens, ANTHROPIC_MAX_OUTPUT_TOKENS); + }); + + it('falls back to github-models defaults for an unknown provider', () => { + const d = getProviderDefaults('unknown-provider'); + assert.equal(d.model, GITHUB_MODELS_DEFAULT_MODEL); + assert.equal(d.maxInputTokens, GITHUB_MODELS_MAX_INPUT_TOKENS); + }); + + it('Anthropic defaults have significantly more input tokens than GitHub Models', () => { + const ghd = getProviderDefaults('github-models'); + const acd = getProviderDefaults('anthropic'); + assert.ok(acd.maxInputTokens > ghd.maxInputTokens * 10); + }); + + it('returned object has all required keys', () => { + for (const provider of ['github-models', 'anthropic']) { + const d = getProviderDefaults(provider); + assert.ok('model' in d, `${provider}: missing model`); + assert.ok('selectionModel' in d, `${provider}: missing selectionModel`); + assert.ok('maxInputTokens' in d, `${provider}: missing maxInputTokens`); + assert.ok('maxOutputTokens' in d, `${provider}: missing maxOutputTokens`); + } + }); +}); diff --git a/.github/actions/overture-projection/scripts/lib/__tests__/diff.test.js b/.github/actions/overture-projection/scripts/lib/__tests__/diff.test.js new file mode 100644 index 0000000..a18c85c --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/__tests__/diff.test.js @@ -0,0 +1,216 @@ +'use strict'; + +const { describe, it } = require('node:test'); +const assert = require('node:assert/strict'); +const { + diffCharBudget, + buildIgnorePatterns, + isIgnored, + applyFileBudget, +} = require('../diff'); + +// --------------------------------------------------------------------------- +// buildIgnorePatterns +// --------------------------------------------------------------------------- + +describe('buildIgnorePatterns', () => { + it('returns an empty array for an empty string', () => { + assert.deepEqual(buildIgnorePatterns(''), []); + }); + + it('returns an empty array for undefined', () => { + assert.deepEqual(buildIgnorePatterns(undefined), []); + }); + + it('compiles a single exact-match pattern', () => { + const [re] = buildIgnorePatterns('package-lock.json'); + assert.ok(re.test('package-lock.json')); + assert.ok(!re.test('package.json')); + }); + + it('compiles a wildcard pattern using *', () => { + const [re] = buildIgnorePatterns('*.lock'); + assert.ok(re.test('yarn.lock')); + assert.ok(re.test('foo.lock')); + }); + + it('escapes regex metacharacters in patterns', () => { + const [re] = buildIgnorePatterns('dist/bundle.min.js'); + assert.ok(re.test('dist/bundle.min.js')); + assert.ok(!re.test('dist/bundleXminYjs')); + }); + + it('ignores empty lines and whitespace-only lines', () => { + const patterns = buildIgnorePatterns('*.lock\n\n \npackage-lock.json'); + assert.equal(patterns.length, 2); + }); + + it('compiles multiple patterns from a newline-separated string', () => { + const patterns = buildIgnorePatterns('*.lock\npackage-lock.json\n*.min.js'); + assert.equal(patterns.length, 3); + }); +}); + +// --------------------------------------------------------------------------- +// isIgnored +// --------------------------------------------------------------------------- + +describe('isIgnored', () => { + it('returns false when no patterns are provided', () => { + assert.equal(isIgnored({ filename: 'src/index.js' }, []), false); + }); + + it('matches on the full path', () => { + const patterns = buildIgnorePatterns('dist/bundle.js'); + assert.equal(isIgnored({ filename: 'dist/bundle.js' }, patterns), true); + }); + + it('matches on the basename (allows *.lock to catch subdir/yarn.lock)', () => { + const patterns = buildIgnorePatterns('*.lock'); + assert.equal(isIgnored({ filename: 'subdir/yarn.lock' }, patterns), true); + }); + + it('does not ignore a file that has no matching pattern', () => { + const patterns = buildIgnorePatterns('*.lock'); + assert.equal(isIgnored({ filename: 'src/index.js' }, patterns), false); + }); + + it('matches an exact basename in a subdirectory', () => { + const patterns = buildIgnorePatterns('package-lock.json'); + assert.equal(isIgnored({ filename: 'frontend/package-lock.json' }, patterns), true); + }); +}); + +// --------------------------------------------------------------------------- +// applyFileBudget +// --------------------------------------------------------------------------- + +/** Build a minimal file fixture. */ +function file(filename, patchLen) { + return { filename, status: 'modified', additions: 1, deletions: 0, patch: 'x'.repeat(patchLen) }; +} + +/** Build a binary (no-patch) file fixture. */ +function binaryFile(filename) { + return { filename, status: 'modified', additions: 0, deletions: 0, patch: undefined }; +} + +describe('applyFileBudget', () => { + it('includes all files when total patch length is within budget', () => { + const files = [file('a.js', 200), file('b.js', 200)]; + const { included, skipped } = applyFileBudget(files, 500); + assert.equal(included.length, 2); + assert.equal(skipped.length, 0); + }); + + it('includes all files when total patch length exactly equals budget', () => { + const files = [file('a.js', 250), file('b.js', 250)]; + const { included, skipped } = applyFileBudget(files, 500); + assert.equal(included.length, 2); + assert.equal(skipped.length, 0); + }); + + it('drops the file that would exceed the budget', () => { + const files = [file('a.js', 400), file('b.js', 400)]; + const { included, skipped } = applyFileBudget(files, 500); + assert.equal(included.length, 1); + assert.equal(included[0].filename, 'a.js'); + assert.equal(skipped.length, 1); + assert.equal(skipped[0].filename, 'b.js'); + }); + + it('never truncates a patch mid-diff — included patches are always complete', () => { + const patch = 'x'.repeat(400); + const files = [{ filename: 'a.js', status: 'modified', additions: 1, deletions: 0, patch }]; + const { included } = applyFileBudget(files, 500); + assert.equal(included[0].patch, patch); + }); + + it('continues including files after a skip when they fit in remaining budget', () => { + // a.js (100) fits (used=100); b.js (400) does not fit (100+400 > 300, skipped); + // c.js (50) fits in remaining 200 (100+50=150 ≤ 300, included) + const files = [file('a.js', 100), file('b.js', 400), file('c.js', 50)]; + const { included, skipped } = applyFileBudget(files, 300); + assert.equal(included.length, 2); + assert.deepEqual(included.map(f => f.filename), ['a.js', 'c.js']); + assert.equal(skipped.length, 1); + assert.equal(skipped[0].filename, 'b.js'); + }); + + it('skips multiple files when budget runs out early', () => { + const files = [file('a.js', 100), file('b.js', 400), file('c.js', 400)]; + const { included, skipped } = applyFileBudget(files, 300); + assert.equal(included.length, 1); + assert.equal(included[0].filename, 'a.js'); + assert.equal(skipped.length, 2); + assert.deepEqual(skipped.map(f => f.filename), ['b.js', 'c.js']); + }); + + it('replaces absent patches with a placeholder string for binary files', () => { + const files = [binaryFile('image.png')]; + const { included } = applyFileBudget(files, 500); + assert.equal(included[0].patch, '(binary or no textual diff)'); + }); + + it('counts binary file patch length as 0 (does not drain budget)', () => { + const files = [binaryFile('image.png'), file('a.js', 400)]; + const { included } = applyFileBudget(files, 400); + assert.equal(included.length, 2); + }); + + it('returns empty included and skipped arrays for empty input', () => { + const { included, skipped } = applyFileBudget([], 1000); + assert.deepEqual(included, []); + assert.deepEqual(skipped, []); + }); + + it('preserves filename, status, additions, deletions on included files', () => { + const f = { filename: 'src/foo.js', status: 'added', additions: 5, deletions: 0, patch: 'abc' }; + const { included } = applyFileBudget([f], 1000); + assert.equal(included[0].filename, 'src/foo.js'); + assert.equal(included[0].status, 'added'); + assert.equal(included[0].additions, 5); + assert.equal(included[0].deletions, 0); + }); + + it('skipped entries are the original raw file objects', () => { + // a.js (100) fits; b.js (600) does not + const files = [file('a.js', 100), file('b.js', 600)]; + const { skipped } = applyFileBudget(files, 500); + assert.equal(skipped[0].filename, 'b.js'); + }); +}); + +// --------------------------------------------------------------------------- +// diffCharBudget +// --------------------------------------------------------------------------- + +describe('diffCharBudget', () => { + it('returns maxInputTokens * 4 when non-diff chars is 0', () => { + assert.equal(diffCharBudget(0, 6200), 6200 * 4); + }); + + it('subtracts non-diff char cost from the total budget', () => { + assert.equal(diffCharBudget(4000, 6200), 6200 * 4 - 4000); + }); + + it('returns 0 when non-diff chars exactly equals the full budget', () => { + assert.equal(diffCharBudget(6200 * 4, 6200), 0); + }); + + it('clamps to 0 when non-diff chars exceeds the full budget', () => { + assert.equal(diffCharBudget(6200 * 4 + 1000, 6200), 0); + }); + + it('never returns a negative value', () => { + assert.ok(diffCharBudget(999999, 6200) >= 0); + }); + + it('scales correctly with a large maxInputTokens (e.g. Claude 200k window)', () => { + assert.equal(diffCharBudget(0, 190000), 190000 * 4); + }); + + it('scales correctly with a small maxInputTokens', () => { + assert.equal(diffCharBudget(0, 1000), 4000); + }); +}); diff --git a/.github/actions/overture-projection/scripts/lib/__tests__/github.test.js b/.github/actions/overture-projection/scripts/lib/__tests__/github.test.js new file mode 100644 index 0000000..29fadbd --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/__tests__/github.test.js @@ -0,0 +1,73 @@ +'use strict'; + +const { describe, it } = require('node:test'); +const assert = require('node:assert/strict'); +const { resolveRepo, resolvePrNumber } = require('../github'); + +// --------------------------------------------------------------------------- +// resolveRepo +// --------------------------------------------------------------------------- + +describe('resolveRepo', () => { + const contextRepo = { owner: 'OvertureMaps', repo: 'omf-devex' }; + + it('returns the parsed env repo when REPOSITORY env is set', () => { + const result = resolveRepo('OvertureMaps/overture-tiles', contextRepo); + assert.deepEqual(result, { owner: 'OvertureMaps', repo: 'overture-tiles' }); + }); + + it('falls back to context.repo when REPOSITORY env is undefined', () => { + const result = resolveRepo(undefined, contextRepo); + assert.deepEqual(result, { owner: 'OvertureMaps', repo: 'omf-devex' }); + }); + + it('falls back to context.repo when REPOSITORY env is empty string', () => { + const result = resolveRepo('', contextRepo); + assert.deepEqual(result, { owner: 'OvertureMaps', repo: 'omf-devex' }); + }); + + it('correctly splits owner and repo from REPOSITORY env', () => { + const { owner, repo } = resolveRepo('acme-org/my-repo', contextRepo); + assert.equal(owner, 'acme-org'); + assert.equal(repo, 'my-repo'); + }); +}); + +// --------------------------------------------------------------------------- +// resolvePrNumber +// --------------------------------------------------------------------------- + +describe('resolvePrNumber', () => { + it('returns the payload PR number when it is a positive integer', () => { + assert.equal(resolvePrNumber(42, undefined), 42); + }); + + it('parses the PR_NUMBER env var when payload is undefined', () => { + assert.equal(resolvePrNumber(undefined, '7'), 7); + }); + + it('returns null when neither source yields a valid number', () => { + assert.equal(resolvePrNumber(undefined, ''), null); + }); + + it('returns null for a non-numeric PR_NUMBER env var', () => { + assert.equal(resolvePrNumber(undefined, 'abc'), null); + }); + + it('returns null for a zero PR_NUMBER env var (not a valid PR)', () => { + assert.equal(resolvePrNumber(undefined, '0'), null); + }); + + it('returns null for a negative PR_NUMBER env var', () => { + assert.equal(resolvePrNumber(undefined, '-5'), null); + }); + + it('prefers the payload number over the env var when both are present', () => { + assert.equal(resolvePrNumber(10, '99'), 10); + }); + + it('returns null when payload is 0 and env var is absent', () => { + // 0 is falsy — treated same as undefined + assert.equal(resolvePrNumber(0, undefined), null); + }); +}); diff --git a/.github/actions/overture-projection/scripts/lib/__tests__/markdown.test.js b/.github/actions/overture-projection/scripts/lib/__tests__/markdown.test.js new file mode 100644 index 0000000..2b20a79 --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/__tests__/markdown.test.js @@ -0,0 +1,57 @@ +'use strict'; + +const { describe, it } = require('node:test'); +const assert = require('node:assert/strict'); +const { compressMarkdown } = require('../markdown'); + +describe('compressMarkdown', () => { + it('strips YAML frontmatter block', () => { + const input = '---\ntitle: foo\nauthor: bar\n---\n# Hello\n\nWorld'; + assert.equal(compressMarkdown(input), '# Hello\n\nWorld'); + }); + + it('leaves content unchanged when no frontmatter is present', () => { + const input = '# Hello\n\nWorld'; + assert.equal(compressMarkdown(input), '# Hello\n\nWorld'); + }); + + it('strips HTML comments', () => { + const input = '# Title\n\n\n\nBody text.'; + assert.equal(compressMarkdown(input), '# Title\n\nBody text.'); + }); + + it('strips multi-line HTML comments', () => { + const input = '# Title\n\n\n\nBody.'; + assert.equal(compressMarkdown(input), '# Title\n\nBody.'); + }); + + it('collapses 3+ consecutive blank lines to a single blank line', () => { + const input = 'Line one\n\n\n\nLine two'; + assert.equal(compressMarkdown(input), 'Line one\n\nLine two'); + }); + + it('preserves a single blank line (paragraph break)', () => { + const input = 'Para one\n\nPara two'; + assert.equal(compressMarkdown(input), 'Para one\n\nPara two'); + }); + + it('trims leading and trailing whitespace', () => { + const input = '\n\n# Hello\n\nWorld\n\n'; + assert.equal(compressMarkdown(input), '# Hello\n\nWorld'); + }); + + it('trims trailing whitespace from individual lines', () => { + const input = 'Line one \nLine two '; + assert.equal(compressMarkdown(input), 'Line one\nLine two'); + }); + + it('handles frontmatter + comment + excess blank lines together', () => { + const input = '---\nname: foo\n---\n\n\n\n\n\n# Body\n\nContent'; + assert.equal(compressMarkdown(input), '# Body\n\nContent'); + }); + + it('returns empty string for a frontmatter-only file', () => { + const input = '---\nname: foo\n---\n'; + assert.equal(compressMarkdown(input), ''); + }); +}); diff --git a/.github/actions/overture-projection/scripts/lib/__tests__/models.test.js b/.github/actions/overture-projection/scripts/lib/__tests__/models.test.js new file mode 100644 index 0000000..1ed1d24 --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/__tests__/models.test.js @@ -0,0 +1,336 @@ +'use strict'; + +const { describe, it, before, after } = require('node:test'); +const assert = require('node:assert/strict'); +const { YELLOW, BLUE, RESET, logRateLimit, callChatCompletion } = require('../models'); + +// --------------------------------------------------------------------------- +// ANSI constants +// --------------------------------------------------------------------------- + +describe('ANSI constants', () => { + it('YELLOW is the correct escape sequence', () => { + assert.equal(YELLOW, '\x1b[33m'); + }); + + it('BLUE is the correct escape sequence', () => { + assert.equal(BLUE, '\x1b[34m'); + }); + + it('RESET is the correct escape sequence', () => { + assert.equal(RESET, '\x1b[0m'); + }); +}); + +// --------------------------------------------------------------------------- +// logRateLimit +// --------------------------------------------------------------------------- + +/** Build a minimal fake Response with the provided headers. */ +function fakeResp(headers = {}) { + return { + headers: { + get: (name) => headers[name] ?? null, + }, + }; +} + +describe('logRateLimit', () => { + it('no-ops when neither remaining-requests nor remaining-tokens header is present', () => { + const calls = []; + const core = { info: (m) => calls.push(['info', m]), warning: (m) => calls.push(['warning', m]) }; + logRateLimit(fakeResp({}), 'test', core); + assert.equal(calls.length, 0); + }); + + it('calls core.info for normal (non-warning) levels', () => { + const calls = []; + const core = { info: (m) => calls.push(['info', m]), warning: (m) => calls.push(['warning', m]) }; + logRateLimit( + fakeResp({ + 'x-ratelimit-remaining-requests': '100', + 'x-ratelimit-remaining-tokens': '50000', + 'x-ratelimit-limit-tokens': '100000', + }), + 'selection', + core, + ); + assert.equal(calls.length, 1); + assert.equal(calls[0][0], 'info'); + assert.match(calls[0][1], /selection rate limit/); + }); + + it('calls core.warning when remaining-requests ≤ 10', () => { + const calls = []; + const core = { info: (m) => calls.push(['info', m]), warning: (m) => calls.push(['warning', m]) }; + logRateLimit( + fakeResp({ + 'x-ratelimit-remaining-requests': '5', + 'x-ratelimit-remaining-tokens': '50000', + 'x-ratelimit-limit-tokens': '100000', + }), + 'review', + core, + ); + assert.equal(calls.length, 1); + assert.equal(calls[0][0], 'warning'); + }); + + it('calls core.warning when remaining-tokens ≤ 1000', () => { + const calls = []; + const core = { info: (m) => calls.push(['info', m]), warning: (m) => calls.push(['warning', m]) }; + logRateLimit( + fakeResp({ + 'x-ratelimit-remaining-requests': '500', + 'x-ratelimit-remaining-tokens': '999', + 'x-ratelimit-limit-tokens': '100000', + }), + 'review', + core, + ); + assert.equal(calls.length, 1); + assert.equal(calls[0][0], 'warning'); + }); + + it('calls core.warning at the exact boundary (remaining-requests = 10)', () => { + const calls = []; + const core = { info: (m) => calls.push(['info', m]), warning: (m) => calls.push(['warning', m]) }; + logRateLimit( + fakeResp({ + 'x-ratelimit-remaining-requests': '10', + 'x-ratelimit-remaining-tokens': '50000', + 'x-ratelimit-limit-tokens': '100000', + }), + 'review', + core, + ); + assert.equal(calls[0][0], 'warning'); + }); + + it('calls core.warning at the exact boundary (remaining-tokens = 1000)', () => { + const calls = []; + const core = { info: (m) => calls.push(['info', m]), warning: (m) => calls.push(['warning', m]) }; + logRateLimit( + fakeResp({ + 'x-ratelimit-remaining-requests': '500', + 'x-ratelimit-remaining-tokens': '1000', + 'x-ratelimit-limit-tokens': '100000', + }), + 'review', + core, + ); + assert.equal(calls[0][0], 'warning'); + }); + + it('includes retry-after in the message when header is present', () => { + const calls = []; + const core = { info: (m) => calls.push(['info', m]), warning: (m) => calls.push(['warning', m]) }; + logRateLimit( + fakeResp({ + 'x-ratelimit-remaining-requests': '100', + 'x-ratelimit-remaining-tokens': '50000', + 'x-ratelimit-limit-tokens': '100000', + 'retry-after': '30', + }), + 'review', + core, + ); + assert.match(calls[0][1], /retry-after: 30s/); + }); + + it('works when only remaining-requests is present (remaining-tokens null)', () => { + const calls = []; + const core = { info: (m) => calls.push(['info', m]), warning: (m) => calls.push(['warning', m]) }; + logRateLimit( + fakeResp({ 'x-ratelimit-remaining-requests': '200' }), + 'review', + core, + ); + assert.equal(calls.length, 1); + }); + + it('works when only remaining-tokens is present (remaining-requests null)', () => { + const calls = []; + const core = { info: (m) => calls.push(['info', m]), warning: (m) => calls.push(['warning', m]) }; + logRateLimit( + fakeResp({ 'x-ratelimit-remaining-tokens': '5000', 'x-ratelimit-limit-tokens': '100000' }), + 'review', + core, + ); + assert.equal(calls.length, 1); + }); +}); + +// --------------------------------------------------------------------------- +// callChatCompletion +// --------------------------------------------------------------------------- + +/** + * Builds a minimal fake fetch Response. + * @param {number} status + * @param {object} body - JSON body to return. + * @param {object} [hdrs] - Headers map. + */ +function fakeFetchResp(status, body, hdrs = {}) { + return { + ok: status >= 200 && status < 300, + status, + text: async () => JSON.stringify(body), + json: async () => body, + headers: { get: (n) => hdrs[n] ?? null }, + }; +} + +/** GitHub Models success response fixture. */ +const GH_SUCCESS = { + choices: [{ message: { content: 'Review text' }, finish_reason: 'stop' }], + usage: { prompt_tokens: 100, completion_tokens: 50, total_tokens: 150 }, +}; + +/** Anthropic success response fixture. */ +const ANTHROPIC_SUCCESS = { + content: [{ type: 'text', text: 'Anthropic review' }], + stop_reason: 'end_turn', + usage: { input_tokens: 120, output_tokens: 60 }, +}; + +describe('callChatCompletion', () => { + let originalFetch; + + before(() => { originalFetch = globalThis.fetch; }); + after(() => { globalThis.fetch = originalFetch; }); + + // ── GitHub Models ────────────────────────────────────────────────────────── + + it('calls the GitHub Models endpoint by default (no provider set)', async () => { + let calledUrl; + globalThis.fetch = async (url) => { calledUrl = url; return fakeFetchResp(200, GH_SUCCESS); }; + await callChatCompletion({ token: 'tok', model: 'm', messages: [], maxTokens: 100 }); + assert.match(calledUrl, /models\.inference\.ai\.azure\.com/); + }); + + it('calls the GitHub Models endpoint when provider is github-models', async () => { + let calledUrl; + globalThis.fetch = async (url) => { calledUrl = url; return fakeFetchResp(200, GH_SUCCESS); }; + await callChatCompletion({ provider: 'github-models', token: 'tok', model: 'm', messages: [], maxTokens: 100 }); + assert.match(calledUrl, /models\.inference\.ai\.azure\.com/); + }); + + it('sends Bearer auth to GitHub Models', async () => { + let calledHeaders; + globalThis.fetch = async (_url, opts) => { + calledHeaders = opts.headers; + return fakeFetchResp(200, GH_SUCCESS); + }; + await callChatCompletion({ provider: 'github-models', token: 'mytoken', model: 'm', messages: [], maxTokens: 100 }); + assert.equal(calledHeaders['Authorization'], 'Bearer mytoken'); + }); + + it('normalises GitHub Models response to ChatResult shape', async () => { + globalThis.fetch = async () => fakeFetchResp(200, GH_SUCCESS); + const result = await callChatCompletion({ token: 'tok', model: 'm', messages: [], maxTokens: 100 }); + assert.equal(result.text, 'Review text'); + assert.equal(result.finishReason, 'stop'); + assert.equal(result.usage.input, 100); + assert.equal(result.usage.output, 50); + assert.equal(result.usage.total, 150); + }); + + it('includes response_format when jsonMode is true (GitHub Models)', async () => { + let sentBody; + globalThis.fetch = async (_url, opts) => { sentBody = JSON.parse(opts.body); return fakeFetchResp(200, GH_SUCCESS); }; + await callChatCompletion({ token: 'tok', model: 'm', messages: [], maxTokens: 100, jsonMode: true }); + assert.deepEqual(sentBody.response_format, { type: 'json_object' }); + }); + + it('omits response_format when jsonMode is false (GitHub Models)', async () => { + let sentBody; + globalThis.fetch = async (_url, opts) => { sentBody = JSON.parse(opts.body); return fakeFetchResp(200, GH_SUCCESS); }; + await callChatCompletion({ token: 'tok', model: 'm', messages: [], maxTokens: 100, jsonMode: false }); + assert.ok(!('response_format' in sentBody)); + }); + + it('throws an Error with status and body on non-2xx GitHub Models response', async () => { + globalThis.fetch = async () => fakeFetchResp(413, { error: { message: 'too large' } }); + await assert.rejects( + () => callChatCompletion({ token: 'tok', model: 'm', messages: [], maxTokens: 100 }), + (err) => { + assert.equal(err.status, 413); + assert.match(err.message, /413/); + return true; + }, + ); + }); + + // ── Anthropic ───────────────────────────────────────────────────────────── + + it('calls the Anthropic endpoint when provider is anthropic', async () => { + let calledUrl; + globalThis.fetch = async (url) => { calledUrl = url; return fakeFetchResp(200, ANTHROPIC_SUCCESS); }; + await callChatCompletion({ provider: 'anthropic', token: 'sk-key', model: 'claude-opus-4-5', messages: [{ role: 'user', content: 'Hi' }], maxTokens: 100 }); + assert.match(calledUrl, /api\.anthropic\.com/); + }); + + it('sends x-api-key and anthropic-version headers', async () => { + let calledHeaders; + globalThis.fetch = async (_url, opts) => { + calledHeaders = opts.headers; + return fakeFetchResp(200, ANTHROPIC_SUCCESS); + }; + await callChatCompletion({ provider: 'anthropic', token: 'sk-abc', model: 'claude-opus-4-5', messages: [{ role: 'user', content: 'Hi' }], maxTokens: 100 }); + assert.equal(calledHeaders['x-api-key'], 'sk-abc'); + assert.ok(calledHeaders['anthropic-version']); + }); + + it('normalises Anthropic response to ChatResult shape', async () => { + globalThis.fetch = async () => fakeFetchResp(200, ANTHROPIC_SUCCESS); + const result = await callChatCompletion({ provider: 'anthropic', token: 'k', model: 'm', messages: [{ role: 'user', content: 'Hi' }], maxTokens: 100 }); + assert.equal(result.text, 'Anthropic review'); + assert.equal(result.finishReason, 'stop'); // end_turn → stop + assert.equal(result.usage.input, 120); + assert.equal(result.usage.output, 60); + assert.equal(result.usage.total, 180); + }); + + it('maps max_tokens stop_reason to length finish reason', async () => { + globalThis.fetch = async () => fakeFetchResp(200, { ...ANTHROPIC_SUCCESS, stop_reason: 'max_tokens' }); + const result = await callChatCompletion({ provider: 'anthropic', token: 'k', model: 'm', messages: [{ role: 'user', content: 'Hi' }], maxTokens: 100 }); + assert.equal(result.finishReason, 'length'); + }); + + it('extracts system messages into the Anthropic system field', async () => { + let sentBody; + globalThis.fetch = async (_url, opts) => { sentBody = JSON.parse(opts.body); return fakeFetchResp(200, ANTHROPIC_SUCCESS); }; + await callChatCompletion({ + provider: 'anthropic', token: 'k', model: 'm', maxTokens: 100, + messages: [ + { role: 'system', content: 'Be helpful.' }, + { role: 'user', content: 'Review this.' }, + ], + }); + assert.equal(sentBody.system, 'Be helpful.'); + assert.equal(sentBody.messages.length, 1); + assert.equal(sentBody.messages[0].role, 'user'); + }); + + it('appends JSON instruction to system prompt when jsonMode is true (Anthropic)', async () => { + let sentBody; + globalThis.fetch = async (_url, opts) => { sentBody = JSON.parse(opts.body); return fakeFetchResp(200, ANTHROPIC_SUCCESS); }; + await callChatCompletion({ + provider: 'anthropic', token: 'k', model: 'm', maxTokens: 100, jsonMode: true, + messages: [{ role: 'user', content: 'Hi' }], + }); + assert.match(sentBody.system, /Respond with valid JSON only/); + }); + + it('throws an Error with status and body on non-2xx Anthropic response', async () => { + globalThis.fetch = async () => fakeFetchResp(401, { error: { message: 'Unauthorized' } }); + await assert.rejects( + () => callChatCompletion({ provider: 'anthropic', token: 'bad', model: 'm', messages: [], maxTokens: 100 }), + (err) => { + assert.equal(err.status, 401); + return true; + }, + ); + }); +}); diff --git a/.github/actions/overture-projection/scripts/lib/__tests__/prompt.test.js b/.github/actions/overture-projection/scripts/lib/__tests__/prompt.test.js new file mode 100644 index 0000000..7a56949 --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/__tests__/prompt.test.js @@ -0,0 +1,397 @@ +'use strict'; + +const { describe, it } = require('node:test'); +const assert = require('node:assert/strict'); +const { + buildSystemPrompt, + buildUserPromptPreamble, + buildUserPrompt, + isTestFile, + DOCS_EXTENSIONS, +} = require('../prompt'); + +// --------------------------------------------------------------------------- +// DOCS_EXTENSIONS +// --------------------------------------------------------------------------- + +describe('DOCS_EXTENSIONS', () => { + it('is a Set', () => { + assert.ok(DOCS_EXTENSIONS instanceof Set); + }); + + it('contains .md', () => { + assert.ok(DOCS_EXTENSIONS.has('.md')); + }); + + it('contains .mdx', () => { + assert.ok(DOCS_EXTENSIONS.has('.mdx')); + }); + + it('contains .rst', () => { + assert.ok(DOCS_EXTENSIONS.has('.rst')); + }); + + it('contains .txt', () => { + assert.ok(DOCS_EXTENSIONS.has('.txt')); + }); +}); + +// --------------------------------------------------------------------------- +// isTestFile +// --------------------------------------------------------------------------- + +describe('isTestFile', () => { + // directory-based matches + it('matches files inside a tests/ directory', () => { + assert.ok(isTestFile('tests/utils.test.js')); + }); + + it('matches files inside a test/ directory', () => { + assert.ok(isTestFile('test/unit/parser.js')); + }); + + it('matches files inside a __tests__/ directory', () => { + assert.ok(isTestFile('src/__tests__/foo.js')); + }); + + it('matches files inside a spec/ directory', () => { + assert.ok(isTestFile('spec/models/user_spec.rb')); + }); + + // suffix-based matches + it('matches *.test.js suffix', () => { + assert.ok(isTestFile('src/utils.test.js')); + }); + + it('matches *.spec.ts suffix', () => { + assert.ok(isTestFile('src/components/Button.spec.ts')); + }); + + it('matches *_test.go suffix', () => { + assert.ok(isTestFile('pkg/parser/parser_test.go')); + }); + + it('matches *-test.js suffix', () => { + assert.ok(isTestFile('lib/helper-test.js')); + }); + + // Python conventions + it('matches test_foo.py at the root', () => { + assert.ok(isTestFile('test_parser.py')); + }); + + it('matches test_foo.py in a subdirectory', () => { + assert.ok(isTestFile('mypackage/test_utils.py')); + }); + + // non-test files + it('returns false for a regular source file', () => { + assert.equal(isTestFile('src/index.js'), false); + }); + + it('returns false for a file named "context.ts"', () => { + assert.equal(isTestFile('src/context.ts'), false); + }); + + it('returns false for a docs file', () => { + assert.equal(isTestFile('docs/README.md'), false); + }); +}); + +// --------------------------------------------------------------------------- +// buildSystemPrompt +// --------------------------------------------------------------------------- + +describe('buildSystemPrompt', () => { + it('returns empty string when skills array is empty', () => { + assert.equal(buildSystemPrompt([], {}), ''); + }); + + it('wraps a single skill in a comment header', () => { + const skills = [{ name: 'pr-review', raw: '---\nname: pr-review\n---\nReview all PRs.' }]; + const result = buildSystemPrompt(skills, {}); + assert.match(result, //); + assert.match(result, /Review all PRs\./); + }); + + it('strips frontmatter from the skill body', () => { + const skills = [{ name: 'pr-review', raw: '---\nname: pr-review\n---\nBody text.' }]; + const result = buildSystemPrompt(skills, {}); + assert.ok(!result.includes('name: pr-review')); + assert.match(result, /Body text\./); + }); + + it('separates multiple skills with --- dividers', () => { + const skills = [ + { name: 'skill-a', raw: '# Skill A' }, + { name: 'skill-b', raw: '# Skill B' }, + ]; + const result = buildSystemPrompt(skills, {}); + assert.match(result, /---/); + assert.match(result, //); + assert.match(result, //); + }); + + it('appends a ## Context Files section when context entries are provided', () => { + const skills = [{ name: 'my-skill', raw: 'Skill body.' }]; + const contextBySkill = { + 'my-skill': [ + { ref: 'owner/repo:path/to/file.md', content: 'File content here.' }, + ], + }; + const result = buildSystemPrompt(skills, contextBySkill); + assert.match(result, /## Context Files/); + assert.match(result, /\[owner\/repo:path\/to\/file\.md\]/); + assert.match(result, /File content here\./); + }); + + it('does not include ## Context Files when context is empty for a skill', () => { + const skills = [{ name: 'my-skill', raw: 'Skill body.' }]; + const result = buildSystemPrompt(skills, { 'my-skill': [] }); + assert.ok(!result.includes('## Context Files')); + }); + + it('does not include ## Context Files when skill has no entry in contextBySkill', () => { + const skills = [{ name: 'my-skill', raw: 'Skill body.' }]; + const result = buildSystemPrompt(skills, {}); + assert.ok(!result.includes('## Context Files')); + }); + + it('preserves skill order in the output', () => { + const skills = [ + { name: 'first', raw: 'First body.' }, + { name: 'second', raw: 'Second body.' }, + ]; + const result = buildSystemPrompt(skills, {}); + assert.ok(result.indexOf('') < result.indexOf('')); + }); +}); + +// --------------------------------------------------------------------------- +// buildUserPrompt +// --------------------------------------------------------------------------- + +/** Minimal valid PRData fixture. */ +function makePRData(overrides = {}) { + return { + number: 1, + title: 'Test PR', + body: 'This is the description.', + totalFiles: 1, + headRef: 'feature/foo', + baseRef: 'main', + authorAssociation: 'CONTRIBUTOR', + linkedIssues: [], + repoLicense: 'Apache-2.0', + budgetSkippedFiles: [], + files: [ + { filename: 'src/index.js', status: 'modified', additions: 5, deletions: 2, patch: '@@ -1 +1 @@\n-old\n+new' }, + ], + ...overrides, + }; +} + +describe('buildUserPrompt', () => { + it('includes the PR title in the output', () => { + const result = buildUserPrompt(makePRData()); + assert.match(result, /Test PR/); + }); + + it('includes branch names', () => { + const result = buildUserPrompt(makePRData()); + assert.match(result, /feature\/foo/); + assert.match(result, /main/); + }); + + it('includes the license when present', () => { + const result = buildUserPrompt(makePRData()); + assert.match(result, /Apache-2\.0/); + }); + + it('shows "License: unknown" when repoLicense is null', () => { + const result = buildUserPrompt(makePRData({ repoLicense: null })); + assert.match(result, /License: unknown/); + }); + + it('shows description: ✅ when body is present', () => { + const result = buildUserPrompt(makePRData()); + assert.match(result, /Description: ✅/); + }); + + it('shows description: ❌ missing when body is empty string', () => { + const result = buildUserPrompt(makePRData({ body: '' })); + assert.match(result, /Description: ❌ missing/); + }); + + it('shows PR type: code for a code-only PR', () => { + const result = buildUserPrompt(makePRData()); + assert.match(result, /PR type: code/); + }); + + it('shows PR type: docs-only when all files are docs extensions', () => { + const result = buildUserPrompt(makePRData({ + files: [ + { filename: 'docs/README.md', status: 'modified', additions: 1, deletions: 0, patch: '+line' }, + ], + })); + assert.match(result, /PR type: docs-only/); + }); + + it('suppresses the tests note for docs-only PRs', () => { + const result = buildUserPrompt(makePRData({ + files: [ + { filename: 'docs/README.md', status: 'modified', additions: 1, deletions: 0, patch: '+line' }, + ], + })); + assert.ok(!result.includes('Tests:')); + }); + + it('shows Tests: ✅ when a test file is present', () => { + const result = buildUserPrompt(makePRData({ + files: [ + { filename: 'src/index.js', status: 'modified', additions: 1, deletions: 0, patch: '+x' }, + { filename: 'tests/index.test.js', status: 'added', additions: 10, deletions: 0, patch: '+test' }, + ], + })); + assert.match(result, /Tests: ✅/); + }); + + it('shows Tests: ❌ none in diff when no test files are present', () => { + const result = buildUserPrompt(makePRData()); + assert.match(result, /Tests: ❌ none in diff/); + }); + + it('includes author association', () => { + const result = buildUserPrompt(makePRData()); + assert.match(result, /Author: CONTRIBUTOR/); + }); + + it('omits the Author note when authorAssociation is null', () => { + const result = buildUserPrompt(makePRData({ authorAssociation: null })); + assert.ok(!result.includes('Author:')); + }); + + it('shows linked issues when present', () => { + const result = buildUserPrompt(makePRData({ linkedIssues: [{ number: 42 }] })); + assert.match(result, /#42/); + }); + + it('shows no linked issue when list is empty', () => { + const result = buildUserPrompt(makePRData({ linkedIssues: [] })); + assert.match(result, /Linked issue: ❌ none/); + }); + + it('includes a diff block for each file', () => { + const result = buildUserPrompt(makePRData({ + files: [ + { filename: 'src/a.js', status: 'added', additions: 1, deletions: 0, patch: '+a' }, + { filename: 'src/b.js', status: 'modified', additions: 2, deletions: 1, patch: '-b\n+B' }, + ], + totalFiles: 2, + })); + assert.match(result, /src\/a\.js/); + assert.match(result, /src\/b\.js/); + assert.match(result, /```diff/); + }); + + it('does not include any omitted/skipped notes when all files fit', () => { + const result = buildUserPrompt(makePRData({ totalFiles: 1, budgetSkippedFiles: [] })); + assert.ok(!result.includes('omitted')); + assert.ok(!result.includes('Not Reviewed')); + }); + + it('shows an API-omitted note when totalFiles exceeds files + budgetSkippedFiles', () => { + // 10 total, 1 included, 0 budget-skipped → 9 not fetched from API + const result = buildUserPrompt(makePRData({ totalFiles: 10, budgetSkippedFiles: [] })); + assert.match(result, /9 additional file\(s\) not fetched/); + }); + + it('does not show API-omitted note when all files accounted for', () => { + // 3 total, 1 included, 2 budget-skipped → 0 API-omitted + const result = buildUserPrompt(makePRData({ + totalFiles: 3, + budgetSkippedFiles: ['src/b.js', 'src/c.js'], + })); + assert.ok(!result.includes('not fetched')); + }); + + it('renders the skipped-files section when budgetSkippedFiles is non-empty', () => { + const result = buildUserPrompt(makePRData({ + totalFiles: 3, + budgetSkippedFiles: ['src/b.js', 'src/c.js'], + })); + assert.match(result, /Files Not Reviewed/); + assert.match(result, /`src\/b\.js`/); + assert.match(result, /`src\/c\.js`/); + }); + + it('instructs the model to recommend smaller PRs in the skipped-files section', () => { + const result = buildUserPrompt(makePRData({ + totalFiles: 2, + budgetSkippedFiles: ['src/big.js'], + })); + assert.match(result, /smaller.*pull request/i); + }); + + it('does not render the skipped-files section when budgetSkippedFiles is empty', () => { + const result = buildUserPrompt(makePRData({ budgetSkippedFiles: [] })); + assert.ok(!result.includes('Files Not Reviewed')); + }); + + it('does not render the skipped-files section when budgetSkippedFiles is absent', () => { + // Legacy prData without the field + const data = makePRData(); + delete data.budgetSkippedFiles; + const result = buildUserPrompt(data); + assert.ok(!result.includes('Files Not Reviewed')); + }); + + it('uses (no description) placeholder when body is empty', () => { + const result = buildUserPrompt(makePRData({ body: '' })); + assert.match(result, /\(no description\)/); + }); +}); + +// --------------------------------------------------------------------------- +// buildUserPromptPreamble +// --------------------------------------------------------------------------- + +describe('buildUserPromptPreamble', () => { + it('is a string', () => { + assert.equal(typeof buildUserPromptPreamble(makePRData()), 'string'); + }); + + it('includes the PR title', () => { + assert.match(buildUserPromptPreamble(makePRData()), /Test PR/); + }); + + it('includes branch names', () => { + const result = buildUserPromptPreamble(makePRData()); + assert.match(result, /feature\/foo/); + assert.match(result, /main/); + }); + + it('does not include any diff fences (no per-file blocks)', () => { + assert.ok(!buildUserPromptPreamble(makePRData()).includes('```diff')); + }); + + it('does not include file patch content', () => { + const result = buildUserPromptPreamble(makePRData()); + assert.ok(!result.includes('-old\n+new')); + }); + + it('length is less than the full buildUserPrompt length (diff blocks are absent)', () => { + const full = buildUserPrompt(makePRData()); + const preamble = buildUserPromptPreamble(makePRData()); + assert.ok(preamble.length < full.length); + }); + + it('includes the skipped-files section when budgetSkippedFiles is non-empty', () => { + const result = buildUserPromptPreamble(makePRData({ totalFiles: 3, budgetSkippedFiles: ['src/b.js'] })); + assert.match(result, /Files Not Reviewed/); + }); + + it('does not include the skipped-files section when budgetSkippedFiles is empty', () => { + assert.ok(!buildUserPromptPreamble(makePRData()).includes('Files Not Reviewed')); + }); +}); diff --git a/.github/actions/overture-projection/scripts/lib/__tests__/skills.test.js b/.github/actions/overture-projection/scripts/lib/__tests__/skills.test.js new file mode 100644 index 0000000..66b87f8 --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/__tests__/skills.test.js @@ -0,0 +1,159 @@ +'use strict'; + +const { describe, it } = require('node:test'); +const assert = require('node:assert/strict'); +const { parseFrontmatter, filterSkills } = require('../skills'); + +// --------------------------------------------------------------------------- +// parseFrontmatter +// --------------------------------------------------------------------------- + +describe('parseFrontmatter', () => { + it('returns empty defaults when no frontmatter block is present', () => { + const result = parseFrontmatter('# Just a heading\n\nBody text.'); + assert.deepEqual(result, { description: '', contextFiles: [], surfaces: null }); + }); + + it('parses an inline description', () => { + const raw = '---\ndescription: Checks container images\nsurfaces: [pr-reviewer]\n---\n# Body'; + const { description } = parseFrontmatter(raw); + assert.equal(description, 'Checks container images'); + }); + + it('strips surrounding quotes from description', () => { + const raw = "---\ndescription: 'Quoted description'\n---\n"; + const { description } = parseFrontmatter(raw); + assert.equal(description, 'Quoted description'); + }); + + it('parses a block-scalar description (> form)', () => { + const raw = '---\ndescription: >\n This is a long\n description.\n---\n'; + const { description } = parseFrontmatter(raw); + // Collapsed to a single line + assert.ok(description.includes('This is a long')); + }); + + it('returns surfaces as an array when present', () => { + const raw = '---\ndescription: foo\nsurfaces: [pr-reviewer, agent]\n---\n'; + const { surfaces } = parseFrontmatter(raw); + assert.deepEqual(surfaces, ['pr-reviewer', 'agent']); + }); + + it('returns surfaces: null when the surfaces field is absent', () => { + const raw = '---\ndescription: foo\n---\n'; + const { surfaces } = parseFrontmatter(raw); + assert.equal(surfaces, null); + }); + + it('returns surfaces: [] for an empty bracket list', () => { + const raw = '---\ndescription: foo\nsurfaces: []\n---\n'; + const { surfaces } = parseFrontmatter(raw); + assert.deepEqual(surfaces, []); + }); + + it('parses context-files list', () => { + const raw = [ + '---', + 'description: foo', + 'context-files:', + ' - OvertureMaps/schema:docs/overview.md', + ' - OvertureMaps/schema:docs/spec.md', + '---', + ].join('\n'); + const { contextFiles } = parseFrontmatter(raw); + assert.deepEqual(contextFiles, [ + 'OvertureMaps/schema:docs/overview.md', + 'OvertureMaps/schema:docs/spec.md', + ]); + }); + + it('returns empty contextFiles when field is absent', () => { + const raw = '---\ndescription: foo\n---\n'; + const { contextFiles } = parseFrontmatter(raw); + assert.deepEqual(contextFiles, []); + }); + + it('handles a single-item surfaces list', () => { + const raw = '---\nsurfaces: [pr-reviewer]\n---\n'; + const { surfaces } = parseFrontmatter(raw); + assert.deepEqual(surfaces, ['pr-reviewer']); + }); +}); + +// --------------------------------------------------------------------------- +// filterSkills +// --------------------------------------------------------------------------- + +describe('filterSkills', () => { + it('includes skills whose surfaces contain pr-reviewer', () => { + const skills = [ + { name: 'a', raw: '---\ndescription: A\nsurfaces: [pr-reviewer]\n---\n' }, + ]; + const result = filterSkills(skills); + assert.equal(result.length, 1); + assert.equal(result[0].name, 'a'); + }); + + it('excludes skills whose surfaces do not include pr-reviewer', () => { + const skills = [ + { name: 'b', raw: '---\ndescription: B\nsurfaces: [agent]\n---\n' }, + ]; + const result = filterSkills(skills); + assert.equal(result.length, 0); + }); + + it('includes skills with no surfaces field (legacy pass-through)', () => { + const skills = [ + { name: 'c', raw: '# No frontmatter at all\n\nBody.' }, + ]; + const result = filterSkills(skills); + assert.equal(result.length, 1); + assert.equal(result[0].name, 'c'); + }); + + it('includes multi-surface skills that contain pr-reviewer', () => { + const skills = [ + { name: 'd', raw: '---\ndescription: D\nsurfaces: [pr-reviewer, agent]\n---\n' }, + ]; + const result = filterSkills(skills); + assert.equal(result.length, 1); + }); + + it('returns the correct shape for each included skill', () => { + const skills = [ + { + name: 'e', + raw: [ + '---', + 'description: My skill', + 'surfaces: [pr-reviewer]', + 'context-files:', + ' - owner/repo:path/to/file.md', + '---', + '# Body', + ].join('\n'), + }, + ]; + const [skill] = filterSkills(skills); + assert.equal(skill.name, 'e'); + assert.equal(skill.description, 'My skill'); + assert.deepEqual(skill.contextFiles, ['owner/repo:path/to/file.md']); + assert.ok(typeof skill.raw === 'string'); + }); + + it('handles a mixed array correctly', () => { + const skills = [ + { name: 'pr', raw: '---\nsurfaces: [pr-reviewer]\n---\n' }, + { name: 'agent', raw: '---\nsurfaces: [agent]\n---\n' }, + { name: 'both', raw: '---\nsurfaces: [pr-reviewer, agent]\n---\n' }, + { name: 'none', raw: '# no frontmatter' }, + ]; + const result = filterSkills(skills); + const names = result.map(s => s.name); + assert.deepEqual(names, ['pr', 'both', 'none']); + }); + + it('returns an empty array for an empty input', () => { + assert.deepEqual(filterSkills([]), []); + }); +}); diff --git a/.github/actions/overture-projection/scripts/lib/context.js b/.github/actions/overture-projection/scripts/lib/context.js new file mode 100644 index 0000000..a1ce1ce --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/context.js @@ -0,0 +1,143 @@ +/** + * @file lib/context.js + * @description Context-file fetching utilities. + * + * Used by fetch-context.js. Extracted for unit-testability — ref parsing, + * content processing, and truncation are pure transformations with no I/O. + */ + +'use strict'; + +const { compressMarkdown } = require('./markdown'); +const { DEFAULT_MAX_CONTEXT_FILE_CHARS } = require('./defaults'); + +/** + * Maximum character length for any single context file after compression + * when no dynamic budget is available (legacy / test fallback only). + * Sourced from defaults.js — update it there. + * + * @type {number} + */ +const MAX_CONTEXT_FILE_CHARS = DEFAULT_MAX_CONTEXT_FILE_CHARS; + +/** + * Computes the per-file character budget for context files. + * + * Allocates 10 % of the total input token budget (converted to chars at + * 4 chars/token) so that context files scale with the model's context window. + * An optional hard cap (`maxOverride`) lets callers enforce an upper bound + * regardless of how large the window is. + * + * @param {number} maxInputTokens - Provider input-token budget (e.g. 6200 or 190000). + * @param {number} [maxOverride=0] - Hard cap in chars; 0 / falsy means no cap. + * @returns {number} Character limit to pass to processContextFile. + * + * @example + * contextFileCharBudget(6200) // => 2480 (10% of 6200*4) + * contextFileCharBudget(190000) // => 76000 (10% of 190000*4) + * contextFileCharBudget(190000, 20000) // => 20000 (cap applied) + */ +function contextFileCharBudget(maxInputTokens, maxOverride = 0) { + const dynamic = Math.floor(maxInputTokens * 4 * 0.1); + return (maxOverride > 0) ? Math.min(dynamic, maxOverride) : dynamic; +} + +/** + * @typedef {Object} ParsedRef + * @property {string} owner - Repository owner. + * @property {string} repo - Repository name. + * @property {string} filePath - Path to the file within the repository. + */ + +/** + * Parses an `owner/repo:path` context-file ref string into its components. + * + * Returns `null` if the ref is malformed (missing colon separator, or the + * repo portion does not contain a `/`). + * + * @param {string} ref - Raw ref string from SKILL.md frontmatter. + * @returns {ParsedRef|null} + * + * @example + * parseContextRef('OvertureMaps/schema:docs/overview.md') + * // => { owner: 'OvertureMaps', repo: 'schema', filePath: 'docs/overview.md' } + * + * parseContextRef('bad-ref') + * // => null + */ +function parseContextRef(ref) { + const sep = ref.indexOf(':'); + if (sep === -1) return null; + const repoFull = ref.slice(0, sep); + const filePath = ref.slice(sep + 1); + const slash = repoFull.indexOf('/'); + if (slash === -1) return null; + const owner = repoFull.slice(0, slash); + const repo = repoFull.slice(slash + 1); + if (!owner || !repo || !filePath) return null; + return { owner, repo, filePath }; +} + +/** + * @typedef {Object} BudgetResult + * @property {string} content - Content, possibly truncated. + * @property {boolean} truncated - `true` if the content was cut to fit the budget. + */ + +/** + * Decodes a base64-encoded file payload, compresses the Markdown, and + * truncates to `maxChars` if necessary. + * + * The truncation notice appended when the content is too long is intentionally + * visible to the model so it knows the context is partial. + * + * @param {string} base64Content - Raw base64 string from the GitHub Contents API. + * @param {number} [maxChars] - Character limit (defaults to MAX_CONTEXT_FILE_CHARS). + * @returns {BudgetResult} + */ +function processContextFile(base64Content, maxChars = MAX_CONTEXT_FILE_CHARS) { + const decoded = Buffer.from(base64Content, 'base64').toString('utf-8'); + let content = compressMarkdown(decoded); + const truncated = content.length > maxChars; + if (truncated) { + content = + content.slice(0, maxChars) + + '\n\n[This context file was intentionally truncated by the review system to fit the token budget. The content above is a partial extract — the remainder is not available in this review.]'; + } + return { content, truncated }; +} + +/** + * Groups an array of fetched context entries (which may include `null` for + * failed fetches) by skill name. + * + * Null entries are silently dropped. Skills with no successful fetches are + * absent from the returned map. + * + * @typedef {Object} ContextEntry + * @property {string} skillName - Skill that owns this context file. + * @property {string} ref - Original `owner/repo:path` ref. + * @property {string} content - Compressed file content (possibly truncated). + * + * @param {(ContextEntry|null)[]} entries - Mixed array of entries and nulls. + * @returns {Record} + * + * @example + * groupBySkill([ + * { skillName: 'a', ref: 'o/r:f', content: '...' }, + * null, + * { skillName: 'a', ref: 'o/r:g', content: '...' }, + * { skillName: 'b', ref: 'o/r:h', content: '...' }, + * ]) + * // => { a: [{...}, {...}], b: [{...}] } + */ +function groupBySkill(entries) { + const result = {}; + for (const entry of entries) { + if (!entry) continue; + (result[entry.skillName] ??= []).push(entry); + } + return result; +} + +module.exports = { parseContextRef, processContextFile, contextFileCharBudget, groupBySkill, MAX_CONTEXT_FILE_CHARS }; diff --git a/.github/actions/overture-projection/scripts/lib/defaults.js b/.github/actions/overture-projection/scripts/lib/defaults.js new file mode 100644 index 0000000..9684741 --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/defaults.js @@ -0,0 +1,136 @@ +'use strict'; + +/** + * @file lib/defaults.js + * @description Canonical runtime defaults for Overture PRojection. + * + * All model IDs and token/budget numbers used as runtime fallbacks live here. + * When Anthropic or GitHub Models releases a new generation, or token limits + * change, update this file only — scripts and tests import from here rather + * than hardcoding values. + * + * NOTE: action.yml and workflow YAML `default:` fields are static and cannot + * reference this file. Mirror any changes there too (see inline comments). + */ + +// ── Model IDs ───────────────────────────────────────────────────────────────── + +/** Default model provider. */ +const DEFAULT_PROVIDER = 'github-models'; + +/** Default review model when model-provider is 'github-models'. */ +const GITHUB_MODELS_DEFAULT_MODEL = 'gpt-4.1'; + +/** Default skill-selection model when model-provider is 'github-models'. */ +const GITHUB_MODELS_DEFAULT_SELECTION = 'gpt-4.1-mini'; + +/** Default review model when model-provider is 'anthropic'. */ +const ANTHROPIC_DEFAULT_MODEL = 'claude-opus-4-6'; + +/** Default skill-selection model when model-provider is 'anthropic'. */ +const ANTHROPIC_DEFAULT_SELECTION = 'claude-haiku-4-6'; + +// ── Token / budget limits — GitHub Models ───────────────────────────────────── + +/** + * Default max input tokens for GitHub Models gpt-4.1. + * 8 000 context − 1 500 output − 300 tokenisation margin = 6 200. + * Mirror: action.yml `max-input-tokens` default. + * + * @type {number} + */ +const GITHUB_MODELS_MAX_INPUT_TOKENS = 6200; + +/** + * Default max output tokens for GitHub Models. + * Mirror: action.yml `max-output-tokens` default. + * + * @type {number} + */ +const GITHUB_MODELS_MAX_OUTPUT_TOKENS = 1500; + +// ── Token / budget limits — Anthropic ──────────────────────────────────────── + +/** + * Default max input tokens for Anthropic Claude models (200k context window). + * 200 000 − 4 096 output − ~6 000 tokenisation margin ≈ 190 000. + * + * @type {number} + */ +const ANTHROPIC_MAX_INPUT_TOKENS = 190000; + +/** + * Default max output tokens for Anthropic Claude models. + * + * @type {number} + */ +const ANTHROPIC_MAX_OUTPUT_TOKENS = 4096; + +// ── Other budget limits ─────────────────────────────────────────────────────── + +/** + * Default fetch ceiling for diff content pulled from the GitHub API (chars). + * This is a fetch guard only — actual context trimming uses the dynamic budget. + * Mirror: action.yml `max-diff-chars` default. + * + * @type {number} + */ +const DEFAULT_MAX_DIFF_CHARS = 100000; + +/** + * Maximum character length for any single context file after compression. + * At ~4 chars/token this is roughly 1 250 tokens per file. + * Mirror: context.js MAX_CONTEXT_FILE_CHARS (kept as a named re-export there). + * + * @type {number} + */ +const DEFAULT_MAX_CONTEXT_FILE_CHARS = 5000; + +// ── Provider defaults lookup ────────────────────────────────────────────────── + +/** + * @typedef {Object} ProviderDefaults + * @property {string} model - Default review model ID. + * @property {string} selectionModel - Default skill-selection model ID. + * @property {number} maxInputTokens - Default max input tokens. + * @property {number} maxOutputTokens - Default max output tokens. + */ + +/** + * Returns the canonical defaults for a given provider. + * Use this instead of branching on the provider string in scripts. + * + * @param {'github-models'|'anthropic'} provider + * @returns {ProviderDefaults} + */ +function getProviderDefaults(provider) { + if (provider === 'anthropic') { + return { + model: ANTHROPIC_DEFAULT_MODEL, + selectionModel: ANTHROPIC_DEFAULT_SELECTION, + maxInputTokens: ANTHROPIC_MAX_INPUT_TOKENS, + maxOutputTokens: ANTHROPIC_MAX_OUTPUT_TOKENS, + }; + } + return { + model: GITHUB_MODELS_DEFAULT_MODEL, + selectionModel: GITHUB_MODELS_DEFAULT_SELECTION, + maxInputTokens: GITHUB_MODELS_MAX_INPUT_TOKENS, + maxOutputTokens: GITHUB_MODELS_MAX_OUTPUT_TOKENS, + }; +} + +module.exports = { + DEFAULT_PROVIDER, + GITHUB_MODELS_DEFAULT_MODEL, + GITHUB_MODELS_DEFAULT_SELECTION, + ANTHROPIC_DEFAULT_MODEL, + ANTHROPIC_DEFAULT_SELECTION, + GITHUB_MODELS_MAX_INPUT_TOKENS, + GITHUB_MODELS_MAX_OUTPUT_TOKENS, + ANTHROPIC_MAX_INPUT_TOKENS, + ANTHROPIC_MAX_OUTPUT_TOKENS, + DEFAULT_MAX_DIFF_CHARS, + DEFAULT_MAX_CONTEXT_FILE_CHARS, + getProviderDefaults, +}; diff --git a/.github/actions/overture-projection/scripts/lib/diff.js b/.github/actions/overture-projection/scripts/lib/diff.js new file mode 100644 index 0000000..7fc0da8 --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/diff.js @@ -0,0 +1,149 @@ +/** + * @file lib/diff.js + * @description PR diff filtering and budget utilities. + * + * Used by fetch-diff.js and post-review.js. Extracted for unit-testability — + * the ignore-pattern compilation and per-file budget trimming are independent + * of any GitHub API calls and can be exercised with plain data. + */ + +'use strict'; + +/** + * Computes the character budget for diff content given the number of tokens + * already consumed by the system prompt and user prompt overhead (everything + * except the diff blocks themselves), and the configured maximum input tokens. + * + * Uses a chars-per-token ratio of 4 as a conservative approximation. + * + * @param {number} nonDiffChars - Characters already consumed by system prompt + + * user prompt preamble (headers, issue section, file count line, skipped-files + * section, etc.) — everything except the `diffBlocks` themselves. + * @param {number} maxInputTokens - Maximum tokens available for the full input + * prompt (context window minus output reserve and safety margin). Passed in + * from config so no provider limits are hardcoded here. + * @returns {number} Remaining character budget for diff content. Minimum 0. + * + * @example + * diffCharBudget(4000, 6200) // => (6200 * 4) - 4000 = 20800 + */ +function diffCharBudget(nonDiffChars, maxInputTokens) { + return Math.max(0, maxInputTokens * 4 - nonDiffChars); +} + +/** + * Compiles a newline-separated list of glob-style patterns into anchored + * regular expressions. + * + * Each pattern supports `*` as a wildcard matching any sequence of characters. + * All other regex metacharacters are escaped. Each compiled regex is tested + * against both the full file path and the basename (see {@link isIgnored}). + * + * Empty lines and lines that are only whitespace are ignored. + * + * @param {string} patternsStr - Newline-separated glob patterns (e.g. from `IGNORE_FILES` env var). + * @returns {RegExp[]} Compiled anchored regexes, one per non-empty pattern. + * + * @example + * buildIgnorePatterns('*.lock\npackage-lock.json') + * // => [/^.*\.lock$/, /^package-lock\.json$/] + */ +function buildIgnorePatterns(patternsStr) { + return (patternsStr || '') + .split('\n') + .map(p => p.trim()) + .filter(Boolean) + .map(p => new RegExp('^' + p.replace(/[.+^${}()|[\]\\]/g, '\\$&').replace(/\*/g, '.*') + '$')); +} + +/** + * Returns `true` if a file entry should be excluded from the diff sent to the + * model, based on a set of compiled ignore patterns. + * + * Each pattern is tested against both the full repo-relative path and the + * basename so that e.g. `*.lock` matches `subdir/foo.lock`. + * + * @param {{ filename: string }} file - File entry (only `filename` is read). + * @param {RegExp[]} patterns - Compiled patterns from {@link buildIgnorePatterns}. + * @returns {boolean} `true` if the file matches any ignore pattern. + * + * @example + * const patterns = buildIgnorePatterns('*.lock'); + * isIgnored({ filename: 'package-lock.json' }, patterns); // true + * isIgnored({ filename: 'src/index.js' }, patterns); // false + */ +function isIgnored(file, patterns) { + const basename = file.filename.split('/').pop(); + return patterns.some(re => re.test(file.filename) || re.test(basename)); +} + +/** + * @typedef {Object} RawFile + * @property {string} filename - Repo-relative file path. + * @property {string} status - Change status from the GitHub API. + * @property {number} additions - Lines added. + * @property {number} deletions - Lines deleted. + * @property {string|undefined} patch - Raw unified-diff patch, or undefined for binary files. + */ + +/** + * @typedef {Object} FileBudgetResult + * @property {Array<{filename:string,status:string,additions:number,deletions:number,patch:string}>} included + * Files whose full patch fits within the character budget, with absent patches replaced by a placeholder. + * @property {RawFile[]} skipped + * Files dropped because the budget was exhausted before their patch could be included. + */ + +/** + * Applies a total character budget across an ordered list of files, keeping + * whole files rather than truncating individual patches mid-diff. + * + * Files are consumed in order. Each file's patch is measured (absent/binary + * patches count as 0 chars). Once adding the next file would exceed + * `totalChars`, that file and all subsequent files are placed in `skipped`. + * + * This produces cleaner model output than mid-patch truncation: the model + * receives complete diffs for the files it does see, and can call out the + * skipped files explicitly in its review. + * + * @param {RawFile[]} files - Files to partition (typically post-ignore-filter). + * @param {number} totalChars - Maximum total characters of patch content to include. + * @returns {FileBudgetResult} + * + * @example + * applyFileBudget([ + * { filename: 'a.js', patch: 'x'.repeat(400), ... }, + * { filename: 'b.js', patch: 'y'.repeat(400), ... }, + * ], 500) + * // => { included: [{ filename: 'a.js', ... }], skipped: [{ filename: 'b.js', ... }] } + */ +function applyFileBudget(files, totalChars) { + const included = []; + const skipped = []; + let used = 0; + + for (const f of files) { + const patchLen = f.patch ? f.patch.length : 0; + if (used + patchLen <= totalChars) { + included.push({ + filename: f.filename, + status: f.status, + additions: f.additions, + deletions: f.deletions, + patch: f.patch || '(binary or no textual diff)', + }); + used += patchLen; + } else { + skipped.push(f); + } + } + + return { included, skipped }; +} + +module.exports = { + diffCharBudget, + buildIgnorePatterns, + isIgnored, + applyFileBudget, +}; diff --git a/.github/actions/overture-projection/scripts/lib/github.js b/.github/actions/overture-projection/scripts/lib/github.js new file mode 100644 index 0000000..eaf428a --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/github.js @@ -0,0 +1,65 @@ +/** + * @file lib/github.js + * @description GitHub context resolution helpers. + * + * Used by fetch-diff.js and post-review.js, both of which need to resolve the + * target repository and PR number from either explicit env var overrides or the + * Actions event context. Extracted for unit-testability — the fallback logic is + * easy to misconfigure and benefits from isolated tests. + */ + +'use strict'; + +/** + * Resolves the target repository as `{ owner, repo }`. + * + * Prefers the `REPOSITORY` environment variable (set from the `repository` + * action input) so that cross-repo reviews work correctly. Falls back to + * `context.repo`, which is derived from the workflow's own repository and + * would be wrong when reviewing a PR in a different repo. + * + * @param {string|undefined} repositoryEnv - Value of `process.env.REPOSITORY`. + * @param {{ owner: string, repo: string }} contextRepo - `context.repo` from the Actions context. + * @returns {{ owner: string, repo: string }} + * + * @example + * resolveRepo('OvertureMaps/overture-tiles', { owner: 'OvertureMaps', repo: 'omf-devex' }) + * // => { owner: 'OvertureMaps', repo: 'overture-tiles' } + * + * resolveRepo(undefined, { owner: 'OvertureMaps', repo: 'omf-devex' }) + * // => { owner: 'OvertureMaps', repo: 'omf-devex' } + */ +function resolveRepo(repositoryEnv, contextRepo) { + if (repositoryEnv) { + const [owner, repo] = repositoryEnv.split('/'); + return { owner, repo }; + } + return { owner: contextRepo.owner, repo: contextRepo.repo }; +} + +/** + * Resolves the pull request number to review. + * + * Prefers the PR number from the event payload (set automatically on + * `pull_request` events). Falls back to parsing `PR_NUMBER` env var, which + * is required for `workflow_dispatch` triggers where no PR payload is present. + * + * Returns `null` when neither source yields a valid positive integer, which + * the caller should treat as a hard failure. + * + * @param {number|undefined} payloadPrNumber - `context.payload.pull_request?.number`. + * @param {string|undefined} prNumberEnv - Value of `process.env.PR_NUMBER`. + * @returns {number|null} PR number, or `null` if unavailable. + * + * @example + * resolvePrNumber(42, undefined) // => 42 + * resolvePrNumber(undefined, '7') // => 7 + * resolvePrNumber(undefined, '') // => null + */ +function resolvePrNumber(payloadPrNumber, prNumberEnv) { + if (payloadPrNumber) return payloadPrNumber; + const parsed = parseInt(prNumberEnv); + return Number.isFinite(parsed) && parsed > 0 ? parsed : null; +} + +module.exports = { resolveRepo, resolvePrNumber }; diff --git a/.github/actions/overture-projection/scripts/lib/markdown.js b/.github/actions/overture-projection/scripts/lib/markdown.js new file mode 100644 index 0000000..f0726e5 --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/markdown.js @@ -0,0 +1,37 @@ +/** + * @file lib/markdown.js + * @description Markdown compression utility. + * + * Shared by fetch-context.js (compressing context files before storage) and + * post-review.js (compressing skill bodies before prompt assembly). + */ + +'use strict'; + +/** + * Strips YAML frontmatter and HTML comments from a Markdown string, then + * collapses runs of three or more blank lines to a single blank line and + * trims leading/trailing whitespace. + * + * Reduces token cost when Markdown is injected into a model prompt without + * losing any instructional content. + * + * @param {string} text - Raw Markdown content, optionally with YAML frontmatter. + * @returns {string} Compressed Markdown. + * + * @example + * compressMarkdown('---\nname: foo\n---\n\n# Hello\n\n\n\nWorld') + * // => '# Hello\n\nWorld' + */ +function compressMarkdown(text) { + const stripped = text.replace(/^---\r?\n[\s\S]*?\r?\n---(?:\r?\n)?/, ''); + const noComments = stripped.replace(//g, ''); + return noComments + .split('\n') + .map(l => l.trimEnd()) + .join('\n') + .replace(/\n{3,}/g, '\n\n') + .trim(); +} + +module.exports = { compressMarkdown }; diff --git a/.github/actions/overture-projection/scripts/lib/models.js b/.github/actions/overture-projection/scripts/lib/models.js new file mode 100644 index 0000000..f75c64d --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/models.js @@ -0,0 +1,261 @@ +/** + * @file lib/models.js + * @description Model API utilities shared by select-skills.js and post-review.js. + * + * Supports two providers: + * - 'github-models' (default) — OpenAI-compatible endpoint proxied by GitHub + * - 'anthropic' — Anthropic Messages API + * + * Both return a normalised {@link ChatResult} so callers don't need to branch + * on the provider themselves. + */ + +'use strict'; + +/** @type {string} ANSI escape for yellow text — used for token counts. */ +const YELLOW = '\x1b[33m'; + +/** @type {string} ANSI escape for blue text — used for model names. */ +const BLUE = '\x1b[34m'; + +/** @type {string} ANSI escape to reset colour. */ +const RESET = '\x1b[0m'; + +/** + * @typedef {Object} ChatMessage + * @property {'system'|'user'|'assistant'} role + * @property {string} content + */ + +/** + * @typedef {Object} ChatOptions + * @property {'github-models'|'anthropic'} provider - Which API to call. + * @property {string} token - Auth token (GitHub token or Anthropic API key). + * @property {string} model - Model ID (e.g. 'gpt-4.1' or 'claude-opus-4-5'). + * @property {ChatMessage[]} messages - Ordered message list. + * @property {number} maxTokens - Maximum output tokens. + * @property {number} [temperature=0.2] - Sampling temperature. + * @property {boolean} [jsonMode=false] - Request JSON output (GitHub Models only; + * for Anthropic, a JSON instruction is injected into the system prompt instead). + */ + +/** + * @typedef {Object} ChatResult + * @property {string} text - The model's response text. + * @property {string|null} finishReason - Normalised stop reason ('stop', 'length', etc.) or null. + * @property {{ input: number, output: number, total: number }} usage - Token counts. + * @property {Response} rawResponse - The raw fetch Response (for rate-limit header inspection). + */ + +/** + * Calls the GitHub Models OpenAI-compatible chat-completions endpoint. + * + * @param {ChatOptions} opts + * @returns {Promise} Raw fetch response. + */ +async function _callGitHubModels(opts) { + const body = { + model: opts.model, + messages: opts.messages, + max_tokens: opts.maxTokens, + temperature: opts.temperature ?? 0.2, + }; + if (opts.jsonMode) { + body.response_format = { type: 'json_object' }; + } + return fetch('https://models.inference.ai.azure.com/chat/completions', { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + 'Authorization': `Bearer ${opts.token}`, + }, + body: JSON.stringify(body), + }); +} + +/** + * Calls the Anthropic Messages API. + * + * System messages are extracted from `opts.messages` (Anthropic takes the + * system prompt as a top-level string, not as a message role). When + * `jsonMode` is true, an instruction to respond with JSON only is appended + * to the system prompt, since Anthropic has no `response_format` field. + * + * @param {ChatOptions} opts + * @returns {Promise} Raw fetch response. + */ +async function _callAnthropic(opts) { + const systemMessages = opts.messages.filter(m => m.role === 'system'); + const userMessages = opts.messages.filter(m => m.role !== 'system'); + + let systemText = systemMessages.map(m => m.content).join('\n\n'); + if (opts.jsonMode) { + systemText = (systemText ? systemText + '\n\n' : '') + + 'Respond with valid JSON only. Do not include any prose before or after the JSON object.'; + } + + const body = { + model: opts.model, + messages: userMessages, + max_tokens: opts.maxTokens, + ...(systemText ? { system: systemText } : {}), + }; + // Anthropic ignores temperature=0; only set it when explicitly provided and non-default + if (opts.temperature !== undefined) { + body.temperature = opts.temperature; + } + + return fetch('https://api.anthropic.com/v1/messages', { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + 'x-api-key': opts.token, + 'anthropic-version': '2023-06-01', + }, + body: JSON.stringify(body), + }); +} + +/** + * Strips a markdown code fence from a model response when `jsonMode` was + * requested. Some models (notably Claude) wrap JSON output in ```json … ``` + * despite being instructed not to. This is applied to all responses so callers + * always receive raw text. + * + * Only strips a fence when the trimmed text starts with ``` — leaves prose + * responses completely untouched. + * + * @param {string} text - Raw model response text. + * @returns {string} Text with leading/trailing code fence removed, trimmed. + */ +function stripCodeFence(text) { + const t = text.trim(); + if (!t.startsWith('```')) return t; + // Remove opening fence line (```json, ```text, ``` etc.) and closing ``` + return t + .replace(/^```[^\n]*\n/, '') + .replace(/\n?```\s*$/, '') + .trim(); +} + +/** + * Normalises a successful GitHub Models response body to {@link ChatResult}. + * + * @param {object} json - Parsed response body. + * @param {Response} resp - Raw fetch Response. + * @returns {ChatResult} + */ +function _normaliseGitHubModels(json, resp) { + const choice = json.choices?.[0]; + return { + text: stripCodeFence(choice?.message?.content ?? ''), + finishReason: choice?.finish_reason ?? null, + usage: { + input: json.usage?.prompt_tokens ?? 0, + output: json.usage?.completion_tokens ?? 0, + total: json.usage?.total_tokens ?? 0, + }, + rawResponse: resp, + }; +} + +/** + * Normalises a successful Anthropic response body to {@link ChatResult}. + * + * @param {object} json - Parsed response body. + * @param {Response} resp - Raw fetch Response. + * @returns {ChatResult} + */ +function _normaliseAnthropic(json, resp) { + const raw = json.content?.find(b => b.type === 'text')?.text ?? ''; + const text = stripCodeFence(raw); + // Anthropic stop_reason values: 'end_turn', 'max_tokens', 'stop_sequence' + const finishMap = { end_turn: 'stop', max_tokens: 'length', stop_sequence: 'stop' }; + return { + text, + finishReason: finishMap[json.stop_reason] ?? json.stop_reason ?? null, + usage: { + input: json.usage?.input_tokens ?? 0, + output: json.usage?.output_tokens ?? 0, + total: (json.usage?.input_tokens ?? 0) + (json.usage?.output_tokens ?? 0), + }, + rawResponse: resp, + }; +} + +/** + * Dispatches a chat-completion request to the configured provider and returns + * a normalised result. + * + * Throws an `Error` (with `.status` and `.body` properties) when the API + * returns a non-2xx response, so callers can handle errors uniformly. + * + * @param {ChatOptions} opts + * @returns {Promise} + * + * @throws {Error} On non-2xx API response. Error has `.status` (number) and `.body` (string). + * + * @example + * const result = await callChatCompletion({ + * provider: 'anthropic', + * token: process.env.ANTHROPIC_API_KEY, + * model: 'claude-opus-4-5', + * messages: [{ role: 'user', content: 'Hello' }], + * maxTokens: 512, + * }); + * console.log(result.text); + */ +async function callChatCompletion(opts) { + const provider = opts.provider || 'github-models'; + const resp = provider === 'anthropic' + ? await _callAnthropic(opts) + : await _callGitHubModels(opts); + + if (!resp.ok) { + const body = await resp.text(); + const err = new Error(`${provider} API error ${resp.status}: ${body}`); + err.status = resp.status; + err.body = body; + throw err; + } + + const json = await resp.json(); + return provider === 'anthropic' + ? _normaliseAnthropic(json, resp) + : _normaliseGitHubModels(json, resp); +} + +/** + * Logs GitHub Models rate-limit response headers as an Actions log line. + * + * Reads the four standard `x-ratelimit-*` headers and `retry-after` from the + * response. Emits `core.warning` when either remaining-requests ≤ 10 or + * remaining-tokens ≤ 1000 (approaching exhaustion); otherwise `core.info`. + * No-ops silently if neither `x-ratelimit-remaining-requests` nor + * `x-ratelimit-remaining-tokens` is present (non-Models endpoints, Anthropic). + * + * @param {Response} resp - Fetch Response from a Models API call. + * @param {string} label - Short label for the log line (e.g. `'selection'`). + * @param {{ warning: Function, info: Function }} core - Actions core logger. + */ +function logRateLimit(resp, label, core) { + const remainingReqs = resp.headers.get('x-ratelimit-remaining-requests'); + const remainingTokens = resp.headers.get('x-ratelimit-remaining-tokens'); + const limitTokens = resp.headers.get('x-ratelimit-limit-tokens'); + const retryAfter = resp.headers.get('retry-after'); + + if (remainingReqs === null && remainingTokens === null) return; + + const tokenPct = (remainingTokens !== null && limitTokens) + ? ` (${YELLOW}${Math.round((remainingTokens / limitTokens) * 100)}%${RESET})` + : ''; + const warn = parseInt(remainingReqs) <= 10 || parseInt(remainingTokens) <= 1000; + const msg = + `📊 ${label} rate limit: ${YELLOW}${remainingReqs ?? '?'}${RESET} requests remaining, ` + + `${YELLOW}${remainingTokens ?? '?'}/${limitTokens ?? '?'} tokens${RESET}${tokenPct}` + + `${retryAfter ? ` — retry-after: ${retryAfter}s` : ''}`; + + warn ? core.warning(`⚠️ ${msg}`) : core.info(msg); +} + +module.exports = { YELLOW, BLUE, RESET, logRateLimit, callChatCompletion }; diff --git a/.github/actions/overture-projection/scripts/lib/prompt.js b/.github/actions/overture-projection/scripts/lib/prompt.js new file mode 100644 index 0000000..ea388c9 --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/prompt.js @@ -0,0 +1,258 @@ +/** + * @file lib/prompt.js + * @description System and user prompt assembly for the review model. + * + * Used by post-review.js. Extracted for unit-testability — both functions are + * pure string transformations over plain data objects with no I/O or API calls, + * making them straightforward to exercise with snapshot tests. + */ + +'use strict'; + +const { compressMarkdown } = require('./markdown'); + +/** + * @typedef {Object} Skill + * @property {string} name - Skill ID. + * @property {string} raw - Full SKILL.md content including frontmatter. + */ + +/** + * @typedef {Object} ContextEntry + * @property {string} ref - `owner/repo:path` ref string. + * @property {string} content - Compressed file content. + */ + +/** + * Assembles the system prompt from the ordered list of selected skills and + * their fetched context files. + * + * Each skill block is formatted as: + * ``` + * + * + * [## Context Files + * [] + * ] + * ``` + * Blocks are separated by `---` horizontal rules so the model can treat them + * as distinct instruction sets. The system prompt is empty string when no + * skills are provided, which signals to the caller to omit the system message + * entirely (letting the model use its built-in review behaviour). + * + * @param {Skill[]} skills - Selected skills in display order. + * @param {Record} contextBySkill - Map of skill name to fetched context entries. + * @returns {string} System prompt string, or empty string if `skills` is empty. + * + * @example + * buildSystemPrompt( + * [{ name: 'pr-review', raw: '---\nname: pr-review\n---\nReview all PRs.' }], + * {} + * ) + * // => '\nReview all PRs.' + */ +function buildSystemPrompt(skills, contextBySkill) { + if (skills.length === 0) return ''; + return skills + .map(s => { + const body = compressMarkdown(s.raw); + const ctxEntries = contextBySkill[s.name] || []; + const ctxBlocks = ctxEntries.map(r => `[${r.ref}]\n${r.content}`).join('\n\n'); + const content = ctxBlocks ? `${body}\n\n## Context Files\n\n${ctxBlocks}` : body; + return `\n${content}`; + }) + .join('\n\n---\n\n'); +} + +/** + * @typedef {Object} FileDiff + * @property {string} filename - Repo-relative file path. + * @property {string} status - Change status (added, modified, removed, renamed, etc.). + * @property {number} additions - Lines added. + * @property {number} deletions - Lines deleted. + * @property {string} patch - Full diff patch text (never truncated mid-diff). + */ + +/** + * @typedef {Object} PRData + * @property {number} number - PR number. + * @property {string} title - PR title. + * @property {string} body - PR description (empty string if absent). + * @property {number} totalFiles - Total changed files (may exceed files.length). + * @property {string} headRef - Source branch. + * @property {string} baseRef - Target branch. + * @property {string|null} authorAssociation - Author association from GitHub API, or null. + * @property {Array} linkedIssues - Closing-issue reference nodes. + * @property {string|null} repoLicense - SPDX licence ID, or null. + * @property {FileDiff[]} files - Files whose full patch fits within the diff budget. + * @property {string[]} budgetSkippedFiles - Filenames dropped because the diff budget was exhausted. + */ + +/** + * File extensions considered documentation-only. + * When every changed file has one of these extensions the PR is flagged as + * `PR type: docs-only` and the tests-present check is suppressed. + * + * @type {Set} + */ +const DOCS_EXTENSIONS = new Set(['.md', '.mdx', '.rst', '.txt', '.adoc']); + +/** + * Returns `true` if the filename looks like a test file. + * + * Matches common conventions: + * - Directory-based: `tests/`, `test/`, `__tests__/`, `spec/` + * - Suffix-based: `.test.js`, `.spec.ts`, `_test.go`, `-test.js` + * - Python: `test_foo.py` (anywhere in path) + * + * @param {string} filename - Repo-relative file path. + * @returns {boolean} + */ +function isTestFile(filename) { + return ( + /\/(tests?|__tests?__|spec)\//i.test(filename) || + /[._-](test|spec)\.[^.]+$/.test(filename) || + /^tests?\//i.test(filename) || + /\/test_[^/]+\.py$/.test(filename) || + /^test_[^/]+\.py$/.test(filename) + ); +} + +/** + * Assembles the user prompt from PR metadata and file diffs. + * + * The prompt is structured as: + * 1. A header line with branch names, licence, description flag, PR type, + * test coverage flag, and author association. + * 2. The PR description body (or a placeholder). + * 3. A linked-issues line. + * 4. One fenced diff block per changed file. + * 5. An optional truncation notice when files were omitted due to MAX_FILES. + * + * All inputs are plain data — no I/O or API calls occur here. + * + * @param {PRData} prData - Structured PR metadata and file diffs. + * @returns {string} User prompt string ready for the chat-completions API. + */ +/** + * @typedef {Object} UserPromptParts + * @property {string} prHeader - Title + branch/metadata line + description. + * @property {string} issuesSection - Linked-issues line. + * @property {string} filesHeader - "## Changed Files (N of M)" line. + * @property {string} apiOmittedNote - Note about files not fetched due to MAX_FILES, or ''. + * @property {string} skippedSection - "## Files Not Reviewed" block, or ''. + */ + +/** + * Builds the structural parts of the user prompt that do not include the per-file + * diff blocks. Used both by {@link buildUserPrompt} (which adds the diff blocks) + * and by `post-review.js` to measure non-diff overhead before computing the + * dynamic diff character budget. + * + * @param {PRData} prData + * @returns {UserPromptParts} + */ +function buildUserPromptParts(prData) { + const licenseNote = prData.repoLicense ? `License: ${prData.repoLicense}` : 'License: unknown'; + const descriptionNote = prData.body ? 'Description: ✅' : 'Description: ❌ missing'; + + const docsOnly = + prData.files.length > 0 && + prData.files.every(f => DOCS_EXTENSIONS.has('.' + f.filename.split('.').pop().toLowerCase())); + const prTypeNote = docsOnly ? 'PR type: docs-only' : 'PR type: code'; + + const hasTests = prData.files.some(f => isTestFile(f.filename)); + const testsNote = docsOnly ? '' : ` | Tests: ${hasTests ? '✅' : '❌ none in diff'}`; + const authorNote = prData.authorAssociation ? ` | Author: ${prData.authorAssociation}` : ''; + + const prHeader = + `## ${prData.title}\n\n` + + `Branch: \`${prData.headRef}\` → \`${prData.baseRef}\` | ${licenseNote} | ${descriptionNote} | ${prTypeNote}${testsNote}${authorNote}\n\n` + + `${prData.body || '(no description)'}`; + + const issuesSection = `Linked issue: ${ + prData.linkedIssues?.length > 0 + ? `✅ ${prData.linkedIssues.map(i => `#${i.number}`).join(', ')}` + : '❌ none' + }`; + + // Files omitted due to MAX_FILES API page limit (not in diff at all) + const apiOmittedCount = prData.totalFiles - prData.files.length - (prData.budgetSkippedFiles?.length ?? 0); + const apiOmittedNote = apiOmittedCount > 0 + ? `\n> Note: ${apiOmittedCount} additional file(s) not fetched (exceeds max-files limit).\n` + : ''; + + const filesHeader = `## Changed Files (${prData.files.length} of ${prData.totalFiles})${apiOmittedNote}`; + + // Files fetched but dropped because the diff character budget was exhausted + const skipped = prData.budgetSkippedFiles ?? []; + const skippedSection = skipped.length > 0 + ? `\n\n## Files Not Reviewed (diff budget exhausted)\n\n` + + `The following ${skipped.length} file(s) were changed in this PR but could not be included ` + + `because the diff is too large to fit in the model's context window. ` + + `List each of these files in an **🚩 Flags** item and recommend that the contributor ` + + `break this PR into smaller, more focused pull requests:\n\n` + + skipped.map(f => `- \`${f}\``).join('\n') + : ''; + + return { prHeader, issuesSection, filesHeader, apiOmittedNote, skippedSection }; +} + +/** + * Returns the user prompt text that does not include per-file diff blocks — + * the preamble (intro, PR header, issues, file-count line) plus the + * skipped-files section. Used by `post-review.js` to measure non-diff overhead + * before computing the dynamic diff character budget. + * + * Note: this uses `prData.files` only for metadata (count, docs-only flag, + * tests flag). Pass the full file list so the flags are accurate before + * trimming occurs. + * + * @param {PRData} prData + * @returns {string} + */ +function buildUserPromptPreamble(prData) { + const { prHeader, issuesSection, filesHeader, skippedSection } = buildUserPromptParts(prData); + return `Review this pull request.\n\n${prHeader}\n\n${issuesSection}\n\n${filesHeader}\n\n` + + skippedSection; +} + +/** + * Assembles the user prompt from PR metadata and file diffs. + * + * The prompt is structured as: + * 1. A header line with branch names, licence, description flag, PR type, + * test coverage flag, and author association. + * 2. The PR description body (or a placeholder). + * 3. A linked-issues line. + * 4. One fenced diff block per changed file. + * 5. An optional truncation notice when files were omitted due to MAX_FILES. + * + * All inputs are plain data — no I/O or API calls occur here. + * + * @param {PRData} prData - Structured PR metadata and file diffs. + * @returns {string} User prompt string ready for the chat-completions API. + */ +function buildUserPrompt(prData) { + const { prHeader, issuesSection, filesHeader, skippedSection } = buildUserPromptParts(prData); + + const diffBlocks = prData.files.map( + f => `### ${f.filename} [${f.status}] +${f.additions} -${f.deletions}\n\`\`\`diff\n${f.patch}\n\`\`\`` + ); + + return ( + `Review this pull request.\n\n${prHeader}\n\n${issuesSection}\n\n` + + `${filesHeader}\n\n` + + diffBlocks.join('\n\n') + + skippedSection + ); +} + +module.exports = { + buildSystemPrompt, + buildUserPromptParts, + buildUserPromptPreamble, + buildUserPrompt, + isTestFile, + DOCS_EXTENSIONS, +}; diff --git a/.github/actions/overture-projection/scripts/lib/skills.js b/.github/actions/overture-projection/scripts/lib/skills.js new file mode 100644 index 0000000..d0e0d77 --- /dev/null +++ b/.github/actions/overture-projection/scripts/lib/skills.js @@ -0,0 +1,111 @@ +/** + * @file lib/skills.js + * @description SKILL.md frontmatter parsing and surface filtering. + * + * Used by load-skills.js. Extracted for unit-testability — the regex logic + * in parseFrontmatter is non-trivial and benefits from isolated testing + * against a variety of frontmatter shapes. + */ + +'use strict'; + +/** + * @typedef {Object} FrontmatterResult + * @property {string} description - Human-readable description used for skill selection. + * @property {string[]} contextFiles - List of `owner/repo:path` context-file refs. + * @property {string[]|null} surfaces - Surfaces the skill targets, or `null` if the field + * is absent (legacy pass-through behaviour). + */ + +/** + * Parses the YAML frontmatter block at the top of a SKILL.md file. + * + * Extracts three custom fields used by the pr-reviewer pipeline: + * - `description` — plain-text summary passed to the selection model. Handles + * both inline (`description: foo`) and block scalar (`description: >\n foo`) + * forms; strips surrounding quotes if present. + * - `context-files` — YAML list of cross-repo `owner/repo:path` refs to inject into the prompt. + * - `surfaces` — inline bracket-list (e.g. `[pr-reviewer, agent]`) declaring which + * surfaces the skill targets. + * + * A missing `surfaces` field returns `null` (not an empty array) so callers can + * distinguish "no field present" (legacy pass-through) from "explicitly empty list". + * + * @param {string} raw - Full SKILL.md file content including frontmatter. + * @returns {FrontmatterResult} + * + * @example + * parseFrontmatter('---\ndescription: Checks containers\nsurfaces: [pr-reviewer]\n---\n# Body') + * // => { description: 'Checks containers', contextFiles: [], surfaces: ['pr-reviewer'] } + */ +function parseFrontmatter(raw) { + const fmMatch = raw.match(/^---\r?\n([\s\S]*?)\r?\n---/); + if (!fmMatch) return { description: '', contextFiles: [], surfaces: null }; + const fm = fmMatch[1]; + + const descMatch = fm.match(/description:\s*[>|]?\s*\n?([\s\S]*?)(?=\n\w|$)/); + const description = descMatch + ? descMatch[1].replace(/\n\s+/g, ' ').trim().replace(/^['"]|['"]$/g, '') + : ''; + + const cfSection = fm.match(/^context-files:\s*\n((?:\s*-\s*.+\n?)*)/m); + const contextFiles = cfSection + ? [...cfSection[1].matchAll(/^\s*-\s*(.+)$/gm)].map(m => m[1].trim()) + : []; + + // null means no surfaces field — skill passes through unfiltered (legacy default) + const surfacesMatch = fm.match(/^surfaces:\s*\[([^\]]*)\]/m); + const surfaces = surfacesMatch + ? surfacesMatch[1].split(',').map(s => s.trim()).filter(Boolean) + : null; + + return { description, contextFiles, surfaces }; +} + +/** + * @typedef {Object} RawSkill + * @property {string} name - Skill folder name (skill ID). + * @property {string} raw - Full SKILL.md content including frontmatter. + */ + +/** + * @typedef {Object} Skill + * @property {string} name - Skill folder name, used as the skill ID. + * @property {string} description - Frontmatter description for the selection model. + * @property {string[]} contextFiles - Context-file refs (`owner/repo:path`) to fetch post-selection. + * @property {string} raw - Full raw SKILL.md content (frontmatter + body). + */ + +/** + * Filters an array of raw skills to those targeting the `pr-reviewer` surface, + * then maps each to a {@link Skill} object by parsing their frontmatter. + * + * Skills with no `surfaces` field are included unconditionally (legacy + * pass-through). Skills whose `surfaces` list does not include `pr-reviewer` + * are excluded. + * + * @param {RawSkill[]} rawSkills - Skills read from disk before any filtering. + * @returns {Skill[]} Skills that should be considered by the pr-reviewer pipeline. + * + * @example + * filterSkills([ + * { name: 'a', raw: '---\nsurfaces: [pr-reviewer]\n---' }, + * { name: 'b', raw: '---\nsurfaces: [agent]\n---' }, + * { name: 'c', raw: '# no frontmatter' }, + * ]) + * // => [{ name: 'a', ... }, { name: 'c', ... }] — 'b' is excluded + */ +function filterSkills(rawSkills) { + return rawSkills + .filter(s => { + const { surfaces } = parseFrontmatter(s.raw); + if (surfaces === null) return true; + return surfaces.includes('pr-reviewer'); + }) + .map(({ name, raw }) => { + const { description, contextFiles } = parseFrontmatter(raw); + return { name, description, contextFiles, raw }; + }); +} + +module.exports = { parseFrontmatter, filterSkills }; diff --git a/.github/actions/overture-projection/scripts/load-skills.js b/.github/actions/overture-projection/scripts/load-skills.js new file mode 100644 index 0000000..e355a1b --- /dev/null +++ b/.github/actions/overture-projection/scripts/load-skills.js @@ -0,0 +1,67 @@ +/** + * @file load-skills.js + * @description Step 1b — Load skills from disk. + * + * Reads every SKILL.md under $SKILLS_DIR, parses YAML frontmatter, filters to + * skills targeting the `pr-reviewer` surface (or with no surfaces field), and + * writes the result to ai-review-skills.json for later steps. + * + * Context files are NOT fetched here — deferred to Step 3b so only skills that + * survive model selection pay the network cost. + * + * Env vars consumed: + * SKILLS_DIR — absolute path to the skills directory on disk + * RUNNER_TEMP — standard Actions temp dir for inter-step artefacts + * + * Outputs written: + * $RUNNER_TEMP/ai-review-skills.json — Skill[] + * + * @param {import('@actions/github-script').AsyncFunctionArguments} args + */ +module.exports = async ({ core }) => { + const fs = require('fs'); + const path = require('path'); + const { filterSkills } = require('./lib/skills'); + + const skillsDir = process.env.SKILLS_DIR; + + if (!skillsDir || !skillsDir.trim() || !fs.existsSync(skillsDir)) { + core.warning(`⚠️ Skills directory not found: ${skillsDir} — no skills will be loaded.`); + fs.writeFileSync(process.env.RUNNER_TEMP + '/ai-review-skills.json', JSON.stringify([])); + return; + } + + const skillFolders = fs.readdirSync(skillsDir, { withFileTypes: true }) + .filter(e => e.isDirectory()) + .map(e => e.name); + core.info(`🗂️ Found ${skillFolders.length} skill(s): ${skillFolders.join(', ')}`); + + const rawSkills = skillFolders.map(name => ({ + name, + raw: fs.readFileSync(path.join(skillsDir, name, 'SKILL.md'), 'utf-8'), + })); + + const skills = filterSkills(rawSkills); + const skippedCount = rawSkills.length - skills.length; + if (skippedCount > 0) { + const skippedNames = rawSkills + .filter(s => !skills.some(k => k.name === s.name)) + .map(s => s.name) + .join(', '); + core.info(`⏭️ Skipped ${skippedCount} skill(s) not targeting pr-reviewer surface: ${skippedNames}`); + } + + core.startGroup(`📚 Skills loaded (${skills.length} of ${rawSkills.length})`); + for (const skill of skills) { + const cfNote = skill.contextFiles.length > 0 + ? ` — ${skill.contextFiles.length} context file(s) pending selection` + : ''; + core.info(` 📄 ${skill.name}${cfNote}`); + } + core.endGroup(); + + fs.writeFileSync( + path.join(process.env.RUNNER_TEMP, 'ai-review-skills.json'), + JSON.stringify(skills) + ); +}; diff --git a/.github/actions/overture-projection/scripts/post-review.js b/.github/actions/overture-projection/scripts/post-review.js new file mode 100644 index 0000000..6b5ca1b --- /dev/null +++ b/.github/actions/overture-projection/scripts/post-review.js @@ -0,0 +1,190 @@ +/** + * @file post-review.js + * @description Step 4 — Compose prompt and post review. + * + * Assembles the system and user prompts from skills, context files, and PR + * data, calls the review model, then posts the result as a PR comment in one + * of three modes: update (upsert), new (fresh review every run), or dry-run + * (log only). + * + * Env vars consumed: + * AI_TOKEN — GitHub token (github-models) or Anthropic API key (anthropic) + * MODEL_PROVIDER — 'github-models' (default) | 'anthropic' + * MODEL_ID — model ID for the review + * MAX_OUTPUT_TOKENS — max tokens the model may generate (default varies by provider) + * MAX_INPUT_TOKENS — max input tokens available (default varies by provider; see defaults.js) + * SELECTED_SKILLS — JSON array of skill names from Step 3 + * COMMENT_MODE — 'update' (default) | 'new' + * COMMENT_TAG — HTML marker for update-mode comment identification + * DRY_RUN — 'true' to skip posting and print instead + * PR_NUMBER — PR number override (falls back to event payload) + * REPOSITORY — target repo in owner/repo format (falls back to context) + * RUNNER_TEMP — standard Actions temp dir for inter-step artefacts + * + * @param {import('@actions/github-script').AsyncFunctionArguments} args + */ +module.exports = async ({ github, context, core }) => { + const fs = require('fs'); + const path = require('path'); + const { YELLOW, BLUE, RESET, logRateLimit, callChatCompletion } = require('./lib/models'); + const { buildSystemPrompt, buildUserPrompt, buildUserPromptPreamble } = require('./lib/prompt'); + const { resolveRepo, resolvePrNumber } = require('./lib/github'); + const { diffCharBudget, applyFileBudget } = require('./lib/diff'); + const { getProviderDefaults, DEFAULT_PROVIDER } = require('./lib/defaults'); + + const token = process.env.AI_TOKEN; + const provider = process.env.MODEL_PROVIDER || DEFAULT_PROVIDER; + const providerDefaults = getProviderDefaults(provider); + const modelId = process.env.MODEL_ID || providerDefaults.model; + const maxOutputTokens = parseInt(process.env.MAX_OUTPUT_TOKENS) || providerDefaults.maxOutputTokens; + const maxInputTokens = parseInt(process.env.MAX_INPUT_TOKENS) || providerDefaults.maxInputTokens; + const selectedNames = JSON.parse(process.env.SELECTED_SKILLS || '[]'); + + const skills = JSON.parse(fs.readFileSync(path.join(process.env.RUNNER_TEMP, 'ai-review-skills.json'), 'utf-8')); + const prData = JSON.parse(fs.readFileSync(path.join(process.env.RUNNER_TEMP, 'ai-review-diff.json'), 'utf-8')); + const contextBySkill = JSON.parse(fs.readFileSync(path.join(process.env.RUNNER_TEMP, 'ai-review-context.json'), 'utf-8')); + + // ── System prompt ──────────────────────────────────────────────────────────── + + const finalSkills = selectedNames.map(n => skills.find(s => s.name === n)).filter(Boolean); + const systemPrompt = buildSystemPrompt(finalSkills, contextBySkill); + + if (finalSkills.length > 0) { + core.info(`📝 Compressed ${finalSkills.length} selected skill(s): ${finalSkills.map(s => s.name).join(', ')}`); + } + + // ── Dynamic diff budget ─────────────────────────────────────────────────────── + // Now that we have the actual system prompt we can compute how many chars + // remain for the diff blocks. We measure the user-prompt preamble (everything + // except the per-file diff blocks) so the budget accounts for PR metadata too. + + const preambleChars = buildUserPromptPreamble(prData).length; + const systemChars = systemPrompt ? systemPrompt.length : 0; + const nonDiffChars = systemChars + preambleChars; + const charBudget = diffCharBudget(nonDiffChars, maxInputTokens); + const estNonDiffToks = Math.round(nonDiffChars / 4); + + core.info( + `📐 Dynamic diff budget: ${maxInputTokens} input tokens − ${estNonDiffToks} non-diff tokens` + + ` = ${YELLOW}${Math.round(charBudget / 4)} tokens (${charBudget} chars) for diffs${RESET}` + ); + + // Trim files to fit the budget (whole-file dropping, same logic as before). + const allFiles = prData.files; + const { included: trimmedFiles, skipped: dynamicSkipped } = applyFileBudget(allFiles, charBudget); + + if (dynamicSkipped.length > 0) { + core.info(`⚠️ Prompt-budget-skipped: ${dynamicSkipped.map(f => f.filename).join(', ')}`); + } + + // Merge any files already skipped at fetch time with newly skipped files. + const allSkipped = [ + ...(prData.budgetSkippedFiles ?? []), + ...dynamicSkipped.map(f => f.filename), + ]; + + const trimmedPrData = { ...prData, files: trimmedFiles, budgetSkippedFiles: allSkipped }; + + // ── User prompt ────────────────────────────────────────────────────────────── + + const userPrompt = buildUserPrompt(trimmedPrData); + + // ── Model call ─────────────────────────────────────────────────────────────── + + const totalChars = systemChars + userPrompt.length; + const estTokens = Math.round(totalChars / 4); + core.info(`📏 Estimated prompt size: ${YELLOW}~${estTokens} tokens${RESET} (${totalChars} chars)`); + if (estTokens > maxInputTokens) { + core.warning(`⚠️ Estimated prompt (${estTokens} tokens) exceeds safe input budget (${maxInputTokens} tokens). The API call may fail.`); + } + + core.info(`🤖 Calling ${BLUE}${modelId}${RESET} for review…`); + let reviewResult; + try { + reviewResult = await callChatCompletion({ + provider, + token, + model: modelId, + messages: [ + // System prompt is omitted entirely when no skills were selected, which + // lets the model fall back to its built-in code review behaviour. + ...(systemPrompt ? [{ role: 'system', content: systemPrompt }] : []), + { role: 'user', content: userPrompt }, + ], + maxTokens: maxOutputTokens, + temperature: 0.2, + }); + } catch (err) { + core.setFailed(`Model API error: ${err.message}`); + return; + } + + logRateLimit(reviewResult.rawResponse, 'review', core); + core.info(`🪙 Review: ${YELLOW}${reviewResult.usage.input} in + ${reviewResult.usage.output} out = ${reviewResult.usage.total} tokens${RESET}`); + + if (reviewResult.finishReason && reviewResult.finishReason !== 'stop') { + core.warning(`⚠️ Review model finish_reason: ${reviewResult.finishReason} — response may be truncated; consider increasing max_tokens`); + } + + const reviewBody = reviewResult.text; + if (!reviewBody) { + core.setFailed('Model returned an empty response'); + return; + } + + // ── Post review ────────────────────────────────────────────────────────────── + + const { owner, repo } = resolveRepo(process.env.REPOSITORY, context.repo); + const prNumber = resolvePrNumber(context.payload.pull_request?.number, process.env.PR_NUMBER); + if (prNumber === null) { + core.setFailed('Could not resolve PR number from context or PR_NUMBER env var'); + return; + } + const commentMode = process.env.COMMENT_MODE || 'new'; + + /** + * Hidden HTML comment embedded in every 'update' mode comment body. + * Used to locate an existing comment to edit on subsequent workflow runs, + * preventing duplicate review comments accumulating on the PR. + */ + const MARKER = ``; + + const green = '\u001b[32;1m'; + const reset = '\u001b[0m'; + + if (process.env.DRY_RUN === 'true') { + // Print to the Actions log without posting — useful for local act testing + core.startGroup('🔎 AI review (dry run)'); + core.info(reviewBody); + core.endGroup(); + core.notice('🧪 Dry run — review not posted to PR.'); + } else if (commentMode === 'update') { + // Upsert: find an existing comment with our marker and edit it in place, + // or create a new one if this is the first run on this PR. + const markedBody = `${MARKER}\n${reviewBody}`; + const comments = await github.rest.issues.listComments({ + owner, repo, issue_number: prNumber, per_page: 100, + }); + const existing = comments.data.find(c => c.body?.includes(MARKER)); + if (existing) { + const { data: updated } = await github.rest.issues.updateComment({ + owner, repo, comment_id: existing.id, body: markedBody, + }); + core.info(`${green}✅ AI review updated → ${updated.html_url}${reset}`); + } else { + const { data: created } = await github.rest.issues.createComment({ + owner, repo, issue_number: prNumber, body: markedBody, + }); + core.info(`${green}✅ AI review posted → ${created.html_url}${reset}`); + } + } else { + // 'new' mode: always create a fresh PR review event + const { data: review } = await github.rest.pulls.createReview({ + owner, repo, + pull_number: prNumber, + body: reviewBody, + event: 'COMMENT', + }); + core.info(`${green}✅ AI review posted → ${review.html_url}${reset}`); + } +}; diff --git a/.github/actions/overture-projection/scripts/select-skills.js b/.github/actions/overture-projection/scripts/select-skills.js new file mode 100644 index 0000000..9d4d82e --- /dev/null +++ b/.github/actions/overture-projection/scripts/select-skills.js @@ -0,0 +1,119 @@ +/** + * @file select-skills.js + * @description Step 3 — Select applicable skills via a fast model. + * + * Splits the loaded skill list into always-skills (included unconditionally) + * and optional skills (submitted to the selection model). The model responds + * with a JSON object nominating which optional skills apply and a per-skill + * reasoning sentence. + * + * The final skill set (always + selected) is written to the `selected` step + * output as a JSON array for consumption by Steps 3b and 4. + * + * Env vars consumed: + * AI_TOKEN — GitHub token (github-models) or Anthropic API key (anthropic) + * MODEL_PROVIDER — 'github-models' (default) | 'anthropic' + * ALWAYS_SKILLS — comma-separated skill names to always include + * SELECTION_MODEL_ID — model ID for skill selection (fast/cheap) + * CHANGED_PATHS — newline-separated file paths from Step 2 + * RUNNER_TEMP — standard Actions temp dir for inter-step artefacts + * + * Step outputs set: + * selected — JSON array of skill names (always-skills first) + * + * @param {import('@actions/github-script').AsyncFunctionArguments} args + */ +module.exports = async ({ core }) => { + const fs = require('fs'); + const path = require('path'); + const { YELLOW, BLUE, RESET, logRateLimit, callChatCompletion } = require('./lib/models'); + const { getProviderDefaults, DEFAULT_PROVIDER } = require('./lib/defaults'); + + const token = process.env.AI_TOKEN; + const provider = process.env.MODEL_PROVIDER || DEFAULT_PROVIDER; + const alwaysSkills = new Set( + (process.env.ALWAYS_SKILLS || '').split(',').map(s => s.trim()).filter(Boolean) + ); + const selectionModelId = process.env.SELECTION_MODEL_ID || + getProviderDefaults(provider).selectionModel; + const changedPaths = process.env.CHANGED_PATHS || ''; + + const skills = JSON.parse(fs.readFileSync(path.join(process.env.RUNNER_TEMP, 'ai-review-skills.json'), 'utf-8')); + const prData = JSON.parse(fs.readFileSync(path.join(process.env.RUNNER_TEMP, 'ai-review-diff.json'), 'utf-8')); + + // Partition skills into always-included and model-selectable groups + const alwaysNames = skills.filter(s => alwaysSkills.has(s.name)).map(s => s.name); + const selectable = skills.filter(s => !alwaysSkills.has(s.name)); + + /** @type {string[]} Names of optional skills chosen by the selection model. */ + let selectedNames = []; + + if (selectable.length > 0) { + core.info(`🔍 Selecting from ${selectable.length} optional skill(s) using ${BLUE}${selectionModelId}${RESET}`); + + // One-line summary per skill fed to the selection model + const indexSummary = selectable.map(s => `- ${s.name}: ${s.description}`).join('\n'); + + let result; + try { + result = await callChatCompletion({ + provider, + token, + model: selectionModelId, + messages: [ + { + role: 'system', + content: 'Select which review skills apply to this pull request. Respond with JSON only: {"skills": ["name"], "reasoning": {"name": "one sentence why included or excluded"}}. Include a reasoning entry for every available skill whether selected or not. Empty skills array if none apply.', + }, + { + role: 'user', + content: + `PR title: ${prData.title}\n\n` + + `PR description: ${prData.body || '(none)'}\n\n` + + `Changed files:\n${changedPaths}\n\n` + + `Optional skills:\n${indexSummary}`, + }, + ], + maxTokens: 600, + temperature: 0.1, + jsonMode: true, + }); + } catch (err) { + core.warning(`⚠️ Skill selection failed (${err.status ?? 'network'}) — using always-skills only: ${err.message}`); + result = null; + } + + if (result) { + logRateLimit(result.rawResponse, 'selection', core); + core.info(`🪙 Selection: ${YELLOW}${result.usage.input} in + ${result.usage.output} out = ${result.usage.total} tokens${RESET}`); + + if (result.finishReason && result.finishReason !== 'stop') { + core.warning(`⚠️ Selection model finish_reason: ${result.finishReason}`); + } + + try { + const parsed = JSON.parse(result.text); + // Guard against the model hallucinating skill names not in the index + selectedNames = (parsed.skills || []).filter(n => selectable.some(s => s.name === n)); + + // Log per-skill reasoning for every selectable skill (selected or not) + const reasoning = parsed.reasoning || {}; + core.startGroup('🧠 Skill selection reasoning'); + for (const skill of selectable) { + const reason = reasoning[skill.name] || '(no reasoning provided)'; + const chosen = selectedNames.includes(skill.name) ? '✅ selected' : '⏭️ skipped'; + core.info(`${chosen} ${skill.name}: ${reason}`); + } + core.endGroup(); + } catch { + core.warning(`⚠️ Skill selection returned non-JSON: ${result.text}`); + } + } + } + + // Always-skills come first so pr-review (the structural skill) is always the + // first system-prompt block regardless of which optional skills were chosen. + const finalNames = [...alwaysNames, ...selectedNames]; + core.info(`✅ Final skill set: ${finalNames.join(', ') || '(none)'}`); + core.setOutput('selected', JSON.stringify(finalNames)); +}; diff --git a/.github/workflows/overture-projection-tests.yml b/.github/workflows/overture-projection-tests.yml new file mode 100644 index 0000000..de49637 --- /dev/null +++ b/.github/workflows/overture-projection-tests.yml @@ -0,0 +1,40 @@ +--- +# Overture PRojection — unit test workflow. +# +# Runs the Node built-in test runner against all lib/__tests__/*.test.js files +# whenever the action's scripts are modified. +# +name: Overture PRojection Tests + +on: + pull_request: + paths: + - .github/actions/overture-projection/scripts/** + +permissions: + contents: read # Required to checkout the repository + +# Cancel in-progress test runs when new commits are pushed. +concurrency: + group: test-${{ github.event.pull_request.number || github.run_id }} + cancel-in-progress: true + +jobs: + test: + name: Unit tests + runs-on: ubuntu-latest + + steps: + - name: Checkout + uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + + - name: Setup Node + uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0 + with: + node-version: 24 + + - name: Run tests + run: | + node --test ".github/actions/overture-projection/scripts/lib/__tests__/*.test.js" diff --git a/.github/workflows/overture-projection.yml b/.github/workflows/overture-projection.yml new file mode 100644 index 0000000..8f0cef7 --- /dev/null +++ b/.github/workflows/overture-projection.yml @@ -0,0 +1,199 @@ +--- +# Overture PRojection — AI PR review workflow. +# +# Triggers automatically on pull_request events, or manually via workflow_dispatch +# to review any open PR by number. +# +# ── Local testing with act ──────────────────────────────────────────────────── +# +# Use workflow_dispatch so --input flags are honoured. Do NOT pass -e with a +# pull_request event payload — act ignores --input when the event file contains +# a different event type. All PR targeting is done via --input pr_number and +# --input repository instead. +# +# dry-run is set automatically (ACT=true env var is set by act). +# +# GitHub Models — PowerShell: +# act workflow_dispatch ` +# -s GITHUB_TOKEN=$(gh auth token) ` +# -P ubuntu-slim=catthehacker/ubuntu:act-latest ` +# --input pr_number= ` +# --input repository=OvertureMaps/ +# +# GitHub Models — bash/zsh: +# act workflow_dispatch \ +# -s GITHUB_TOKEN=$(gh auth token) \ +# -P ubuntu-slim=catthehacker/ubuntu:act-latest \ +# --input pr_number= \ +# --input repository=OvertureMaps/ +# +# Anthropic — PowerShell: +# act workflow_dispatch ` +# -s GITHUB_TOKEN=$(gh auth token) ` +# -s ANTHROPIC_API_KEY=sk-ant-... ` +# -P ubuntu-slim=catthehacker/ubuntu:act-latest ` +# --input pr_number= ` +# --input repository=OvertureMaps/ ` +# --input model_provider=anthropic +# # model, max_input_tokens, max_output_tokens, selection_model all default +# # automatically for anthropic (claude-opus-4-6, 190000, 4096, claude-haiku-4-6) +# +# Anthropic — bash/zsh: +# act workflow_dispatch \ +# -s GITHUB_TOKEN=$(gh auth token) \ +# -s ANTHROPIC_API_KEY=sk-ant-... \ +# -P ubuntu-slim=catthehacker/ubuntu:act-latest \ +# --input pr_number= \ +# --input repository=OvertureMaps/ \ +# --input model_provider=anthropic +# # model, max_input_tokens, max_output_tokens, selection_model all default +# # automatically for anthropic (claude-opus-4-6, 190000, 4096, claude-haiku-4-6) +# +# ── Manual trigger (no act) ─────────────────────────────────────────────────── +# +# gh workflow run overture-projection.yml \ +# -f pr_number= \ +# -f dry_run=true +# +# ── Required permissions ────────────────────────────────────────────────────── +# +# Token scopes: repo (read), pull-requests (write), issues (read), models (read) +# Org secret OVERTURE_PROJECTION_APP_PEM used for cross-repo context file reads. +# +name: Overture PRojection + +on: + # Trigger on real PRs in this repo + pull_request: + types: [opened, synchronize] + + # Reusable workflow for calling from other repositories + workflow_call: + inputs: + repository: + description: "Target repo in owner/repo format" + required: true + type: string + pr_number: + description: PR number to review + required: true + type: number + devex_ref: + description: omf-devex ref to load skills from + required: false + type: string + default: main + model: + description: "Review model ID (e.g. gpt-4.1 or claude-opus-4-6). Defaults automatically by provider when not set." + required: false + type: string + default: "" + selection_model: + description: "Skill selection model ID. Defaults automatically by provider when not set." + required: false + type: string + default: "" + model_provider: + description: "'github-models' (default) or 'anthropic'" + required: false + type: string + default: github-models + max_input_tokens: + description: "Max input tokens. Defaults automatically by provider when not set (6200 for github-models, 190000 for anthropic)." + required: false + type: string + default: "" + max_output_tokens: + description: "Max tokens the model may generate. Defaults automatically by provider when not set (1500 for github-models, 4096 for anthropic)." + required: false + type: string + default: "" + dry_run: + description: Print composed prompt without posting a review comment + required: false + type: boolean + default: false + secrets: + OVERTURE_PROJECTION_APP_PEM: + description: App private key for cross-repo context file reads + required: true + ANTHROPIC_API_KEY: + description: Anthropic API key (required only if using anthropic model_provider) + required: false + + # Manual trigger: supply any PR number from this repo to test against + workflow_dispatch: + inputs: + repository: + description: "Target repo in owner/repo format (default: this repo)" + required: false + default: "" + pr_number: + description: PR number to review + required: true + type: number + devex_ref: + description: omf-devex ref to load skills from + required: false + default: main + model: + description: "Review model ID (e.g. gpt-4.1 or claude-opus-4-6). Defaults automatically by provider when not set." + required: false + default: "" + selection_model: + description: "Skill selection model ID. Defaults automatically by provider when not set." + required: false + default: "" + model_provider: + description: "'github-models' (default) or 'anthropic'" + required: false + default: github-models + max_input_tokens: + description: "Max input tokens. Defaults automatically by provider when not set (6200 for github-models, 190000 for anthropic)." + required: false + default: "" + max_output_tokens: + description: "Max tokens the model may generate. Defaults automatically by provider when not set (1500 for github-models, 4096 for anthropic)." + required: false + default: "" + dry_run: + description: Print composed prompt without posting a review comment + required: false + type: boolean + default: false + +permissions: + contents: read # checkout omf-devex skills + +jobs: + overture-projection: + name: Overture PRojection + runs-on: ubuntu-slim + + # One active run per PR; cancel in-progress if a new commit is pushed. + concurrency: + group: overture-projection-${{ github.event.pull_request.number || inputs.pr_number || github.run_id }} + cancel-in-progress: true + + permissions: + pull-requests: write # post/update review comment + issues: read # closingIssuesReferences GraphQL query + models: read # GitHub Models API (not needed for anthropic provider) + + steps: + - name: Run Overture PRojection + uses: OvertureMaps/workflows/.github/actions/overture-projection@main # zizmor: ignore[unpinned-uses] -- self-referential; SHA updated in follow-up commit + with: + always-skills: "pr-review" + repository: ${{ github.event_name == 'pull_request' && github.event.repository.full_name || inputs.repository || github.event.repository.full_name }} + pr-number: ${{ github.event_name == 'pull_request' && github.event.pull_request.number || inputs.pr_number || github.event.pull_request.number }} + devex-ref: ${{ github.event_name == 'pull_request' && 'main' || inputs.devex_ref || 'main' }} + model: ${{ github.event_name != 'pull_request' && inputs.model || '' }} + selection-model: ${{ github.event_name != 'pull_request' && inputs.selection_model || '' }} + model-provider: ${{ github.event_name == 'pull_request' && 'github-models' || inputs.model_provider || 'github-models' }} + max-input-tokens: ${{ github.event_name != 'pull_request' && inputs.max_input_tokens || '' }} + max-output-tokens: ${{ github.event_name != 'pull_request' && inputs.max_output_tokens || '' }} + dry-run: ${{ github.event_name == 'pull_request' && env.ACT == 'true' || inputs.dry_run == true || inputs.dry_run == 'true' || env.ACT == 'true' }} + github-token: ${{ secrets.GITHUB_TOKEN }} + app-private-key: ${{ secrets.OVERTURE_PROJECTION_APP_PEM }} # zizmor: ignore[secrets-outside-env] -- org secret, not environment-gated + anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} # zizmor: ignore[secrets-outside-env] -- org secret, not environment-gated