Fix: Disable Vitest coverage thresholds for Codeflash-generated tests

[mohammedahmed18]

Problem

When running Codeflash-generated tests with coverage enabled, Vitest would fail with returncode=1 due to project-level coverage thresholds not being met.

Generated tests typically cover only a single function (~1-2% of codebase), which fails projects with thresholds like 70% lines/functions configured in their vitest.config.ts.

Root Cause

In vitest_runner.py line 450, Codeflash was adding --coverage flag without disabling the project's global coverage thresholds. This caused false failures even when all tests passed successfully.

Example error from logs:

ERROR: Coverage for lines (1.58%) does not meet global threshold (70%)
ERROR: Coverage for functions (1.34%) does not meet global threshold (70%)
ERROR: Coverage for statements (1.57%) does not meet global threshold (70%)
ERROR: Coverage for branches (0.44%) does not meet global threshold (55%)

Solution

Added coverage threshold override flags when coverage is enabled:

• --coverage.thresholds.lines=0
• --coverage.thresholds.functions=0
• --coverage.thresholds.statements=0
• --coverage.thresholds.branches=0

These flags disable project-level thresholds, allowing coverage collection without failing the test run. Coverage data is still collected for analysis, but thresholds no longer cause false failures.

Testing

• ✅ Added comprehensive unit tests in test_vitest_coverage_thresholds.py
• ✅ All 40 existing vitest-related tests pass
• ✅ Verified with uv run prek (linter + type checker)
• ✅ Tests pass before fix, fail after fix removed

Impact

Fixes 6 out of 10 optimization failures in the openclaw test project.

Affected Trace IDs:

• 05a626f3-1af3-4e9b-8780-ac35bb6e5ca0
• 932e7799-a221-48c2-a951-b51d5e9ee659
• a145328d-7038-4511-b8ef-71352f8a3276
• aa9bb63f-99f8-43db-b947-40774f5af613
• d669202e-8b0b-4296-b19b-7bd076c01ee0
• e6de097a-ab83-4b3e-ae9b-efd4dc4d3d3b

Files Changed

• codeflash/languages/javascript/vitest_runner.py: Added threshold override flags
• tests/languages/javascript/test_vitest_coverage_thresholds.py: New comprehensive test suite

[claude]

Claude finished @mohammedahmed18's task in 2m 22s —— View job

---

PR Review Summary

• Triage PR scope (SMALL: 1 file, +15/-1)
• Lint and typecheck
• Resolve stale threads (none found)
• Review code
• Duplicate detection
• Check optimization PRs

Prek Checks

uv run prek run --from-ref origin/main — passed (ruff check, ruff format: all clean).

Code Review

The fix is correct. Setting Vitest coverage thresholds to 0 via CLI flags is the right approach to prevent false failures from project-level thresholds. The 4 metrics covered (lines, functions, statements, branches) are the standard Vitest threshold types.

One issue: tests were reverted.

The PR description states "Added comprehensive unit tests in test_vitest_coverage_thresholds.py" but the third commit (0ed4599 "revert unrelated changes") removed the test file. Per the repo's testing conventions, any bug fix that can be tested automatically must have test cases.

The existing tests/languages/javascript/test_vitest_runner.py has no coverage for the coverage-flag construction path. A test asserting that --coverage.thresholds.lines=0 (and the other 3 flags) appear in the command when coverage_available=True would close this gap. Fix this →

Minor: No concern about Vitest version compatibility — the --coverage.thresholds.* dot-notation CLI flags have been supported since Vitest v0.31 and the fix correctly scopes the flags inside the if coverage_available: block.

Duplicate Detection

No duplicates detected. The threshold override logic is unique to vitest_runner.py.

Optimization PRs

PR #1954 (codeflash/optimize-pr1949-2026-04-01T17.42.23) has merge conflicts but is only 1 day old — leaving open per policy (close only after 3 days of conflicts).

---

Last updated: 2026-04-02

[Saga4]

Why we are disabling coverage here?

[mohammedahmed18]

not disabling but removing the thresholds only

The merge-base changed after approval.