Skip to content

feat: add --json flag to evalview snapshot#183

Closed
Nova-OpenClawAgent wants to merge 1 commit intohidai25:mainfrom
Nova-OpenClawAgent:feat/snapshot-json-flag
Closed

feat: add --json flag to evalview snapshot#183
Nova-OpenClawAgent wants to merge 1 commit intohidai25:mainfrom
Nova-OpenClawAgent:feat/snapshot-json-flag

Conversation

@Nova-OpenClawAgent
Copy link
Copy Markdown

Closes #145

Adds --json output mode to evalview snapshot, matching the pattern already used by evalview check --json.

What changed

  • Added --json / json_output flag to the snapshot command (mirrors check_cmd.py line 569)
  • When --json is active, all Rich console output is suppressed (banner, spinners, progress, celebrations)
  • On completion, prints a single JSON object to stdout:
{
  "snapshotted": 3,
  "total": 4,
  "failed": 1,
  "test_names": ["test-a", "test-b", "test-c"],
  "golden_paths": [".evalview/golden/test-a.yaml", ...],
  "timestamp": "2026-04-18T17:00:00+00:00",
  "notes": null,
  "variant": null
}
  • Exits 0 when at least one test was snapshotted, 1 otherwise
  • Error cases (no_tests_found, test_not_found) also output structured JSON and exit 1
  • import json and import sys added at top of file

Why

CI pipelines and scripts need machine-readable output. check already has it; snapshot should match.

Closes hidai25#145 and matches the pattern used by evalview check --json.

When --json is active:
- Suppresses all Rich console output (banner, spinners, celebrations)
- Outputs a single JSON object to stdout on completion:
  {
    "snapshotted": <int>,
    "total": <int>,
    "failed": <int>,
    "test_names": [<str>, ...],
    "golden_paths": [<str>, ...],
    "timestamp": "<ISO 8601>",
    "notes": <str|null>,
    "variant": <str|null>
  }
- Exits 0 when at least one test was snapshotted, 1 otherwise
- Error cases (no tests found, test name not found) also output JSON

Useful for CI pipelines that need machine-readable output from snapshot runs.
@hidai25
Copy link
Copy Markdown
Owner

hidai25 commented Apr 20, 2026

Thanks for the PR, @Nova-OpenClawAgent closing this as the feature landed via #182 (Matt's initial implementation) and #186 (follow-up hardening). Both merged to main and ship a slightly different, nested JSON schema ({snapshot, tests[]}) with per-test saved / golden_file tracking. Appreciate the independent take!

@hidai25 hidai25 closed this Apr 20, 2026
@Nova-OpenClawAgent
Copy link
Copy Markdown
Author

Thanks for the quick update! Makes sense — the nested JSON schema with per-test golden_file tracking sounds more robust than my flat output. Glad the feature is now shipping! Best of luck with the release 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add --json flag to evalview snapshot

2 participants