Skip to content

Show token cost breakdown in check output#185

Open
gauravxthakur wants to merge 4 commits intohidai25:mainfrom
gauravxthakur:token-cost-breakdown
Open

Show token cost breakdown in check output#185
gauravxthakur wants to merge 4 commits intohidai25:mainfrom
gauravxthakur:token-cost-breakdown

Conversation

@gauravxthakur
Copy link
Copy Markdown
Contributor

@gauravxthakur gauravxthakur commented Apr 19, 2026

Description

Add a summary table to evalview check output showing token usage breakdown: input tokens, output tokens, cached tokens, and estimated cost.

Related Issue

Fixes #146

Type of Change

  • New feature (non-breaking change which adds functionality)

Changes Made

What I added:

  • Token summary table: Shows input/output/cached tokens and total cost
  • Baseline comparison: Shows if you're using more/less tokens than before
  • JSON output: Includes the same data when you use --json flag
  • Graceful handling: Won't crash if token data isn't available

Testing

Test Configuration

  • Python Version: 3.13.7
  • OS: Windows

Tests Added/Modified

  • Unit tests added/updated
  • All tests pass locally
  • Test coverage maintained or improved

Manual Testing Steps

Run unit tests to verify token summary functionality: py -m pytest tests/test_check_cmd.py::test_check_json_includes_token_usage_and_cost_summary tests/test_check_cmd.py::test_check_json_omits_token_usage_when_unavailable -v

Checklist

Code Quality

  • My code follows the project's style guidelines
  • I have performed a self-review of my own code

Documentation

  • I have added docstrings to new functions/classes

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Dependencies

  • I have checked that no new dependencies were added unnecessarily

Breaking Changes

None.

Additional Notes

There's a type checking error in the CI/CD pipeline. The functionality works correctly (all tests pass locally), but there's a type annotation issue that needs to be resolved. I would really appreciate some help in identifying and fixing this error.


By submitting this pull request, I confirm that:

  • I have read and followed the CONTRIBUTING.md guidelines
  • I agree to the project's CODE_OF_CONDUCT.md
  • My contribution is my own work and I have the right to contribute it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Show token cost breakdown in check output

1 participant