Skip to content

feat: add math cairo tests under vm/src/tests/cairo_test_suite#2379

Open
naor-starkware wants to merge 31 commits intonaor/feat/add_test_helpersfrom
naor/feat/add_math_cairo_test
Open

feat: add math cairo tests under vm/src/tests/cairo_test_suite#2379
naor-starkware wants to merge 31 commits intonaor/feat/add_test_helpersfrom
naor/feat/add_math_cairo_test

Conversation

@naor-starkware
Copy link
Copy Markdown
Collaborator

@naor-starkware naor-starkware commented Mar 29, 2026 •
edited by phil-starkware
Loading

TITLE

Description

Description of the pull request changes and motivation.

Checklist

  • Linked to Github Issue
  • Unit tests added
  • Integration tests added.
  • This change requires new documentation.
    • Documentation has been added/updated.
    • CHANGELOG has been updated.

This change is Reviewable

@naor-starkware naor-starkware mentioned this pull request Mar 29, 2026
Closed
6 tasks
@naor-starkware Graphite App
Copy link
Copy Markdown
Collaborator Author

naor-starkware commented Mar 29, 2026 •
edited
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

This was referenced Mar 29, 2026
Closed
Merged
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 29, 2026 •
edited
Loading

Benchmark Results for unmodified programs 🚀

Command Mean [s] Min [s] Max [s] Relative
base big_factorial 2.096 ± 0.017 2.080 2.128 1.00 ± 0.02
head big_factorial 2.095 ± 0.030 2.075 2.168 1.00
Command Mean [s] Min [s] Max [s] Relative
base big_fibonacci 2.020 ± 0.011 2.012 2.049 1.00 ± 0.01
head big_fibonacci 2.017 ± 0.006 2.010 2.028 1.00
Command Mean [s] Min [s] Max [s] Relative
base blake2s_integration_benchmark 7.404 ± 0.133 7.297 7.729 1.01 ± 0.02
head blake2s_integration_benchmark 7.358 ± 0.085 7.291 7.579 1.00
Command Mean [s] Min [s] Max [s] Relative
base compare_arrays_200000 2.151 ± 0.019 2.137 2.198 1.01 ± 0.01
head compare_arrays_200000 2.140 ± 0.014 2.115 2.162 1.00
Command Mean [s] Min [s] Max [s] Relative
base dict_integration_benchmark 1.404 ± 0.006 1.397 1.415 1.00
head dict_integration_benchmark 1.413 ± 0.010 1.402 1.437 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base field_arithmetic_get_square_benchmark 1.214 ± 0.004 1.208 1.220 1.00
head field_arithmetic_get_square_benchmark 1.215 ± 0.004 1.207 1.219 1.00 ± 0.00
Command Mean [s] Min [s] Max [s] Relative
base integration_builtins 7.475 ± 0.086 7.414 7.703 1.00 ± 0.01
head integration_builtins 7.438 ± 0.051 7.392 7.565 1.00
Command Mean [s] Min [s] Max [s] Relative
base keccak_integration_benchmark 7.561 ± 0.134 7.491 7.937 1.01 ± 0.02
head keccak_integration_benchmark 7.518 ± 0.016 7.479 7.535 1.00
Command Mean [s] Min [s] Max [s] Relative
base linear_search 2.123 ± 0.015 2.112 2.160 1.00
head linear_search 2.123 ± 0.020 2.108 2.170 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base math_cmp_and_pow_integration_benchmark 1.465 ± 0.008 1.459 1.488 1.00 ± 0.01
head math_cmp_and_pow_integration_benchmark 1.465 ± 0.012 1.452 1.496 1.00
Command Mean [s] Min [s] Max [s] Relative
base math_integration_benchmark 1.441 ± 0.013 1.431 1.471 1.00
head math_integration_benchmark 1.462 ± 0.080 1.432 1.689 1.01 ± 0.06
Command Mean [s] Min [s] Max [s] Relative
base memory_integration_benchmark 1.192 ± 0.003 1.187 1.196 1.00
head memory_integration_benchmark 1.195 ± 0.010 1.188 1.221 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base operations_with_data_structures_benchmarks 1.519 ± 0.004 1.514 1.527 1.00
head operations_with_data_structures_benchmarks 1.522 ± 0.008 1.514 1.538 1.00 ± 0.01
Command Mean [ms] Min [ms] Max [ms] Relative
base pedersen 527.5 ± 1.0 526.0 529.0 1.00
head pedersen 528.9 ± 2.5 526.6 534.6 1.00 ± 0.01
Command Mean [ms] Min [ms] Max [ms] Relative
base poseidon_integration_benchmark 601.5 ± 2.8 599.2 609.1 1.00 ± 0.01
head poseidon_integration_benchmark 600.9 ± 2.6 598.6 607.5 1.00
Command Mean [s] Min [s] Max [s] Relative
base secp_integration_benchmark 1.796 ± 0.007 1.788 1.813 1.00
head secp_integration_benchmark 1.797 ± 0.008 1.786 1.810 1.00 ± 0.01
Command Mean [ms] Min [ms] Max [ms] Relative
base set_integration_benchmark 654.7 ± 1.4 652.2 656.0 1.00
head set_integration_benchmark 655.9 ± 2.4 653.4 661.5 1.00 ± 0.00
Command Mean [s] Min [s] Max [s] Relative
base uint256_integration_benchmark 4.193 ± 0.023 4.173 4.249 1.00
head uint256_integration_benchmark 4.194 ± 0.021 4.162 4.247 1.00 ± 0.01

@naor-starkware naor-starkware changed the base branch from naor/feat/add_test_helpers to graphite-base/2379 March 29, 2026 13:00
@naor-starkware naor-starkware force-pushed the naor/feat/add_math_cairo_test branch from 421425b to 981fce6 Compare March 29, 2026 13:01
@naor-starkware naor-starkware changed the base branch from graphite-base/2379 to naor/refactor/Makefile-include-test_cairo_suite_target March 29, 2026 13:01
@naor-starkware naor-starkware force-pushed the naor/refactor/Makefile-include-test_cairo_suite_target branch from 646dd9e to 2bc600b Compare March 29, 2026 13:13
@naor-starkware naor-starkware force-pushed the naor/feat/add_math_cairo_test branch from 981fce6 to 4e1af25 Compare March 29, 2026 13:13
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 29, 2026 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.07%. Comparing base (f0bb2d0) to head (d6a6c58).

Additional details and impacted files 
@@                                   Coverage Diff                                   @@
##           naor/refactor/Makefile-include-test_cairo_suite_target    #2379   +/-   ##
=======================================================================================
  Coverage                                                   96.07%   96.07%           
=======================================================================================
  Files                                                         107      107           
  Lines                                                       37852    37852           
=======================================================================================
  Hits                                                        36366    36366           
  Misses                                                       1486     1486

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

naor-starkware added a commit that referenced this pull request Mar 29, 2026
@naor-starkware @claude
chore: update CHANGELOG for PR #2379
fa0fe77 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware naor-starkware force-pushed the naor/feat/add_math_cairo_test branch from 4e1af25 to fa0fe77 Compare March 29, 2026 14:19
@naor-starkware naor-starkware force-pushed the naor/refactor/Makefile-include-test_cairo_suite_target branch from 2bc600b to 6fd8964 Compare March 29, 2026 14:19
naor-starkware added a commit that referenced this pull request Mar 29, 2026
@naor-starkware @claude
chore: update CHANGELOG for PR #2379
fd27b62 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware naor-starkware force-pushed the naor/feat/add_math_cairo_test branch from fa0fe77 to fd27b62 Compare March 29, 2026 16:11
@naor-starkware naor-starkware force-pushed the naor/refactor/Makefile-include-test_cairo_suite_target branch 2 times, most recently from c106a57 to 106cbb4 Compare March 30, 2026 08:43
naor-starkware added a commit that referenced this pull request Mar 30, 2026
@naor-starkware @claude
chore: update CHANGELOG for PR #2379
1e0b48e 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware naor-starkware force-pushed the naor/refactor/Makefile-include-test_cairo_suite_target branch from 106cbb4 to ff56720 Compare March 30, 2026 09:32
@naor-starkware naor-starkware force-pushed the naor/feat/add_math_cairo_test branch from fd27b62 to 1e0b48e Compare March 30, 2026 09:32
@naor-starkware naor-starkware marked this pull request as ready for review March 30, 2026 10:35
naor-starkware added a commit that referenced this pull request Apr 6, 2026
@naor-starkware @claude
chore: update CHANGELOG for PR #2379
842e9f9 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
naor-starkware added a commit that referenced this pull request Apr 6, 2026
@naor-starkware @claude
refactor: replace function_runner cfg gates with test_utils in cairo_…
fd5cce6 
…test_suite

Also fix duplicate and stale CHANGELOG entries for PRs #2377-#2379.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware naor-starkware force-pushed the naor/feat/add_math_cairo_test branch from 1e0b48e to fd5cce6 Compare April 6, 2026 09:53
@naor-starkware naor-starkware force-pushed the naor/refactor/Makefile-include-test_cairo_suite_target branch from ff56720 to 6c03214 Compare April 6, 2026 09:53
naor-starkware and others added 4 commits April 6, 2026 20:39
@naor-starkware @claude
feat: add test_helpers module (error_utils, test_utils) behind functi…
f97e022 
…on_runner flag

- Create vm/src/test_helpers/ with error_utils.rs and test_utils.rs
- Move from cairo_test_suite/ (fix filename typo: utlis → utils)
- Fix crate:: import paths (were cairo_vm:: when outside the crate)
- Fix $crate in macro_export macro (clippy::crate_in_macro_def)
- Simplify load_cairo_program! path using with_file_name()
- Gate module behind function_runner feature in lib.rs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
test(test_helpers): add unit tests for assert_mr_eq!, load_cairo_prog…
0edf8b8 
…ram! and error_utils checkers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
style: cargo fmt
298b0c0 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
test: add conversion-failure and allow-large-err fixes to test_helpers
26c995c 
- Add AlwaysFailConversion helper + 2 tests for assert_mr_eq! unwrap_or_else
  panic branch (no-message and message variants)
- Allow clippy::result_large_err on hint_err test helper

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
naor-starkware added a commit that referenced this pull request Apr 7, 2026
@naor-starkware @claude
chore: update CHANGELOG for PR #2379
9e93e67 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
naor-starkware added a commit that referenced this pull request Apr 7, 2026
@naor-starkware @claude
refactor: replace function_runner cfg gates with test_utils in cairo_…
a508d41 
…test_suite

Also fix duplicate and stale CHANGELOG entries for PRs #2377-#2379.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware naor-starkware force-pushed the naor/feat/add_math_cairo_test branch from 985c1fe to f0f279f Compare April 7, 2026 07:11
@naor-starkware naor-starkware changed the base branch from graphite-base/2379 to naor/refactor/Makefile-include-test_cairo_suite_target April 7, 2026 07:11
naor-starkware and others added 21 commits April 9, 2026 12:13
@naor-starkware @claude
refactor: replace assert_mr_eq! macro with generic function
53a159f 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
chore: apply cargo fmt
3e693fb 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
fix: expect_diff_type_comparison and expect_diff_index_comp unwrap Hi…
9b1b8f5 
…nt::Internal

These errors arrive wrapped as Hint(Internal(...)) since they originate
inside hint execution, not as bare VirtualMachineError variants.
Remove now-unused expect_vm_error helper and vm_err test helper.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
feat(makefile,ci): add cairo_test_suite_programs target and CI integr…
c4a8e04 
…ation

Makefile:
- Add CAIRO_TEST_SUITE_ROOT, CAIRO_TEST_SUITE_FILES, COMPILED_CAIRO_TEST_SUITE
  variables to track cairo test suite sources
- Add pattern rule to compile each .cairo to .json via cairo-compile
- Add cairo_test_suite_programs target
- Hook cairo_test_suite_programs into the test target
- Add cairo_test_suite_programs to .PHONY

CI (rust.yml):
- Add cairo_test_suite_programs to the build-programs matrix
- Add vm/src/tests/cairo_test_suite/**/*.json to CAIRO_PROGRAMS_PATH cache
- Update all program cache keys to hash test suite source files
- Add function_runner feature flag to the tests job

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
chore: update CHANGELOG for PR #2380
828320a 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware
update CHANGELOG.md
d8f7f68
@naor-starkware @claude
fix(ci): remove stale function_runner feature flag from coverage command
f0bb2d0 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
feat: add math cairo tests under vm/src/tests/cairo_test_suite
aa8d90d 
- Create cairo_test_suite module skeleton (mod.rs, test_math/mod.rs)
- Add main_math_test.cairo stub (imports all math.cairo functions)
- Add math_test_utils.rs (RC_BOUND, MAX_DIV, is_quad_residue_mod_prime)
- Move test_math_cairo.rs from cairo_test_suite/, update all imports
  - cairo_vm:: → crate:: for all internal paths
  - crate::error_utils → crate::test_helpers::error_utils
  - cairo_function_runner → function_runner module path
  - Fix 5x runner.runner.vm → runner.vm (old struct design remnant)
- Register cairo_test_suite mod behind function_runner feature

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
fix: correct CairoFunctionRunner usage in test_math_cairo.rs
15d89e7 
- new() → new_for_testing() (type alias has no 1-arg constructor)
- runner.get_return_values() → runner.vm.get_return_values() (method is on VirtualMachine)
- Add missing macro imports: assert_mr_eq! and load_cairo_program!

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
fix(cairo_test_suite): rename cairo file and add horner_eval import
902b78a 
Rename main_math_test.cairo → math_test.cairo for clarity.
Add `from starkware.cairo.common.math_utils import horner_eval` so the
compiled JSON includes horner_eval (used in test_horner_eval).
Update the load_cairo_program! call to reference math_test.json.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
fix(cairo_test_suite): import all functions from math.cairo
e41b80f 
Add missing assert_not_nullptr, safe_div, safe_mult.
Move horner_eval and assert_is_power_of_2 into the math.cairo import
block (both are defined there, not in math_utils).
Match declaration order to match math.cairo.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
refactor(cairo_test_suite): remove redundant numeric type suffixes
c775891 
- Remove isize from MaybeRelocatable tuple literals (inferred from From impl)
- Remove usize from add_usize() calls (inferred from param type)
- Remove _i64 suffixes in split_int cases (inferred from param types)
- Remove usize from shift/bit operations
- Use type annotation on variable instead of inline i64 suffixes on literals

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
fix: remove assert_not_nullptr import not available in cairo-lang 0.13.5
a3f510b 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
test(math_test_utils): add unit tests for is_quad_residue_mod_prime
78fdc0a 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
style: cargo fmt
db0283d 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
chore: update CHANGELOG for PR #2379
fc935bc 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
refactor: replace function_runner cfg gates with test_utils in cairo_…
8c01849 
…test_suite

Also fix duplicate and stale CHANGELOG entries for PRs #2377-#2379.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
fix: replace CairoFunctionRunner with CairoRunner in test_math_cairo
454d393 
CairoFunctionRunner type alias was removed, use CairoRunner directly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware
update CHANGELOG.md
9326c23
@naor-starkware @claude
refactor: migrate assert_mr_eq! macro calls to function in test suite
fa68c82 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware @claude
chore: apply cargo fmt
d6a6c58 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@naor-starkware naor-starkware force-pushed the naor/feat/add_math_cairo_test branch from f0f279f to d6a6c58 Compare April 9, 2026 09:46
@naor-starkware naor-starkware force-pushed the naor/refactor/Makefile-include-test_cairo_suite_target branch from b2799c2 to f0bb2d0 Compare April 9, 2026 09:46
Base automatically changed from naor/refactor/Makefile-include-test_cairo_suite_target to naor/feat/add_test_helpers April 19, 2026 07:09
@OmriEshhar1 OmriEshhar1 force-pushed the naor/feat/add_test_helpers branch from 67106b9 to dc5490c Compare April 19, 2026 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@naor-starkware