feat: add test_helpers module (error_utils, test_utils) behind test_utlis flag

[naor-starkware]

TITLE

Description

Description of the pull request changes and motivation.

Checklist

• Linked to Github Issue
• Unit tests added
• Integration tests added.
• This change requires new documentation.
  • Documentation has been added/updated.
  • CHANGELOG has been updated.

---

This change is 

[naor-starkware]

• feat: add math cairo tests under vm/src/tests/cairo_test_suite #2379
• feat(makefile,ci): add cairo_test_suite_programs target and CI integration #2380
• feat: add test_helpers module (error_utils, test_utils) behind test_utlis flag #2381 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[github-actions]

**Hyper Thereading Benchmark results**




hyperfine -r 2 -n "hyper_threading_main threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_main' -n "hyper_threading_pr threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 1
  Time (mean ± σ):     23.055 s ±  0.036 s    [User: 22.337 s, System: 0.715 s]
  Range (min … max):   23.029 s … 23.081 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 1
  Time (mean ± σ):     23.113 s ±  0.205 s    [User: 22.433 s, System: 0.676 s]
  Range (min … max):   22.969 s … 23.258 s    2 runs
 
Summary
  hyper_threading_main threads: 1 ran
    1.00 ± 0.01 times faster than hyper_threading_pr threads: 1




hyperfine -r 2 -n "hyper_threading_main threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_main' -n "hyper_threading_pr threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 2
  Time (mean ± σ):     12.269 s ±  0.016 s    [User: 22.257 s, System: 0.649 s]
  Range (min … max):   12.258 s … 12.280 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 2
  Time (mean ± σ):     12.286 s ±  0.016 s    [User: 22.315 s, System: 0.652 s]
  Range (min … max):   12.275 s … 12.297 s    2 runs
 
Summary
  hyper_threading_main threads: 2 ran
    1.00 ± 0.00 times faster than hyper_threading_pr threads: 2




hyperfine -r 2 -n "hyper_threading_main threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_main' -n "hyper_threading_pr threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 4
  Time (mean ± σ):      9.627 s ±  0.012 s    [User: 34.759 s, System: 0.778 s]
  Range (min … max):    9.618 s …  9.635 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 4
  Time (mean ± σ):      9.565 s ±  0.203 s    [User: 34.837 s, System: 0.799 s]
  Range (min … max):    9.421 s …  9.708 s    2 runs
 
Summary
  hyper_threading_pr threads: 4 ran
    1.01 ± 0.02 times faster than hyper_threading_main threads: 4




hyperfine -r 2 -n "hyper_threading_main threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_main' -n "hyper_threading_pr threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 6
  Time (mean ± σ):      9.530 s ±  0.309 s    [User: 35.317 s, System: 0.778 s]
  Range (min … max):    9.311 s …  9.748 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 6
  Time (mean ± σ):      9.487 s ±  0.250 s    [User: 35.453 s, System: 0.777 s]
  Range (min … max):    9.311 s …  9.664 s    2 runs
 
Summary
  hyper_threading_pr threads: 6 ran
    1.00 ± 0.04 times faster than hyper_threading_main threads: 6




hyperfine -r 2 -n "hyper_threading_main threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_main' -n "hyper_threading_pr threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 8
  Time (mean ± σ):      9.370 s ±  0.016 s    [User: 35.233 s, System: 0.822 s]
  Range (min … max):    9.358 s …  9.381 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 8
  Time (mean ± σ):      9.511 s ±  0.196 s    [User: 35.236 s, System: 0.816 s]
  Range (min … max):    9.373 s …  9.650 s    2 runs
 
Summary
  hyper_threading_main threads: 8 ran
    1.02 ± 0.02 times faster than hyper_threading_pr threads: 8




hyperfine -r 2 -n "hyper_threading_main threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_main' -n "hyper_threading_pr threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 16
  Time (mean ± σ):      9.739 s ±  0.047 s    [User: 35.303 s, System: 0.855 s]
  Range (min … max):    9.706 s …  9.772 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 16
  Time (mean ± σ):      9.577 s ±  0.063 s    [User: 35.628 s, System: 0.807 s]
  Range (min … max):    9.533 s …  9.621 s    2 runs
 
Summary
  hyper_threading_pr threads: 16 ran
    1.02 ± 0.01 times faster than hyper_threading_main threads: 16

[codecov]

Codecov Report

❌ Patch coverage is 95.72650% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.18%. Comparing base (51d8b5f) to head (dc5490c).

Files with missing lines	Patch %	Lines
vm/src/test_helpers/error_utils.rs	95.18%	4 Missing ⚠️
vm/src/test_helpers/test_utils.rs	96.87%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #2381    +/-   ##
========================================
  Coverage   96.17%   96.18%            
========================================
  Files         105      107     +2     
  Lines       37770    37885   +115     
========================================
+ Hits        36326    36438   +112     
- Misses       1444     1447     +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Benchmark Results for unmodified programs 🚀

Command	Mean [s]	Min [s]	Max [s]	Relative
base big_factorial	2.246 ± 0.010	2.234	2.261	1.00
head big_factorial	2.248 ± 0.018	2.233	2.291	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base big_fibonacci	2.155 ± 0.005	2.147	2.162	1.00
head big_fibonacci	2.161 ± 0.011	2.148	2.184	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base blake2s_integration_benchmark	7.600 ± 0.115	7.468	7.839	1.00
head blake2s_integration_benchmark	7.653 ± 0.096	7.524	7.862	1.01 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
base compare_arrays_200000	2.293 ± 0.019	2.268	2.332	1.00
head compare_arrays_200000	2.294 ± 0.014	2.270	2.321	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base dict_integration_benchmark	1.498 ± 0.016	1.484	1.529	1.00
head dict_integration_benchmark	1.512 ± 0.017	1.493	1.536	1.01 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
base field_arithmetic_get_square_benchmark	1.283 ± 0.030	1.254	1.353	1.01 ± 0.03
head field_arithmetic_get_square_benchmark	1.275 ± 0.025	1.259	1.344	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base integration_builtins	7.722 ± 0.029	7.674	7.779	1.00
head integration_builtins	7.751 ± 0.056	7.680	7.823	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base keccak_integration_benchmark	7.934 ± 0.222	7.807	8.557	1.01 ± 0.03
head keccak_integration_benchmark	7.841 ± 0.058	7.758	7.944	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base linear_search	2.266 ± 0.013	2.250	2.287	1.00
head linear_search	2.288 ± 0.017	2.268	2.318	1.01 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base math_cmp_and_pow_integration_benchmark	1.579 ± 0.014	1.563	1.606	1.00
head math_cmp_and_pow_integration_benchmark	1.579 ± 0.012	1.562	1.603	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base math_integration_benchmark	1.543 ± 0.004	1.538	1.553	1.00
head math_integration_benchmark	1.552 ± 0.008	1.544	1.570	1.01 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base memory_integration_benchmark	1.314 ± 0.022	1.287	1.352	1.02 ± 0.02
head memory_integration_benchmark	1.287 ± 0.003	1.283	1.294	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base operations_with_data_structures_benchmarks	1.638 ± 0.010	1.627	1.657	1.00
head operations_with_data_structures_benchmarks	1.648 ± 0.014	1.634	1.672	1.01 ± 0.01

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
base pedersen	573.1 ± 11.0	563.4	598.1	1.01 ± 0.02
head pedersen	568.4 ± 4.2	563.1	577.5	1.00

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
base poseidon_integration_benchmark	668.9 ± 5.7	660.8	678.8	1.00
head poseidon_integration_benchmark	674.1 ± 7.5	664.8	687.1	1.01 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base secp_integration_benchmark	1.939 ± 0.014	1.923	1.964	1.00
head secp_integration_benchmark	1.939 ± 0.011	1.929	1.963	1.00 ± 0.01

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
base set_integration_benchmark	662.4 ± 1.6	659.9	664.6	1.00
head set_integration_benchmark	665.4 ± 6.0	661.0	679.1	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base uint256_integration_benchmark	4.359 ± 0.026	4.323	4.419	1.00
head uint256_integration_benchmark	4.382 ± 0.039	4.343	4.466	1.01 ± 0.01

[YairVaknin-starkware]

@YairVaknin-starkware reviewed 9 files and all commit messages, and made 4 comments.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on naor-starkware).

---

vm/src/test_helpers/error_utils.rs line 44 at r2 (raw file):

/// Type alias for check functions that validate test results.
pub type VmCheck<T> = fn(&std::result::Result<T, CairoRunError>);

can just be Result, right? pls looks for similar instances where you can shorten.

Code quote:

&std::result::Result<T, CairoRunError>

---

vm/src/test_helpers/error_utils.rs line 62 at r2 (raw file):

}

/// Asserts that the result is `HintError::AssertNotEqualFail`.

This funcs have very repetitive boiler plate. Pls extract to a single func which these func will invoke that will get the res and the predicate u check.

Code quote:

/// Asserts that the result is `HintError::AssertNotEqualFail`.

---

vm/src/test_helpers/error_utils.rs line 220 at r2 (raw file):

    }

    /// `expect_hint_assert_not_zero` does not panic on `HintError::AssertNotZero`.

these error tests could be parameterized using rtest to reduce alot of repetitive boilerplate.

---

vm/src/test_helpers/test_utils.rs line 48 at r2 (raw file):

            Ok(v) => v,
            Err(e) => panic!("conversion to MaybeRelocatable failed: {e:?}"),
        };

pls factor out the conversion logic of both cases into a single func which also enforces coercion into MaybeRelocatable (currently it will work for any right that is able to be try_into'd left's type).

Code quote:

        let right_mr = match ($right).try_into() {
            Ok(v) => v,
            Err(e) => panic!("conversion to MaybeRelocatable failed: {e:?}"),
        };

[naor-starkware]

@naor-starkware made 4 comments.
Reviewable status: 7 of 10 files reviewed, 4 unresolved discussions (waiting on YairVaknin-starkware).

---

vm/src/test_helpers/error_utils.rs line 44 at r2 (raw file):

Previously, YairVaknin-starkware wrote…

can just be Result, right? pls looks for similar instances where you can shorten.

Done.

---

vm/src/test_helpers/error_utils.rs line 62 at r2 (raw file):

Previously, YairVaknin-starkware wrote…

This funcs have very repetitive boiler plate. Pls extract to a single func which these func will invoke that will get the res and the predicate u check.

Done.

---

vm/src/test_helpers/error_utils.rs line 220 at r2 (raw file):

Previously, YairVaknin-starkware wrote…

these error tests could be parameterized using rtest to reduce alot of repetitive boilerplate.

Done.

---

vm/src/test_helpers/test_utils.rs line 48 at r2 (raw file):

Previously, YairVaknin-starkware wrote…

pls factor out the conversion logic of both cases into a single func which also enforces coercion into MaybeRelocatable (currently it will work for any right that is able to be try_into'd left's type).

Done.

[OmriEshhar1]

@OmriEshhar1 reviewed 7 files and all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on YairVaknin-starkware).

[OmriEshhar1]

@OmriEshhar1 reviewed 2 files and all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on YairVaknin-starkware).

[OmriEshhar1]

Reviewable status: all files reviewed, 4 unresolved discussions (waiting on YairVaknin-starkware).