[CHIA-3854] Expose generator_interned_weight() to Python by richardkiss · Pull Request #1418 · Chia-Network/chia_rs

richardkiss · 2026-04-16T21:20:49Z

Summary

Stacked on top of #1377 (now merged).

Splits the interned tree cost calculation into interned_weight() (raw weight) and total_cost_from_tree() (weight × COST_PER_BYTE)
Exposes generator_interned_weight(program) -> int to Python: deserializes a generator, interns the tree, returns atom_bytes + 2*atoms + 3*pairs
Python callers multiply by constants.COST_PER_BYTE themselves, reusing the existing consensus constant instead of hardcoding 12000

This lets BlockTools.compute_block_cost in chia-blockchain compute the correct size cost when INTERNED_GENERATOR is active:

if prev_tx_height >= constants.HARD_FORK2_HEIGHT:
    size_cost = generator_interned_weight(bytes(generator)) * constants.COST_PER_BYTE
else:
    size_cost = len(bytes(generator)) * constants.COST_PER_BYTE

Test plan

Existing generator_cost Rust tests pass (refactor is behavior-preserving)
CI

Made with Cursor

Split the interned tree cost calculation into two parts: - interned_weight(): returns atom_bytes + 2*atoms + 3*pairs - total_cost_from_tree(): interned_weight() * COST_PER_BYTE The new generator_interned_weight() Python function deserializes a generator program, interns the tree, and returns the raw weight. Callers multiply by the COST_PER_BYTE consensus constant themselves, avoiding a hardcoded multiplier on the Python side. Made-with: Cursor

coveralls-official · 2026-04-16T22:06:21Z

Coverage Report for CI Build 24805747779

Coverage decreased (-0.6%) to 80.056%

Details

Coverage decreased (-0.6%) from the base build.
Patch coverage: 149 uncovered changes across 6 files (11 of 160 lines covered, 6.88%).
No coverage regressions found.

Uncovered Changes

File	Changed	Covered	%
crates/chia-consensus/src/build_interned_block.rs	121	0	0.0%
wheel/src/run_generator.rs	9	0	0.0%
crates/chia-consensus/src/solution_generator.rs	8	0	0.0%
wheel/src/api.rs	9	2	22.22%
crates/chia-consensus/src/run_block_generator.rs	3	0	0.0%
crates/chia-consensus/src/spendbundle_conditions.rs	1	0	0.0%

Coverage Regressions

No coverage regressions found.

Coverage Stats


Relevant Lines:	18542
Covered Lines:	14844
Line Coverage:	80.06%
Coverage Strength:	12162646.73 hits per line

💛 - Coveralls

The .pyi had it manually but generate_type_stubs.py didn't, causing the Check chia_rs.pyi CI job to fail. Made-with: Cursor

A clean, separate builder that avoids the Serializer/sentinel/restore complexity of BlockBuilder. Cost is computed from total_cost_from_tree on the interned quoted generator tree after each add; serialization happens once in finalize() via node_to_bytes_backrefs. Includes Python bindings matching the BlockBuilder interface. Made-with: Cursor

- Patch workspace clvmr to Chia-Network/clvm_rs@serde_2026 (v0.17.1, feature ser-2026) and lower version requirement from 0.17.4 to 0.17 - Add solution_generator_2026() using node_to_bytes_serde_2026 - Switch INTERNED_GENERATOR path in run_block_generator2 from node_from_bytes_backrefs to node_from_bytes_auto (auto-detect format) - Replace intern_tree_limited / intern_tree (not in serde_2026 branch) with intern() in run_block_generator, spendbundle_conditions, generator_cost tests - Drop ClvmFlags::LIMITS / ClvmFlags::MALACHITE mappings in flags.rs; those flags were added in clvmr 0.17.4 which post-dates serde_2026 - Add Python binding solution_generator_2026 + pyi / stub stubs Made-with: Cursor

Made-with: Cursor

The serde_2026 branch of clvmr uses intern() instead of intern_tree_limited(). Update InternedBlockBuilder and generator_interned_weight Python binding to match. Made-with: Cursor

…s_auto Block2026Builder: anytime block builder for post-HF2 (serde_2026). Accepts candidates in priority order, packs greedily with upper-bound cost estimates, validates with exact interned_weight costing, then refines in a background thread. Caller retrieves best block instantly via best() which also returns included_indices for metadata tracking. Usable as Python context manager. Non-consensus CLVM readers (additions_and_removals, get_puzzle_and_solution, get_coinspends_for_trusted_block, run_chia_program, generator_interned_weight) switched from node_from_bytes_backrefs to node_from_bytes_auto so they accept classic, backrefs, and serde_2026 formats transparently. Consensus-critical paths unchanged (gated on INTERNED_GENERATOR flag). Made-with: Cursor

Program.from_program_bytes(blob) wraps raw bytes as a Program without CLVM structure validation. Needed for serde_2026 format generators which are validated at execution time by run_block_generator / node_from_bytes_auto, not at construction time. Also adds Block2026Builder type stubs to the .pyi file. Made-with: Cursor

tree_hash_auto() deserializes using node_from_bytes_auto (handles standard CLVM, backrefs, and serde_2026) then computes the tree hash. Needed for generator_root computation on serde_2026 blocks. Made-with: Cursor

serde_2026 blocks don't start with [0xff, 0x01] (standard CLVM quote prefix) since the serialization format is different. When INTERNED_GENERATOR is active, bypass the SIMPLE_GENERATOR quote check in both the raw bytes and deserialized node validators. Made-with: Cursor

When deserializing a Program from JSON, try standard CLVM serialized_length first. If that fails, fall back to node_from_bytes_auto which handles backrefs and serde_2026. This allows FullBlock JSON round-trips for blocks with serde_2026 generators (e.g. via RPC). Made-with: Cursor

- 5 tests for check_generator_quote/check_generator_node: verify SIMPLE_GENERATOR still rejects non-quote generators when INTERNED_GENERATOR is NOT set (pre-HF2 safety) - 1 test for tree_hash_auto: verify it produces the same hash as tree_hash_from_bytes for standard CLVM and backrefs, and also works for serde_2026 (which tree_hash_from_bytes rejects) Made-with: Cursor

Program::parse() now auto-detects serde_2026 magic prefix and uses serialized_length_serde_2026() to delimit the blob. Without this, FullBlock serialization/deserialization fails for post-HF2 blocks that use serde_2026 generators. Also updates clvmr dep to pick up the new function, and adapts to API renames (intern -> intern_tree, node_from_bytes_auto gains DeserializeLimits parameter). Made-with: Cursor

socket-security · 2026-04-27T19:44:39Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	cargo/chia-bls@0.28.1 ⏵ 0.36.1
	cargo/chia-sha2@0.28.1 ⏵ 0.36.1
	cargo/chia-traits@0.28.1 ⏵ 0.36.1
	cargo/chia_streamable_macro@0.28.1 ⏵ 0.36.1			⁺⁴

View full report

richardkiss added 5 commits April 21, 2026 21:13

Add generator_interned_weight to type stub generator

09d1962

The .pyi had it manually but generate_type_stubs.py didn't, causing the Check chia_rs.pyi CI job to fail. Made-with: Cursor

pin clvmr patch to explicit commit rev for determinism

47dc39c

Made-with: Cursor

Fix intern API for serde_2026 clvmr: intern_tree_limited -> intern

e967731

The serde_2026 branch of clvmr uses intern() instead of intern_tree_limited(). Update InternedBlockBuilder and generator_interned_weight Python binding to match. Made-with: Cursor

richardkiss mentioned this pull request Apr 22, 2026

[CHIA-3854] Add INTERNED_GENERATOR cost model to BlockBuilder #1413

Closed

3 tasks

richardkiss added 6 commits April 23, 2026 13:09

add tree_hash_auto() for serde_2026 / backrefs programs

bf0ff53

tree_hash_auto() deserializes using node_from_bytes_auto (handles standard CLVM, backrefs, and serde_2026) then computes the tree hash. Needed for generator_root computation on serde_2026 blocks. Made-with: Cursor

danieljperry changed the title ~~Expose generator_interned_weight() to Python~~ [CHIA-3854] Expose generator_interned_weight() to Python Apr 24, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CHIA-3854] Expose generator_interned_weight() to Python#1418

[CHIA-3854] Expose generator_interned_weight() to Python#1418
richardkiss wants to merge 13 commits intomainfrom
expose-interned-weight

richardkiss commented Apr 16, 2026

Uh oh!

coveralls-official Bot commented Apr 16, 2026 •

edited

Loading

Uh oh!

socket-security Bot commented Apr 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

richardkiss commented Apr 16, 2026

Summary

Test plan

Uh oh!

coveralls-official Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report for CI Build 24805747779

Coverage decreased (-0.6%) to 80.056%

Details

Uncovered Changes

Coverage Regressions

Coverage Stats

💛 - Coveralls

Uh oh!

socket-security Bot commented Apr 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

coveralls-official Bot commented Apr 16, 2026 •

edited

Loading