Skip to content

Roll a custom TestEnv and drop lazy_static#176

Open
luisschwab wants to merge 3 commits intobitcoindevkit:masterfrom
luisschwab:feat/custom-testenv
Open

Roll a custom TestEnv and drop lazy_static#176
luisschwab wants to merge 3 commits intobitcoindevkit:masterfrom
luisschwab:feat/custom-testenv

Conversation

@luisschwab
Copy link
Copy Markdown
Member

@luisschwab luisschwab commented Mar 8, 2026

Closes #149

Implement a custom TestEnv, which is instantiated per-test and fixes the un-dropped bitcoind issue. This also allows running tests with more than 1 thread, since they no longer share bitcoind and electrsd instances.

Also adds helpers to fetch externally sourced addresses, fixing the issue where bitcoind's wallet would send change outputs to it's own wallet address, which would mess with test_get_scripthash_stats.

The test recipe and CI job are updated to now use 16 threads.

@luisschwab
Copy link
Copy Markdown
Member Author

Need to pin some stuff...

@luisschwab luisschwab force-pushed the feat/custom-testenv branch 3 times, most recently from 28bba06 to 4f5ee42 Compare March 8, 2026 16:36
@luisschwab luisschwab added the bug Something isn't working label Mar 8, 2026
Copy link
Copy Markdown
Member

@notmandatory notmandatory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran this through my AI friend 🤖 and it found two issue and one nit that I think are worth mentioning. Otherwise very good improvement to the testing!

@ValuedMammal
Copy link
Copy Markdown
Contributor

Concept ACK

@luisschwab luisschwab force-pushed the feat/custom-testenv branch from 4f5ee42 to 8653954 Compare March 11, 2026 22:58
@luisschwab
Copy link
Copy Markdown
Member Author

Addressed Steve's review and added a headers field to the AsyncClient that's pub(crate), for testing purposes (it will be removed once we move to bitreq).

@luisschwab luisschwab requested a review from notmandatory March 11, 2026 23:00
@ValuedMammal
Copy link
Copy Markdown
Contributor

Looks like your code is correctly setting the headers on the Builder for both the async and blocking clients, so no need for the unused AsyncClient::headers.

@luisschwab
Copy link
Copy Markdown
Member Author

Looks like your code is correctly setting the headers on the Builder for both the async and blocking clients, so no need for the unused AsyncClient::headers.

The extra field is for the test only, will be handy to catch any regressioms when we switch to bitreq.

Copy link
Copy Markdown
Collaborator

@oleonardolima oleonardolima left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good! However, we shouldn't have the new field on AsyncClient, left an explanation how to properly test it in a comment below.

Comment on lines +411 to +416
/// Get a `Legacy` regtest address.
fn get_legacy_address(&self) -> Address {
Address::from_str("mvUsRD2pNeQQ8nZq8CDEx6fjVQsyzqyhVC")
.unwrap()
.assume_checked()
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the rationale for having fixed addresses, instead o fetching one from the client. If it's required only for test_get_scripthash_stats they should probably constants under that test only.

Copy link
Copy Markdown
Member Author

@luisschwab luisschwab Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bitcoind would use an address as destination for change for another addresses transaction. This is why tests were flaky. I see no issue in hardcoding these, as we only want to check transaction count and balance for an arbitrary address; in turn we also save a bunch of calls to bitcoind by using this approach.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bitcoind would use an address as destination for change for another addresses transaction. This is why tests were flaky. I see no issue in hardcoding these, as we only want to check transaction count and balance for an arbitrary address; in turn we also save a bunch of calls to bitcoind by using this approach.

Which tests were flaky ?

It was flaky due to:

(i) The address issue you mentioned
(ii) The conflicting bitcoind usage ?

If was due to (i) it's probably an issue on the test setup itself. If it was due to (ii) it's basically solved by using the new TestEnv, isn't ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was because of (ii). IIRC bitcoind would use the legacy address to send change when sending to the other 3, so we would get 2 transactions vs the expected 1 transaction.

@oleonardolima
Copy link
Copy Markdown
Collaborator

I pushed 5bdab27 to address the test for HTTP headers, it does in such a way that asserts it in the final HTTP request.

I wasn't able to do it following the approach I mentioned #176 (comment), because reqwest only builds it's final requests by calling .send(), which calls .execute_request() and builds the final requests with headers internally (there's no public API for it).

Copy link
Copy Markdown
Collaborator

@oleonardolima oleonardolima left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cACK f6faee5

I applied a few changes regarding the MSRV and the HTTP headers test, it still needs addressing for the exponential backoff and addresses comments above.

@oleonardolima oleonardolima moved this to Needs Review in BDK Chain Mar 19, 2026
luisschwab and others added 3 commits March 30, 2026 22:17
Implement a custom `TestEnv`, which is instantiated per-test and
fixes the un-dropped `bitcoind` issue. This also allows running
tests with more than 1 thread, since they no longer share `bitcoind`
and `electrsd` instances.

Also adds helpers to fetch externally sourced addresses, fixing
the issue where `bitcoind`'s wallet would send change outputs to
it's own wallet address, which would mess with `test_get_scripthash_stats`.

The `test` recipe and CI job are updated to now use 16 threads.
- updates the `test_get_tx_with_http_header` to assert for expected HTTP
  headers in final request.
- updates `setup_clients_with_headers` to expect custom `url`.
@luisschwab luisschwab force-pushed the feat/custom-testenv branch from f6faee5 to 1b54666 Compare March 31, 2026 01:46
@luisschwab
Copy link
Copy Markdown
Member Author

luisschwab commented Mar 31, 2026

Pushed 1b54666 reinstating the exponential backoff when waiting for electrsd to see the block. I kept the hardcoded addresses on all tests, since there's no harm in using them since they are already have to be there.

cc @oleonardolima

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working chore

Projects

Status: Needs Review

Development

Successfully merging this pull request may close these issues.

[test] Consider removing lazy_static

4 participants