Skip to content

feat(bindings/python): Add HttpClientLayer for custom HTTP client configuration#7247

Open
rgidda wants to merge 10 commits intoapache:mainfrom
rgidda:feat/python-http-client-layer
Open

feat(bindings/python): Add HttpClientLayer for custom HTTP client configuration#7247
rgidda wants to merge 10 commits intoapache:mainfrom
rgidda:feat/python-http-client-layer

Conversation

@rgidda
Copy link
Copy Markdown

@rgidda rgidda commented Mar 18, 2026

…figuration

This adds support for HttpClientLayer in Python bindings, enabling users to customize the HTTP client used for all operations. This is particularly useful for:

  • Testing with self-signed SSL certificates via danger_accept_invalid_certs
  • Setting custom request timeouts
  • Future advanced HTTP client configurations

The implementation includes:

  • HttpClient class with danger_accept_invalid_certs and timeout parameters
  • HttpClientLayer to apply custom HTTP client to operators
  • Comprehensive documentation and examples
  • Unit tests for both sync and async usage

Closes #TODO

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

AI Usage Statement

…figuration

This adds support for HttpClientLayer in Python bindings, enabling users to customize the HTTP client used for all operations. This is particularly useful for:

- Testing with self-signed SSL certificates via danger_accept_invalid_certs
- Setting custom request timeouts
- Future advanced HTTP client configurations

The implementation includes:
- HttpClient class with danger_accept_invalid_certs and timeout parameters
- HttpClientLayer to apply custom HTTP client to operators
- Comprehensive documentation and examples
- Unit tests for both sync and async usage

Closes #TODO
@rgidda rgidda requested review from Zheaoli and messense as code owners March 18, 2026 07:47
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels Mar 18, 2026
rgidda and others added 8 commits March 18, 2026 13:24
HttpClient is a top-level class (opendal.HttpClient), not in the layers
submodule, so it shouldn't be documented in layers.md. Its usage is shown
in the examples and docstrings.
- Remove trailing whitespace from test file
- Format reqwest dependency in Cargo.toml (multiline array)
- Remove HttpClientLayer from API docs (will be auto-generated)
- Add return type annotations to all functions
- Fix docstring formatting (one-line docstrings, periods, imperative mood)
- Remove trailing whitespace from all files
- Sort imports in test file
@rgidda
Copy link
Copy Markdown
Author

rgidda commented Mar 18, 2026

Business Impact & Adoption Plan

Hi maintainers! 👋

I wanted to highlight the importance of this feature for our team's immediate needs:

Use Case: We're working with self-hosted S3-compatible storage services (MinIO, etc.) in development and testing environments that use self-signed SSL certificates. Currently, the Python bindings don't provide a way to configure the HTTP client to accept these certificates, which blocks our ability to test against these environments.

Immediate Adoption: Once this PR is merged and released, we will immediately integrate this into our testing infrastructure. The HttpClientLayer will enable us to:

  • Test against local MinIO instances with HTTPS
  • Configure custom timeouts for slow network conditions
  • Maintain secure production deployments while having flexibility in dev/test

Feature Completeness: This brings the Python bindings to parity with the Rust core's HttpClientLayer, which already supports custom HTTP client configuration.

All CI checks have now passed after addressing linting, formatting, and compilation issues. The implementation includes:

  • ✅ Comprehensive test coverage (7 test cases)
  • ✅ Usage examples with security warnings
  • ✅ Documentation in README and upgrade guide
  • ✅ Proper security warnings about danger_accept_invalid_certs

We're excited to start using this feature and would greatly appreciate your review and feedback. Thank you for maintaining this excellent project! 🙏


Note: The danger_accept_invalid_certs parameter has intentionally scary naming (following Rust conventions) and includes prominent warnings in all documentation to prevent production misuse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant