[External]Add S3 blob storage with cashier billing to ic-gateway

[shilingwang]

#NODE-1941

Summary

• Adds a full blob storage API to ic-gateway, enabling upload/download of content-addressed blobs backed by a single AWS S3 bucket with per-owner billing through the cashier canister.
• Introduces new /v1/ HTTP endpoints for blob metadata, chunk operations, and owner data management, gated behind --s3-endpoint and --cashier-canister-id CLI flags.
• Integrates billing (budget checks, usage reporting) via a CashierConnector that caches budgets locally and flushes usage counters periodically, wired into ic-gateway's existing TaskManager and HealthManager.

New modules

• src/s3/ — S3 client abstraction (BucketLike trait, AWSBucket impl, RamFakeBucket for dev), config
• src/cashier/ — CashierClient (4 canister calls: whoami, pricelist, budget, usage reporting), CashierConnector (local billing cache + periodic flush)
• src/storage/ — Shared types (blob metadata, hash tree, chunk constants), S3 key paths, IC egress certificate auth
• src/routing/storage/ — Axum handlers + router for all /v1/ endpoints

HTTP endpoints (under /v1/)

• HEAD /v1/blob — Blob metadata headers (size, content type)
• GET /v1/blob — Download blob with Range header support
• GET /v1/blob-tree — Raw blob metadata JSON
• PUT /v1/blob-tree — Upload blob metadata (with IC egress cert auth)
• GET /v1/chunk — Download a single chunk
• PUT /v1/chunk — Upload a single chunk (SHA-256 verified)
• DELETE /v1/owner — Delete all data for an owner (host-gated)

Design decisions

• Single S3 bucket: One bucket configured via CLI, no multi-bucket routing. Simpler than the multi-instance model in object-storage.
• Billing gated: Storage routes are only mounted when both --s3-endpoint and --cashier-canister-id are provided. Without either, ic-gateway serves only normal IC traffic.
• Budget caching: Per-owner budgets cached for 30s to avoid hitting the cashier canister on every request. Usage counters flushed every 10s.
• IC egress auth: PUT /blob-tree verifies an OwnerEgressSignature certificate from the request body. Bypassable with --fake-ingress-auth for local dev.

[github-actions]

Thank you for contributing! Unfortunately this repository does not accept external contributions yet.

We are working on enabling this by aligning our internal processes and our CI setup to handle external contributions. However this will take some time to set up so in the meantime we unfortunately have to close this Pull Request.

We hope you understand and will come back once we accept external PRs.

— The DFINITY Foundation