Skip to content

Introduce a pool-indexer to provide univ3 liquidity#4349

Open
fafk wants to merge 95 commits intomainfrom
teamathon/indexer
Open

Introduce a pool-indexer to provide univ3 liquidity#4349
fafk wants to merge 95 commits intomainfrom
teamathon/indexer

Conversation

@fafk
Copy link
Copy Markdown
Contributor

@fafk fafk commented Apr 21, 2026

Description

Introduces pool-indexer service that indexes Uniswap V3 pool state from on-chain events and serves it over HTTP, plus the driver-side plumbing to use it as a drop-in replacement for the Uniswap V3 subgraph. The goal is to remove the driver's request-path dependency on external subgraphs.

How it works at a high level. Pool-indexer maintains a Postgres snapshot of every Uniswap V3 pool on the factories it's configured to watch. The lifecycle is:

  1. Bootstrap (one-time per network, two options):
  • Cold seed (used when no subgraph is available): runs in three phases.
    • Phase 1 — pool discovery: scan PoolCreated logs on the factory from genesis to current head, chunked ~10k blocks per eth_getLogs call with bisecting retry on "range too large" rejections. Sparse filter → cheap. Collects ERC-20 decimals() for every referenced token via concurrent eth_calls.
    • Phase 2 — state snapshot: for each discovered pool, concurrently fetch slot0() + liquidity() at the snapshot block. Populates (sqrt_price, tick, liquidity) for all pools.
    • Phase 3 — tick reconstruction: for pools with non-zero current liquidity, fetch their Mint/Burn log history using an address-filtered eth_getLogs (server-side-selective, cheap even over full history) and accumulate signed liquidity_net deltas per tick. Dead pools skip this phase. Persistence happens in pool-address-batch-sized chunks so memory stays bounded and operators see progress.
  • Subgraph seed (used when a subgraph is available): single-shot pull of pools + ticks from the Uniswap V3 subgraph. Faster than cold seed for chains like mainnet where the subgraph is public and up-to-date.
  1. Live indexing (continuous): once seeded, the indexer polls for new finalized blocks and processes them in chunks. For each chunk it fetches all V3-shaped events (PoolCreated, Initialize, Swap, Mint, Burn) in one eth_getLogs call — topic-filtered, no address filter (addresses of pools can't fit; the SQL
    writers filter unknown pools out instead). Within a single chunk transaction: new pools are inserted, Swap/Initialize events update pool state (they contain sqrtPrice/liquidity/tick in the event payload — free state), Mint/Burn adjust tick liquidity_net and trigger a pool.liquidity() refresh if no Swap was
    seen in the same chunk, and the checkpoint advances.
  2. Async token-symbol backfill: symbol lookups are deliberately decoupled from the ingest hot path (a hung symbol() RPC call must never stall pool inserts). A separate task polls for tokens with NULL symbols and fills them in, writing "" as a sentinel on failure so broken tokens don't get retried on every
    pass.
  3. HTTP API surface: list pools (cursor-paginated), lookup pools by address list (bulk), fetch ticks for one pool, fetch ticks for many pools (bulk grouped by address). These shapes exactly mirror the two queries the driver currently makes against the subgraph, so the swap-in is a minimal trait impl.

Driver integration. Behind a new V3PoolDataSource trait, the existing UniV3SubgraphClient and a new PoolIndexerClient are interchangeable. Config grows an optional pool-indexer-url; when set, it's used instead of the subgraph. The rest of the driver's V3 machinery (checkpoint, event replay, merge on new
blocks) is unchanged — pool-indexer just plugs in as the source of truth for initial + on-demand pool snapshots. Per-network switchover is one config line, fully backward compatible.

How to test

Added E2E tests.

I also deployed this to mainnet staging and did a swap and baseline participated. I did a bunch of smoke test that compare the results to the subgraph results and they were identical.

jmg-duarte and others added 17 commits March 24, 2026 14:23
- Split `api/uniswap_v3.rs` into `pools.rs` and `ticks.rs` submodules
  with shared helpers (`internal_error`, `parse_hex_address`) in `mod.rs`
- Add `{network}` path segment to routes; handlers return 404 for unknown networks
- Add `token0`/`token1` query params for symbol-based pool search
  (partial, case-insensitive, order-independent for pairs)
- Extract `search_pools` and `list_pools` as focused internal helpers
- Document all public structs, fields, and handlers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@fafk fafk changed the title Teamathon/indexer Introduce a pool-indexer to provide univ3 liquidity Apr 21, 2026
fafk added 2 commits April 21, 2026 09:24
# Conflicts:
#	Cargo.lock
#	contracts/generated/contracts-facade/Cargo.toml
#	contracts/generated/contracts-facade/src/lib.rs
#	contracts/src/main.rs
@github-actions
Copy link
Copy Markdown

Reminder: Please update the DB Readme and comment whether migrations are reversible (include rollback scripts if applicable).

  • If creating new tables, update the tables list.
  • When adding a new index, consider using CREATE INDEX CONCURRENTLY for tables involved in the critical execution path.
  • For breaking changes, remember that during rollout k8s starts the new autopilot, runs the Flyway migration, and only then shuts down the old pod. That overlap means the previous version can still be processing requests on the migrated schema, so make it compatible first and ship the breaking DB change in the following release.

Caused by:

@squadgazzz
Copy link
Copy Markdown
Contributor

Is there a way to split this PR into multiple parts and move with smaller iterations?

@fafk
Copy link
Copy Markdown
Contributor Author

fafk commented Apr 21, 2026

Is there a way to split this PR into multiple parts and move with smaller iterations?

I am open to ideas.

Comment thread crates/driver/src/infra/liquidity/config.rs Outdated
Comment thread crates/liquidity-sources/src/uniswap_v3/pool_indexer.rs Outdated
Comment thread crates/pool-indexer/src/indexer/uniswap_v3.rs Outdated
Comment thread crates/pool-indexer/src/indexer/uniswap_v3.rs Outdated
Comment thread crates/pool-indexer/src/indexer/uniswap_v3.rs Outdated
Comment thread crates/pool-indexer/src/run.rs Outdated
Comment thread crates/pool-indexer/src/config.rs Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants