Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion app/spicedb/concepts/consistency/page.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,23 @@ Note that the snapshot used will be loaded at the beginning of the API call, and

<Callout type="warning">
This consistency mode explicitly bypasses caching, dramatically impacting latency.
If you need read-after-write consistency, consider using a [ZedToken].

If you need read-after-write consistency, consider using a [ZedToken]
**CockroachDB users:** `fully_consistent` does not guarantee read-after-write consistency on CockroachDB.

At request time, SpiceDB directs the query towards one node in the CockroachDB cluster, and then runs [`cluster_logical_timestamp()`](https://www.cockroachlabs.com/docs/stable/functions-and-operators#system-info-functions).
However, because CockroachDB is a distributed database, this timestamp can differ from other nodes by up to the cluster's configured [`max_offset`][max-offset] (default: 500ms).
If the node handling the read picks a timestamp older than the one assigned to a recent write on a different node, the read will not see that write even though the data has been committed.

This does not affect CockroachDB's normal read-after-write guarantees -- those are bypassed specifically because SpiceDB chooses its own evaluation revision rather than letting CockroachDB select one.

The [overlap strategy][overlap-strategy] does not mitigate this.
The overlap strategy prevents two concurrent writes from receiving commit timestamps in the reverse order, but it does not address clock skew between nodes during reads.

If you need read-after-write consistency with CockroachDB, use a [ZedToken] with [`at_least_as_fresh`](#at-least-as-fresh) instead.

[max-offset]: https://www.cockroachlabs.com/docs/stable/cockroach-start.html#flags-max-offset
[overlap-strategy]: /spicedb/concepts/datastores#overlap-strategy
[ZedToken]: #zedtokens

</Callout>
Expand Down
20 changes: 20 additions & 0 deletions app/spicedb/concepts/datastores/page.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,26 @@ AuthZed has standardized our managed services on CockroachDB, but we give self-h
- Query and data balanced across the CockroachDB
- Setup and operational complexity of running CockroachDB

### Consistency Considerations

<Callout type="warning">
`fully_consistent` does not guarantee read-after-write consistency on CockroachDB.

SpiceDB picks a revision using CockroachDB's `cluster_logical_timestamp()`, which can differ across nodes by up to the cluster's [`max_offset`][crdb-max-offset] (default: 500ms).
A read may not see a recently committed write if the reading node's timestamp is behind the writing node's.

This is specific to CockroachDB's distributed clock model and does not apply to single-node datastores like PostgreSQL.
For read-after-write guarantees, use a [ZedToken] with [`at_least_as_fresh`][at-least-as-fresh].

See [Consistency: Fully Consistent][fully-consistent] for details.

[crdb-max-offset]: https://www.cockroachlabs.com/docs/stable/cockroach-start.html#flags-max-offset
[ZedToken]: /spicedb/concepts/consistency#zedtokens
[at-least-as-fresh]: /spicedb/concepts/consistency#at-least-as-fresh
[fully-consistent]: /spicedb/concepts/consistency#fully-consistent

</Callout>

### Developer Notes

- Code can be found [here][crdb-code]
Expand Down
Loading