Skip to content

Document fully_consistent nuances on CockroachDB#529

Open
ivanauth wants to merge 1 commit intoauthzed:mainfrom
ivanauth:fix/issue-525-crdb-fully-consistent-nuances
Open

Document fully_consistent nuances on CockroachDB#529
ivanauth wants to merge 1 commit intoauthzed:mainfrom
ivanauth:fix/issue-525-crdb-fully-consistent-nuances

Conversation

@ivanauth
Copy link
Copy Markdown
Contributor

Fixes #525

Adds callouts to the consistency and CockroachDB datastore pages
explaining that fully_consistent does not guarantee read-after-write
on CockroachDB due to cluster_logical_timestamp() clock skew across
nodes.

</Callout>

<Callout type="warning">
**CockroachDB users:** `fully_consistent` does not guarantee read-after-write consistency on CockroachDB.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line 103 ends with "If you need read-after-write consistency, consider using a [ZedToken]"

can we merge this callout with that one?

Comment on lines +112 to +114
SpiceDB selects a revision using CockroachDB's `cluster_logical_timestamp()` at request time.
Because CockroachDB is a distributed database, this timestamp can differ from other nodes by up to the cluster's configured [`max_offset`][max-offset] (default: 500ms).
If the node handling the read picks a timestamp older than the one assigned to a recent write on a different node, the read will not see that write even though the data has been committed.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
SpiceDB selects a revision using CockroachDB's `cluster_logical_timestamp()` at request time.
Because CockroachDB is a distributed database, this timestamp can differ from other nodes by up to the cluster's configured [`max_offset`][max-offset] (default: 500ms).
If the node handling the read picks a timestamp older than the one assigned to a recent write on a different node, the read will not see that write even though the data has been committed.
At request time, SpiceDB directs the query towards one node in the CockroachDB cluster, and then runs [`cluster_logical_timestamp()`](https://www.cockroachlabs.com/docs/stable/functions-and-operators#system-info-functions).
However, because CockroachDB is a distributed database, this timestamp can differ from other nodes by up to the cluster's configured [`max_offset`][max-offset] (default: 500ms).
If the node handling the read picks a timestamp older than the one assigned to a recent write on a different node, the read will not see that write even though the data has been committed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document CRDB fully_consistent nuances

2 participants