Indexes 7: Adds a lock file around index generation and updates#1354
Open
dcookspi wants to merge 1 commit intoindex-5-zplus-1-versionfilters-not-stringsfrom
Open
Indexes 7: Adds a lock file around index generation and updates#1354dcookspi wants to merge 1 commit intoindex-5-zplus-1-versionfilters-not-stringsfrom
dcookspi wants to merge 1 commit intoindex-5-zplus-1-versionfilters-not-stringsfrom
Conversation
dcookspi
added
enhancement
dcookspi
force-pushed
the
index-7-lockfile-for-indexgen
branch
from
a0324cf to
85050e7
Compare
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This was referenced Apr 17, 2026
dcookspi
force-pushed
the
index-5-zplus-1-versionfilters-not-strings
branch
from
81fb685 to
f31a655
Compare
dcookspi
force-pushed
the
index-7-lockfile-for-indexgen
branch
from
85050e7 to
d4d81db
Compare
dcookspi
force-pushed
the
index-5-zplus-1-versionfilters-not-strings
branch
from
f31a655 to
d9f7cef
Compare
dcookspi
force-pushed
the
index-7-lockfile-for-indexgen
branch
from
d4d81db to
cddd514
Compare
dcookspi
force-pushed
the
index-5-zplus-1-versionfilters-not-strings
branch
from
d9f7cef to
7aa00a9
Compare
Signed-off-by: David Gilligan-Cook <dcook@imageworks.com>
dcookspi
force-pushed
the
index-7-lockfile-for-indexgen
branch
from
cddd514 to
5ac3a7b
Compare
jrray
requested changes
May 6, 2026
Comment on lines
+153
to
+156
| if result.is_err() { | ||
| // Need to keep these if-statements separate | ||
| // to allow for two different error handling cases. | ||
| #[allow(clippy::collapsible_if)] |
Comment on lines
+191
to
+194
| Ok(_file) => { | ||
| // Write any given contents to the lock file as well | ||
| if let Some(ref data) = contents | ||
| && let Err(err) = tokio::fs::write(&lock_file, data).await |
Collaborator
There was a problem hiding this comment.
Suggested change
| Ok(_file) => { | |
| // Write any given contents to the lock file as well | |
| if let Some(ref data) = contents | |
| && let Err(err) = tokio::fs::write(&lock_file, data).await | |
| Ok(mut file) => { | |
| // Write any given contents to the lock file as well | |
| if let Some(ref data) = contents | |
| && let Err(err) = tokio::io::AsyncWriteExt::write_all(&mut file, data).await |
Use the file handle already opened.
| // file for the lock data and add it to the returned error. | ||
| use std::io::Read; | ||
| let mut lock_data = String::new(); | ||
| let held_for = match std::fs::File::open(&lock_file) { |
Collaborator
There was a problem hiding this comment.
It's weird to switch to synchronous file apis here in an async function that was using async file apis earlier.
Comment on lines
+284
to
+285
| /// # Safety: | ||
| /// File locks are used to ensure that only one process is writing |
Collaborator
There was a problem hiding this comment.
Suggested change
| /// # Safety: | |
| /// File locks are used to ensure that only one process is writing | |
| /// # Safety | |
| /// | |
| /// File locks are used to ensure that only one process is writing |
The style convention we're using almost consistently throughout the project.
| ); | ||
|
|
||
| return Err(unlock_error); | ||
| } |
Collaborator
There was a problem hiding this comment.
Suggested change
| } | |
| } else { | |
| self.has_been_deleted = true; | |
| } |
If the file didn't exist, then it had been deleted. Or does this variable mean to say was_deleted_by_us? Since this var is used to decide to attempt to delete the file in the drop handler then it should be set true here too, IMO.
| if !self.has_been_deleted | ||
| && let Err(err) = unsafe { self.unlock() } | ||
| { | ||
| // Can only the problem here. The unlock() method will |
Collaborator
There was a problem hiding this comment.
Suggested change
| // Can only the problem here. The unlock() method will | |
| // Can only <word> the problem here. The unlock() method will |
| let repo = &repos[0].1; | ||
|
|
||
| // Lock the index file until the data has been updated | ||
| let mut _lock = lock_index_file(repo).await?; |
Collaborator
There was a problem hiding this comment.
It will reduce programming bugs if the return value can be turned into a lock guard that is required as a parameter to the function call(s) that write to the index.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a lock file around index generation and update processing. The lock for the index file is created, via a
FileLockobject, before the underlying repo is read and kept until the index has been generated and written out. The same duration of locking is done for updating the index.A
FileLockhas configurable settings for the maximum number of tries to get the lock before giving up, and the amound of time to sleep between tries, as well as the kind of file being locked (used in log messages). It also contains sentry integration when the sentry feature is enabled.This is the 7th of the chained PRs for adding indexes to spk solves:
new_unchecked()constructors to spk schema objects #1337spk repo indexsubcommand for index generation and updates #1340version_filterfield in index schema #1344