Skip to content

fix(sam3_image): prevent unwrap panic caused by LruCache eviction during mixed queries#254

Open
darkSuperman wants to merge 2 commits intojamjamjon:mainfrom
darkSuperman:main
Open

fix(sam3_image): prevent unwrap panic caused by LruCache eviction during mixed queries#254
darkSuperman wants to merge 2 commits intojamjamjon:mainfrom
darkSuperman:main

Conversation

@darkSuperman
Copy link
Copy Markdown

🐛 The Bug
Calling self.text_cache.get(&p.text).unwrap() occasionally panics with Option::unwrap() on a None value.
This happens when:
The number of unique prompts in a single batch exceeds the default LruCache capacity (16).
Or, a previously cached prompt is pushed to the tail (least recently used), and a subsequent query mixes old (tail) prompts with new uncached ones.

🕵️ Root Cause
The original code used self.text_cache.contains(&p.text) which only checks existence but does not promote the item to the Most Recently Used (MRU) position.
When new prompts are put() into the cache, the capacity overflows, evicting the oldest items (even if they are about to be accessed in the current batch).
The subsequent unwrap() encounters a None value.

🛠️ The Fix
Promote MRU: Replaced .contains() with .get(). This inherently promotes accessed items to the front, granting them immunity from eviction during the same batch processing.
Dynamic Resize: Dynamically check seen.len(). If it exceeds current capacity, .resize() the cache to guarantee the entire batch fits safely.
Avoid Double Encoding: Extracted encode_texts into a variable encoded_feats to prevent running the heavy vision-language model twice for the same inputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant