-
Notifications
You must be signed in to change notification settings - Fork 5
Description
I'm not sure if this is an issue with all models, but at least the particular ones I'm working with all fail with this plugin unless/until they are manually loaded.
The problem is the lms cli in exact mode isn't expecting the model id (that can be found either with lms list or llm models), but the full path to the model file. For example, unsloth/gpt-oss-120b loads fine with lms load unsloth/gpt-oss-120b But it fails with lms load --exact unsloth/gpt-oss-120b. For exact to work in my case, I would need lms load --exact unsloth/gpt-oss-120b-GGUF/gpt-oss-120b-UD-Q6_K_XL-00001-of-00002.gguf. As far as I know, that's not possible with llm. I tried llm -m unsloth/gpt-oss-120b-GGUF/gpt-oss-120b-UD-Q6_K_XL-00001-of-00002.gguf "query" for the hey of it, and naturally it doesn't work because LLM only knows about the basic model ids.
Short of a better solution, you could offer a flag to remove --exact. That should be fine I think in cases where you don't have variations of the same model.