Skip to content

Models are not auto loading due to --exact flag #11

@unleashit

Description

@unleashit

I'm not sure if this is an issue with all models, but at least the particular ones I'm working with all fail with this plugin unless/until they are manually loaded.

The problem is the lms cli in exact mode isn't expecting the model id (that can be found either with lms list or llm models), but the full path to the model file. For example, unsloth/gpt-oss-120b loads fine with lms load unsloth/gpt-oss-120b But it fails with lms load --exact unsloth/gpt-oss-120b. For exact to work in my case, I would need lms load --exact unsloth/gpt-oss-120b-GGUF/gpt-oss-120b-UD-Q6_K_XL-00001-of-00002.gguf. As far as I know, that's not possible with llm. I tried llm -m unsloth/gpt-oss-120b-GGUF/gpt-oss-120b-UD-Q6_K_XL-00001-of-00002.gguf "query" for the hey of it, and naturally it doesn't work because LLM only knows about the basic model ids.

Short of a better solution, you could offer a flag to remove --exact. That should be fine I think in cases where you don't have variations of the same model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions