Fix Qwen3.5 MoE load path in vLLM Metal#129
Conversation
| # MLX - Required for Apple Silicon GPU acceleration | ||
| "mlx>=0.29.2; platform_system == 'Darwin' and platform_machine == 'arm64'", | ||
| "mlx-lm>=0.28.4; platform_system == 'Darwin' and platform_machine == 'arm64'", | ||
| "mlx-lm>=0.30.8; platform_system == 'Darwin' and platform_machine == 'arm64'", |
There was a problem hiding this comment.
This change mixes two conflicting strategies: it raises the minimum mlx-lm version to one that should already include native qwen3_5* support, and also adds a runtime alias shim for older versions.
|
|
||
| try: | ||
| resolved = importlib.import_module(target_module) | ||
| except ModuleNotFoundError: |
There was a problem hiding this comment.
except ModuleNotFoundError is too broad here and can hide real import problems inside the imported module. Please only suppress the error when e.name matches the exact module you attempted to import; otherwise re-raise.
There was a problem hiding this comment.
Thanks for working on the Qwen3.5 MoE compatibility issue. I’m requesting changes for now.
The PR needs more work before it is ready to merge:
- the compatibility strategy is currently inconsistent (hard minimum-version bump plus fallback shim),
- and the exception handling is too broad and can mask real import failures.
Please rework these points and we can re-review quickly.
This comment was marked as duplicate.
This comment was marked as duplicate.
Sorry, something went wrong.
|
Hi @LxYuan0420, I think we can close this PR. Qwen3.5 support is already handled in #174 and #169 with a safer approach. Thanks! |
3 participants
Summary\n- bump minimum \ to \ (includes qwen3_5/qwen3_5_moe model modules)\n- add a guarded mlx_lm alias shim in \ so older mlx_lm builds can still map \ to \ as a fallback\n- avoid overriding native modules when mlx_lm already provides \n\n## Motivation\nQwen3.5 MoE snapshots currently fail to load on some vLLM Metal installations with:\n.\n\nThis patch ensures native support is used when present and provides a fallback alias path for older environments.