Skip to content

Use huggingface_hub_rust for MoE share uploads#335

Closed
i386 wants to merge 2 commits intomainfrom
codex/extract-moe-share-hf-hub-rust
Closed

Use huggingface_hub_rust for MoE share uploads#335
i386 wants to merge 2 commits intomainfrom
codex/extract-moe-share-hf-hub-rust

Conversation

@i386
Copy link
Copy Markdown
Collaborator

@i386 i386 commented Apr 18, 2026

Summary

mesh-llm moe share now opens dataset contribution PRs through huggingface_hub_rust instead of building a raw NDJSON commit request by hand.

This was extracted from #330 so the Hugging Face upload-path refactor can land independently of the broader MoE expert component sharing work.

What changed

  • replace the manual reqwest NDJSON commit flow in mesh-llm moe share with hf_hub::upload_folder
  • stage the ranking bundle into a temporary directory and upload that directory as a PR
  • show upload phase and byte/file progress while the Hugging Face upload is running
  • validate dataset_repo as owner/name before building the dataset client
  • keep the existing duplicate-prefix checks and friendly missing-token error path

CLI

Example command:

mesh-llm moe share "allenai/OLMoE-1B-7B-0924-Instruct-GGUF:Q4_K_M"

Representative output still looks like:

📤 MoE ranking share
⬆️ Opening contribution PR...
   ⏫ Uploading files ...
✅ Opened MoE dataset contribution
   commit: <oid>
   url: <commit-url>
   pr: <pr-url>

Validation

  • cargo fmt --all -- --check
  • cargo check -p mesh-llm
  • cargo test -p mesh-llm parse_dataset_repo_requires_owner_and_name
  • cargo test -p mesh-llm classify_share_prefix_distinguishes_new_existing_and_partial
  • cargo test -p mesh-llm format_upload_bytes_uses_human_units

@i386 i386 marked this pull request as ready for review April 18, 2026 04:58
@i386 i386 marked this pull request as draft April 18, 2026 05:01
@i386 i386 closed this Apr 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant