Xtoken/off policy distillation gh#2245

Open

avenkateshha wants to merge 18 commits intoNVIDIA-NeMo:mainfrom

avenkateshha:xtoken/off-policy-distillation-gh

Commits on Apr 10, 2026

Add off-policy distillation with MATH/MMLU eval and IPC optimization
Adithyakrishna Hanasoge
committed
Commit before refactoring
Adithyakrishna Hanasoge
committed
Simplify off-policy distillation IPC path and config
Adithyakrishna Hanasoge
committed
Working IPC TP=1
Adithyakrishna Hanasoge
committed
Per-microbatch IPC teacher logits with TP=4 support
Adithyakrishna Hanasoge
committed
Clean up unused scripts and old distillation module
Adithyakrishna Hanasoge
committed
Add IPC/non-IPC toggle for off-policy distillation
Adithyakrishna Hanasoge
committed
Integrate cross-tokenizer distillation (TokenAligner) into NeMo RL
Adithyakrishna Hanasoge
committed
Add cross-tokenizer off-policy distillation README for code review
Adithyakrishna Hanasoge
committed
Add gold loss, Phi-4 teacher support, MMLU few-shot eval, and Py3.10 compat
Adithyakrishna Hanasoge
committed
made teacher inference efficient, not passing loss_fn
Adithyakrishna Hanasoge
committed
Cache loss_fn on workers to avoid Ray re-serialization and optimize sparse projection
Adithyakrishna Hanasoge
committed
Build cross-tokenizer loss fn locally on workers and shard data per DP rank
Adithyakrishna Hanasoge
committed
Add char-offset alignment method and DP fallback
Adithyakrishna Hanasoge
committed
Add CUDA-backed token alignment DP implementation.
Adithyakrishna Hanasoge
committed
Refactor off-policy cross-tokenizer processing to pipeline next-batch preprocessing with current-step GPU training while keeping alignment behavior unchanged.
Adithyakrishna Hanasoge
committed
Tune cross-tokenizer worker configuration for large-scale runs.
Adithyakrishna Hanasoge
committed

Commits on Apr 12, 2026

Rebased code based on main branch.
Adithyakrishna Hanasoge
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Xtoken/off policy distillation gh#2245

Xtoken/off policy distillation gh#2245
avenkateshha wants to merge 18 commits intoNVIDIA-NeMo:mainfrom
avenkateshha:xtoken/off-policy-distillation-gh

Commits on Apr 10, 2026

Add off-policy distillation with MATH/MMLU eval and IPC optimization

Commit before refactoring

Simplify off-policy distillation IPC path and config

Working IPC TP=1

Per-microbatch IPC teacher logits with TP=4 support

Clean up unused scripts and old distillation module

Add IPC/non-IPC toggle for off-policy distillation

Integrate cross-tokenizer distillation (TokenAligner) into NeMo RL

Add cross-tokenizer off-policy distillation README for code review

Add gold loss, Phi-4 teacher support, MMLU few-shot eval, and Py3.10 compat

made teacher inference efficient, not passing loss_fn

Cache loss_fn on workers to avoid Ray re-serialization and optimize sparse projection

Build cross-tokenizer loss fn locally on workers and shard data per DP rank

Add char-offset alignment method and DP fallback

Add CUDA-backed token alignment DP implementation.

Refactor off-policy cross-tokenizer processing to pipeline next-batch preprocessing with current-step GPU training while keeping alignment behavior unchanged.

Tune cross-tokenizer worker configuration for large-scale runs.

Commits on Apr 12, 2026

Rebased code based on main branch.