Add span-based masking to DataCollatorForCompletionOnlyLM#2368

Closed

shanghongsim wants to merge 14 commits intomainfrom

shanghong/collator-span-masking

Commits on Apr 14, 2026

Add span-based masking to DataCollatorForCompletionOnlyLM
shanghongsim
committed
Narrow TextCompletionsCollatorWithPadding template params to str
shanghongsim
committed
Validate end_of_turn_template is provided for span-based masking
shanghongsim
committed
Extract _tokenize_template helper and move _KNOWN_MASKING_METHODS to class level
shanghongsim
committed
Fix docstring: mask_tool_calls is not a valid collator_kwargs param
shanghongsim
committed
Add deprecation warning for legacy instruction-based masking
shanghongsim
committed
Infer assistant_turn_no_tools when tool_call_start_template is provided
shanghongsim
committed
Extract _resolve_masking_method to simplify __init__ logic
shanghongsim
committed