Add open-r1/OpenR1-Math-220k dataset to RL

[SurbhiJainUSC]

Description

• Add open-r1/OpenR1-Math-220k dataset to RL

Tests

Tested on v5p-8:

python3 -m maxtext.trainers.post_train.rl.train_rl \
model_name=llama3.1-8b-Instruct \
run_name=$RUN_NAME \
base_output_directory=$BASE_OUTPUT_DIRECTORY \
batch_size=4 \
dataset_name=open-r1/OpenR1-Math-220k \
train_split=train \
num_batches=5 \
scan_layers=True \
hbm_utilization_vllm=0.4 \
rollout_data_parallelism=2 \
rollout_tensor_parallelism=2 \
allow_split_physical_axes=true \
load_parameters_path=$MAXTEXT_CKPT_PATH

python3 -m maxtext.trainers.post_train.rl.train_rl \
model_name=llama3.1-8b-Instruct \
load_parameters_path=$MAXTEXT_CKPT_PATH \
run_name=$RUN_NAME \
base_output_directory=$BASE_OUTPUT_DIRECTORY \
dataset_name=nvidia/OpenMathInstruct-2 train_split=train_1M steps=10 \
rollout_tensor_parallelism=1 \
hf_train_files=hf://datasets/nvidia/OpenMathInstruct-2/data/train_1M-*.parquet \
skip_jax_distributed_system=True

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

[codecov]

Codecov Report

❌ Patch coverage is 8.39161% with 131 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/maxtext/trainers/post_train/rl/utils_rl.py	8.92%	102 Missing ⚠️
src/maxtext/trainers/post_train/rl/evaluate_rl.py	8.00%	23 Missing ⚠️
src/maxtext/trainers/post_train/rl/train_rl.py	0.00%	6 Missing ⚠️

📢 Thoughts on this report? Let us know!