feat: Add pinned memory optimizer offload for Megatron policy worker

[snivertynv]

This significantly improves performance for the optimizer_offload_before_refit pass which is quite expensive in co-located/syncRL cases.

Enabled/disabled using the use_pinned_optimizer_offload setting (default=disabled). It has been set to false in a couple of grpo_math* yaml configs as an example. Added test cases for this feature in test_megatron_worker.py

What does this PR do ?

Optimizer D2H/H2D transfers used per-tensor pageable allocations, causing expensive cudaHostAlloc calls and synchronous memcpy on every step. This adds an opt-in mode (use_pinned_optimizer_offload) that coalesces all optimizer state into a single cached pinned buffer, eliminating cudaHostAlloc from the hot path and enabling non-blocking DMA transfers.

Issues

List issues that this PR closes (syntax):

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[snivertynv]

/ok to test 13e7f01