Upgrade to torch 2.9.1

[dxqb]

git fetch origin pull/1189/head:pr-1189
git switch pr-1189

This won't be merged very soon, but I'd suggest we merge it into dev branches that get tested again, as we did with torch 2.8.1

• Upgraded other packages that are related - are there any packages that break often like torch, where we should be careful to upgrade to the latest version?
• somebody on windows should test torch.compile with triton please. At the time the torch.compile PR was merged, the latest triton-windows version did not work. Does the latest version now work?

we don't know much about AMD and MPS, but according to a previous thread, torch 2.9 works on ROCM (unlike torch 2.8)

if there are any concerns with upgrading all these packages, we can be more careful and not upgrade the packages in -global.txt

[O-J1]

Upgraded other packages that are related - are there any packages that break often like torch, where we should be careful to upgrade to the latest version?

• transformers
• numpy(last time Nero tried to update it we found out they dropped 3.10, so when something else we need requires a newer numpy thats when we drop 3.10)
• tqdm (you and I are both aligned on our feelings for it, it does too much)

[dxqb]

• error messages when using torch.compile in this configuration, on linux and the transiently installed triton 3.5.1. Before this upgrade it was triton 3.4
  torch.compile regression 2.9.1: Triton kernel name '_Runner__kernel_name_0' is not defined pytorch/pytorch#170398

[dxqb]

upgrading to 2.9.1 works now

[O-J1]

Using this PR and manually installing the latest triton-windows (3.5.1-post23) worked fine on SDXL with Windows 11. Editing your post to tick that off.

However I would strongly advise we also add a manual cudnn update if you comfortable doing that, theres a tonne of bugs and they werent planning to fix in place (i.e re-release 2.9.1)

we don't know much about AMD

triton-windows as of v3.5.1-windows.post23 added support for AMD gpus via 'TheRock'

MPS

No clue lol

[O-J1]

@dxqb pytorch/pytorch#150381 (comment)

Pytorch has fixed the gloo bug, but it has to wait for Pytorch 2.10 (same for the cudnn fixes), they arent seeming like they are going to cherry pick and make a 2.9.2 to fix both :(

Pytorch 2.10 tracker

pytorch/pytorch#170119

[hameerabbasi]

We can add bitsandbytes here but not in requirements-global.txt or requirements-cpu.txt because macOS doesn't support 8-bit optimizers and that's like 90% of what we use bitsandbytes for.

[O-J1]

Sounds good but that needs to be seperate (and tested!), I dont have a AMD gpu ;).

[O-J1]

Using this PR with Torch 2.10 worked when training SDXL, no errors in the console or sampling issues.

[dxqb]

all points discussed here are resolved and this PR is superceeded by #1266