How to use Multiple GPUs?

[balnazzar (Andrea de Luca)]

@GrahamAcademy

Have you found anything out about this? I have dual Titan RTXs with nvlink and I would love to be able to pool resources.

I was unable to find anything solid about that. Should you succeed, please inform us. Thanks!

[GrahamAcademy (Robin Graham)]

I don’t think it can be done in windows. I need to dual boot with Linux to do more testing.

[dtpdx (dtp)]

So I followed Jeremy’s advice to use the multiple GPUs to run experiments in parallel, which is outstanding. However, my data wasn’t saved. Currently troubleshooting. If anyone knows how to save the model from the alternate GPU back to disk, that would be great. I’m currently perusing this link to find what it is I think I need: Saving and loading models across devices in PyTorch — PyTorch Tutorials 1.9.0+cu102 documentation