What changes to do when I shift to multi-GPU setup?

(Kushajveer Singh) #1

I have started using Google credits to work on GCP. I currently use a single GPU but would be using multi GPU after some time.

So I wanted to know what things to change in my code. Are there are best practices when using multi-GPU that I should be aware of?

Setup:-

  • Google Cloud Platform
  • PyTorch + fastai
0 Likes

(Divyansh Jha) #2

Try this out.
https://docs.fast.ai/distributed.html

1 Like

(William Minshew) #3

I’d check out horovod: https://github.com/horovod/horovod#usage

Not sure how it compares to pytorch’s distributed package, but afaik it compares quite favorably to TF’s (https://eng.uber.com/horovod/)

1 Like

#4

If you’re working with fastai, this (distributed) is the first answer.

0 Likes