3.9 is the latest stable version of Python, but I use 3.10 and have not thus far encountered any errors with PyTorch or fastai. However, note that some libraries like TensorFlow only support up to 3.9, so that’s what you might be better off with.
The nice thing about PyTorch is that it takes care of CUDA dependencies for you (regardless of whether your package manager is pip or Anaconda), therefore no need to worry about them. You can check out the PyTorch docs for more info on that.
Oh, and fastai recently announced fastchan, a “conda mini-distribution with a focus on the PyTorch ecosystem” that makes the “installation and updates of libraries such as PyTorch and RAPIDS […] faster, easier, and more reliable”. Considering your situation, it may be worthwhile to give it a look.