Wow, thanks for this, Vijayabhaskar J!
Thank you Archaeologist!
I agree with Archaeologist and Vijay that this is not a bug.
I have looked around and saw that other people had issues similar to this, specifically for reproducible research.
It seems like the problem is that there are several different randomness going on: for example, for python, and pytorch.
I tried to set seed for randomnesses in multiple applications as suggested in other posts, but did not get the same batch as some people did.