Hi @everyone,
Upon executing this line:
I am getting this error:
I am using a computer with 12GB Nvidia Titan Xp.
Upon doing Nvidia-smi, I am getting this:
Should I reset my GPU or is there any other good option?
Hi @everyone,
Upon executing this line:
I am getting this error:
I am using a computer with 12GB Nvidia Titan Xp.
Upon doing Nvidia-smi, I am getting this:
Should I reset my GPU or is there any other good option?
Just keep reducing the batch size until it runs. Jeremy also recommends restarting the kernel after a cuda failure.
I think the problem is not the batch size I used. The problem is my GPU memory is being used without running many processes. I need to free my space first. I have used a lot of PyTorch cuda processes and they have not freed the memory I guess. I might be wrong but I think we need to free memory after a process has used it in PyTorch.
That’s a big model, so you’ll need a small batchsize. Try bs=24.
I am getting memory errors when I try changing batch sizes. The same error comes up whenever I play with bs (tried higher, lower, all resnets). Despite the error, it still seems to complete. Am I setting the batch size wrong?
A snippet of code from planets.
i have a 8GB GTX 1080 - needed to decrease the batch size to 16 to make it work
That’s the thing. It appears to work (currently at bs = 10), it just tosses those two errors initially. As I go up into larger size images it does not trigger again.
Update: Looks like this is just a warning and I can ignore it
@mindtrinket that’s not showing any memory error. It’s just a warning about an unrelated issue - there’s another thread about it if you’re interested in learning more