I have pythorch 0.3.1 and it works fine. I guess you are trying to run it with the latest pytorch version? I think fast.ai can have some breakages above pytorch 0.3.1. I see recently there are commits to be 0.4 compatible though.
Thanks for the hint! I’m relatively attached to running PyTorch / master.
I have dug a bit further: Apparently the cudnn rnn backwards behaves strange with which grads are enabled and which are not.
So if anyone else runs into this: setting torch.backends.cudnn.enabled = False gets you around this.
Apparently there still is a bug in PyTorch to be fixed, though.
I look forward to that! My plan was to use sentencepiece as I hope to benefit from the subwords with German compound nouns - I did have lots of UNK in some previous experiments and it also seems to work well for OpenNMT.
In the meantime: Did I miss something about more compatibility issues?
I’m now at learner.lr_find(start_lr=lrs/10, end_lr=lrs*10, linear=True) and get
TypeError: cannot assign 'torch.cuda.FloatTensor' as parameter 'weight_hh_l0' (torch.nn.Parameter or None expected)
I think I should know how to fix it if noone else did yet.
I’m pretty sure the nn.Module code in PyTorch wants to keep you from overwriting a parameter with a variable - as the regularizer does (there is a PyTorch issue where I have opinions on how to deal with “calculated parameters” which would be a neat solution here, too) - but it might depend on other factors like the python version whether that works. Interestingly, the code has been there for a long time.
I made it work (but not without also some _raw vs. not so _raw hacking in load_module). I’m not sure that I want to impose it on you if the problem doesn’t exist for anyone else.
I’m still getting the error, inconsistent range of Tensor input. Has it been fixed ? I’m on Pytorch master (0.4) . Plus I think there is some sort of memory leak happening. The code doesn’t run but whole RAM is occupied.