Using QRNN in Language Models


#21

This was very specific since there is a part of the model where we need to use C-accelerated libraries (here cupy) because the QRNN block isn’t implemented in pytorch. For the rest, I just took the pytorch existing implementation and adapted it inside the existing modules of fastai.

I never said this, I said there was no shared pretrained QRNN model yet. On my experiments, using a QRNN model pretrained on WT-103 then finetuned on imdb worked better, as with the LSTMs.


#22

@sgugger I’ve installed cupy-cuda91 with pip according to https://docs-cupy.chainer.org/en/stable/install.html and am using fastai 1.0.39. The package itself seem to be installed and import goes through, but when I run text_classifier_learner(data_clas, qrnn=True) I get FileNotFoundError: [Errno 2] No such file or directory: ‘/home/ubuntu/.local/share/virtualenvs/ubuntu-7Wf190Ea/lib/python3.6/site-packages/fastai/text/qrnn/forget_mult_cuda.cpp’
Not sure why the file is not installed, any suggestions would be most appreciated!


#23

You don’t need cupy anymore with fastai v1 since I rewrote the c extensions. I’m not sure why you don’t have those files with fastai 1.0.39, they are definitely in the repo. Maybe try a dev install?