RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

fastai fastai dev
1

When running:

from fastai2.text.all import *

dls = TextDataLoaders.from_folder(untar_data(URLs.IMDB), valid='test')
learn = text_classifier_learner(dls, AWD_LSTM, drop_mult=0.5, metrics=accuracy)
learn.fine_tune(4, 1e-2)

I get:

RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

Enyone else experiencing this as well?

Setup:
fastai2 0.0.13 pypi_0 pypi
pytorch 1.4.0 py3.7_cuda10.1.243_cudnn7.6.3_0 pytorch
torchvision 0.5.0 py37_cu101 pytorch

1 Like
2

It looks like a problem in your CUDNN installation. Could you tell us more about your enviromnent?

3

I am on Paperspace + Fast.AI 2.0 (V4)

Python 3.7.6 and packages are:

Package Version

adal 1.2.2
asn1crypto 1.3.0
attrs 19.3.0
azure-cognitiveservices-search-imagesearch 2.0.0
azure-common 1.1.25
backcall 0.1.0
beautifulsoup4 4.8.2
bleach 3.1.0
blis 0.4.1
Bottleneck 1.3.2
catalogue 1.0.0
certifi 2019.11.28
cffi 1.14.0
chardet 3.0.4
cryptography 2.8
cycler 0.10.0
cymem 2.0.2
dataclasses 0.6
decorator 4.4.2
defusedxml 0.6.0
entrypoints 0.3
fastai 1.0.60
fastai2 0.0.13
fastcore 0.1.15
fastprocess 0.1.0
fastprogress 0.2.2
fastscript 0.1.4
graphviz 0.13.2
idna 2.9
importlib-metadata 1.5.0
ipykernel 5.1.4
ipython 7.13.0
ipython-genutils 0.2.0
ipywidgets 7.5.1
isodate 0.6.0
jedi 0.16.0
Jinja2 2.11.1
joblib 0.14.1
jsonschema 3.2.0
jupyter-client 6.0.0
jupyter-console 6.1.0
jupyter-core 4.6.1
kiwisolver 1.1.0
MarkupSafe 1.1.1
matplotlib 3.2.0
mistune 0.8.4
msrest 0.6.11
msrestazure 0.6.3
murmurhash 1.0.2
nbconvert 5.6.1
nbdev 0.2.14
nbformat 5.0.4
notebook 6.0.3
numexpr 2.7.1
numpy 1.18.1
nvidia-ml-py3 7.352.0
oauthlib 3.1.0
olefile 0.46
packaging 20.3
pandas 1.0.2
pandocfilters 1.4.2
parso 0.6.2
pexpect 4.8.0
pickleshare 0.7.5
Pillow 7.0.0
pip 20.0.2
plac 0.9.6
preshed 3.0.2
prometheus-client 0.7.1
prompt-toolkit 3.0.3
ptyprocess 0.6.0
pycparser 2.20
Pygments 2.6.1
PyJWT 1.7.1
pyOpenSSL 19.1.0
pyparsing 2.4.6
PyQt5 5.12.3
PyQt5-sip 4.19.18
PyQtWebEngine 5.12.1
pyrsistent 0.15.7
PySocks 1.7.1
python-dateutil 2.8.1
pytz 2019.3
PyYAML 5.3
pyzmq 18.1.1
qtconsole 4.7.1
QtPy 1.9.0
requests 2.23.0
requests-oauthlib 1.3.0
scikit-learn 0.22.2.post1
scipy 1.4.1
Send2Trash 1.5.0
setuptools 46.0.0.post20200309
six 1.14.0
soupsieve 2.0
spacy 2.2.3
srsly 1.0.0
terminado 0.8.3
testpath 0.4.4
thinc 7.3.0
torch 1.4.0
torchvision 0.5.0
tornado 6.0.4
tqdm 4.43.0
traitlets 4.3.3
urllib3 1.25.8
utils 1.0.1
wasabi 0.6.0
wcwidth 0.1.8
webencodings 0.5.1
wheel 0.34.2
widgetsnbextension 3.5.1
zipp 2.2.0

5

Good Evening - has anyone found a solution to this? I had the same issue Intro (01_intro.ipynb) problem - RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

Update: I may have found a solution, but I’m not sure if it’s the “right” way to do it.

1 Like
6

I ran into what appeared to be the same problem. I’m trying to run everything on my laptop, so it is not unlikely that I have something misconfigured. Nevertheless, restarting the kernel and running a second time worked without any errors.

7

I have the same problem running the same lines of code from the fastbook lesson 1 in the example about sentiment analysis. I am running the code in Google Colab. At the beginning of my notebook I update the fastai library to avoid some issues with the version imported by default in Colab with

!pip install fastai --upgrade