Error while importing tensorflow

Part 2 (2017)
1

Hello,

I am getting error while installing setting up the environment for Part 2.
I followed Jeremy’s steps(posted by him on the forum), but when i do import tensorflow command i get the following error:

import tensorflow

ImportError Traceback (most recent call last)
~/anaconda3/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py in ()
40 sys.setdlopenflags(_default_dlopen_flags | ctypes.RTLD_GLOBAL)
—> 41 from tensorflow.python.pywrap_tensorflow_internal import *
42 from tensorflow.python.pywrap_tensorflow_internal import version

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py in ()
27 return _mod
—> 28 _pywrap_tensorflow_internal = swig_import_helper()
29 del swig_import_helper

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py in swig_import_helper()
23 try:
—> 24 _mod = imp.load_module(’_pywrap_tensorflow_internal’, fp, pathname, description)
25 finally:

~/anaconda3/lib/python3.6/imp.py in load_module(name, file, filename, details)
241 else:
–> 242 return load_dynamic(name, filename, file)
243 elif type_ == PKG_DIRECTORY:

~/anaconda3/lib/python3.6/imp.py in load_dynamic(name, path, file)
341 name=name, loader=loader, origin=path)
–> 342 return _load(spec)
343

ImportError: libcudnn.so.6: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

ImportError Traceback (most recent call last)
in ()
----> 1 import tensorflow

~/anaconda3/lib/python3.6/site-packages/tensorflow/init.py in ()
22
23 # pylint: disable=wildcard-import
—> 24 from tensorflow.python import *
25 # pylint: enable=wildcard-import
26

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/init.py in ()
47 import numpy as np
48
—> 49 from tensorflow.python import pywrap_tensorflow
50
51 # Protocol buffers

~/anaconda3/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py in ()
50 for some common reasons and solutions. Include the entire stack trace
51 above this error message when asking for help.""" % traceback.format_exc()
—> 52 raise ImportError(msg)
53
54 # pylint: enable=wildcard-import,g-import-not-at-top,unused-import,line-too-long

ImportError: Traceback (most recent call last):
File “/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py”, line 41, in
from tensorflow.python.pywrap_tensorflow_internal import *
File “/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py”, line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File “/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py”, line 24, in swig_import_helper
_mod = imp.load_module(’_pywrap_tensorflow_internal’, fp, pathname, description)
File “/home/ubuntu/anaconda3/lib/python3.6/imp.py”, line 242, in load_module
return load_dynamic(name, filename, file)
File “/home/ubuntu/anaconda3/lib/python3.6/imp.py”, line 342, in load_dynamic
return _load(spec)
ImportError: libcudnn.so.6: cannot open shared object file: No such file or directory

Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/install_sources#common_installation_problems

for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.

I am unable to resolve this. Any help is much appreciated!

Thanks.
Naman

2

I think you have installed cudnn 5.0 while it needs libcudnn.so.6, and it couldn’t find libcudnn.so.6. That means your tensorflow needs cudnn 6.x, so better install cudnn 6.x

1 ) In command line check the output of nvidia-smi. Check the driver version. If error then you need to install drivers.
Then install CudaNN version relative to that driver version above.

2 ) If step one is okay.Then try adding these two steps:-

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:~/cuda/lib64"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/home/yourusername/cuda/lib64"

3 ) If the above two steps does not work out then check out this link for a clean slate install. You need to remove existing installations and perform a sudo apt-get update before following the steps in the link.

Edit 1:-
You may check this out:-

  1. Ubuntu 16.04
  2. Cuda Toolkit 8.0’
  3. Tensorflow-gpu
  4. CudaNN v6
3

Hi ,

I am just following the commands shared by Jeremy in the below post:

Could you please tell what needs to be modified in this post to get it to work just fine.

Thanks,
Naman

4

Check the 3rd comment by Jeremy. You need to use tensorflow v1.0 only.Then steps suggested by Jeremy will work. When you are executing pip install tensorflow-gpu it is installing the latest version. You need to specify it to v1.0.

5

So the only change i need to make is,

pip install tensorflow-gpu ==1.0 And all the commands will be as is, right?

6

Yes. Try that.

7

HI,

I am unable to install tensorflow version 1.0.0 by the above command. It doesn’t seem to work.

Command - nvidia-smi gives below o/p:
-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.66 Driver Version: 384.66 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K80 Off | 00000000:00:1E.0 Off | 0 |
| N/A 41C P0 57W / 149W | 0MiB / 11439MiB | 99% Default |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
±----------------------------------------------------------------------------+

Also in the command – export LD_LIBRARY_PATH=/usr/local/cuda/lib64/
the path doesn’t exist by default.

As per Jeremy’s code i am installing everything in cd/downloads/cuda

Path : /downloads/cuda/lib64 gives the o/p as,
libcudnn.so libcudnn.so.5.1.10 libcudnn_static.a
libcudnn.so.5 libcudnn.so.5.1.3

Furthermore, if i run below two commands, it doesn’t do anything because i think the path doesn’t exist by default.
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:~/cuda/lib64"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/home/yourusername/cuda/lib64"

Kindly please help. I am stuck and unable to move forward.

Thanks,
Naman

8

The issue is you are installing mismatched version. Nvidia driver install is ok.
This is what you need to have as of today (This is different from Jeremy.)

  1. Ubuntu 16.04
  2. Cuda Toolkit 8.0’ or cudart 8.0 for Linux
  3. Tensorflow-gpu (Linux)
  4. CudaNN v6.0 for cuda toolkit 8 (Linux)

Are you running sudo apt-get update after installing any drivers or tools ?
In stackoverflow link that I provided above check the answer for Alexander Yau. He gave a step by step instructions of how to install. Mind that the answer is related to CudaNN v5.1 while you need v6.0.
So only select cuda v6.0 when you are at download page.

9

Hi,

I run the commands in the order as:-

  1. sudo apt update

  2. sudo apt upgrade

  3. cd downloads/

  4. wget https://repo.continuum.io/archive/Anaconda3-4.3.0-Linux-x86_64.sh

  5. bash Anaconda3-4.3.0-Linux-x86_64.sh -b

  6. cd

  7. vim .bashrc

  8. sudo shutdown -r now

  9. cd downloads/

  10. wget http://files.fast.ai/files/cudnn-8.0-linux-x64-v5.1.tgz

  11. tar -zxf cudnn-8.0-linux-x64-v5.1.tgz

  12. cd cuda/

  13. sudo cp lib64/* /usr/local/cuda/lib64/

  14. sudo cp include/* /usr/local/cuda/include/

  15. pip install tensorflow-gpu

  16. conda install bcolz

  17. conda update --all

  18. pip install git+git://github.com/fchollet/keras.git

  19. echo ‘{
    “image_dim_ordering”: “tf”,
    “epsilon”: 1e-07,
    “floatx”: “float32”,
    “backend”: “tensorflow”
    }’ > ~/.keras/keras.json

  20. To check whether Tensorflow is installed properly:-
    Launch iPython with command – > ipython
    Then , import tensorflow
    Now after this step i get the error.

So technically which steps do i need to change and what will be the commands to do the same.
Your help is very much appreciated!!!

Also, where can i get CuDNN v6.0 ? I think this is the only part i am missing.
Thank you,
Naman

10

I see where the problem is. First you are using a script which is a update to previous script that used theano, keras and now using tensorflow-gpu. This is why in these 19 step script there are a few missing steps after 7 and before 8. You are using vim .bashrc. Then you are not adding the anaconda path there. Assuming you are a beginner mere executing the 19 steps wont give you a clear picture (you are getting errors). Follow the link below that I found on medium. The blog explains step by step installation better. Tough it is slightly dissimilar to the above mentioned steps end results will be the same e.g setting up an environment.

11

Hi,

I am changing the path of anaconda2 to anaconda3 as mentioned in the Jeremy’s post (to be performed after Step 7 and before Step 8) as ,

Once vim is open, replace the anaconda 2 path with anaconda 3 by simply changing the 2 to 3 (it’ll probably be the last line of the file). As a reminder, press ‘i’ to enter insert (editing) mode, Esc when done editing, and ‘:wq’ to write the file and exit the editor. Next, reboot the instance to ensure the new drivers are loaded:

However, i will also go through the post shared by you and will let you know how it goes.

Thank you very much. Appreciate your help!

Will let you know soon.

Thanks,
Naman

12

‘pip install tensorflow-gpu ==1.0’ is not right.
run the following command will install tensorflow v1.0 for python3.6 and GPU support, then it should be ok.
pip install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.0.1-cp36-cp36m-linux_x86_64.whl

13

Pip install --upgrade is not needed always. So it is not required in every case.

14

Correct me if i am wrong. You are saying to follow Jeremy’s steps and run pip install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.0.1-cp36-cp36m-linux_x86_64.whl
in the very beginning(should be step 1). Is that right?

15

just replace step 15 with this command, or run it at last if you finished all steps and found tensorflow didn’t work.

16

Is it working ?

17

Thank you so much. It Worked!!!

18

HI,
Now i am getting the below error while running the code:-

Obtain_input_shape() got an unexpected keyword argument ‘dim_ordering

While running the code
model = VGG16_Avg(include_top=False)
in Neural Transfer notebook i am getting the error as:
ypeError Traceback (most recent call last)
in ()
----> 1 model = VGG16_Avg(include_top=False)

~/nbs/vgg16_avg.py in VGG16_Avg(include_top, weights, input_tensor, input_shape, classes)
36 min_size=48,
37 data_format=K.image_data_format(),
—> 38 include_top=include_top)
39
40 if input_tensor is None:

TypeError: obtaininput_shape() got an unexpected keyword argument ‘dim_ordering’

As per the suggestion in the post i changed the ‘dim_ordering’ part in the code to ‘data_format’, but am still getting the same error.
input_shape = obtaininput_shape(input_shape,
default_size=224,
min_size=48,
data_format=K.image_data_format(),
include_top=include_top)

I also changed

load weights
if weights == ‘imagenet’:
if K.image_dim_ordering() == ‘th’:
to K.image_data_format() == ‘th’:
but to no avail.

I am using the below commands for installing all the softwares in my p2.xlarge machine:-

sudo apt update
sudo apt upgrade
cd downloads/
wget https://repo.continuum.io/archive/Anaconda3-4.3.0-Linux-x86_64.sh
bash Anaconda3-4.3.0-Linux-x86_64.sh -b
cd
vim .bashrc
sudo shutdown -r now
cd downloads/
wget http://files.fast.ai/files/cudnn-8.0-linux-x64-v5.1.tgz
tar -zxf cudnn-8.0-linux-x64-v5.1.tgz
cd cuda/
sudo cp lib64/* /usr/local/cuda/lib64/
sudo cp include/* /usr/local/cuda/include/
pip install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.0.1-cp36-cp36m-linux_x86_64.whl
conda install bcolz
conda update --all
pip install git+git://github.com/fchollet/keras.git
echo ‘{
“image_dim_ordering”: “tf”,
“epsilon”: 1e-07,
“floatx”: “float32”,
“backend”: “tensorflow”
}’ > ~/.keras/keras.json

Please help!

Thanks,
Naman

19

Nothing wrong with installation. As you mentioned before it is working. Now you installed tensorflow but using codes for theano. FYI, Image dimension ordering is different in Theano and Tensorflow where Channels comes first in former and last in later.

Anyway coming to your problem.Do you see the typo “input_shape = obtaininput_shape()” in code below. I think that is the error you made.

Change and see if it works.

20

HI,

I had already changed the code from ,

input_shape = _obtain_input_shape(input_shape,
default_size=224,
min_size=48,
dim_ordering=K.image_dim_ordering(),
include_top=include_top)

To,

input_shape = obtaininput_shape(input_shape,
default_size=224,
min_size=48,
data_format=K.image_data_format(),
include_top=include_top)

And it wasn’t working.

That’s the only change i made after the following the above 19 steps for installation.

Can you please clearly specify the change that i need to make Also,and what else do i need to change?

Thanks,
naman