Fantastic, thank you, I am training dogs_cats without any issues.
I have noticed that the GPU utilization is significantly lower than with fastai 0.7, averaging only 30% compared to nearly 100% before, is this likely to be because I have reduced the number of workers and therefore the speed at which data is fed to the GPU?
Yup, youâre limited by the CPU now since you canât multiprocess the data fetching.
And Iâm guessing there is no easy fix to the multiprocessing issue on windows?
Hi Fellas. I was playing with V1âs augmentor, and had a nasty surprise. I then replicated the issue inside an Ipython notebook, just to leave out any jupyter-related issue.
Above you can see a python prompt from my personal machine. fastai.vision.transform is imported without any fuss, BUT the kernel does not see the function apply_tmfs() (Still, it is present in image.py).
Below, you can see a python prompt from a machine at my workplace upon which I installed V1. Same install method. The kernel sees apply_tmfs() without any complaint.
Any suggestion?
In the first one youâve not spelt it correctly. Use tab-completion to help.
I am starting to move to jupyter-lab (instaed of notebook) and in the enviroment I created to use fastai, jupyter notebook imports fastai without problems, but jupyter-lab does not find the library.
Any idea?
We need a lot more info to help you.
Iâll merge this in to the installation thread.
Thanks boss. You are right, I didnât spell it correctly since tab-completion didnât work, and it didnt work since apply_tfms() is not seen. Look:
If I just import it in another way, the kernel does see apply_tfms()! This is madnessâŚ
I even wiped the whole environment, did a conda clean --all to flush any broken package in the cache repository, and reinstalled the whole env from scratch. Same behaviour.
Importing the whole fastai and calling vision.transform.apply_tfms() DOES WORK. It doesnât like from fastai.vision.transform import *, but just for transform, since everything else imported this way does work as wellâŚ
From within the jupyter lab, I try to execute the tabular nb:
[1] !conda list
# packages in environment at /Users/thomascapelle/anaconda3/envs/fastai:
#
# Name Version Build Channel
appnope 0.1.0 py37_0
asn1crypto 0.24.0 py37_0
backcall 0.1.0 py37_0
blas 1.0 mkl
bleach 3.0.2 py37_0
ca-certificates 2018.03.07 0
certifi 2018.10.15 py37_0
cffi 1.11.5 py37h6174b99_1
chardet 3.0.4 py37_1
cryptography 2.3.1 py37hdbc3d79_0
cycler 0.10.0 py37_0
cymem 1.31.2 py37h04f5b5a_0
cytoolz 0.9.0.1 py37h1de35cc_1
dataclasses 0.6 py_0 fastai
dbus 1.13.2 h760590f_1
decorator 4.3.0 py37_0
dill 0.2.8.2 py37_0
entrypoints 0.2.3 py37_2
expat 2.2.6 h0a44026_0
fastai 1.0.6 py_1 fastai
fastprogress 0.1.10 py_0 fastai
freetype 2.9.1 hb4e5f40_0
gettext 0.19.8.1 h15daf44_3
glib 2.56.2 hd9629dc_0
icu 58.2 h4b95b61_1
idna 2.7 py37_0
intel-openmp 2019.0 118
ipykernel 5.0.0 py37h39e3cac_0
ipython 7.0.1 py37h39e3cac_0
ipython_genutils 0.2.0 py37_0
ipywidgets 7.4.2 py37_0
jedi 0.13.1 py37_0
jinja2 2.10 py37_0
jpeg 9b he5867d9_2
jsonschema 2.6.0 py37_0
jupyter 1.0.0 py37_7
jupyter_client 5.2.3 py37_0
jupyter_console 6.0.0 py37_0
jupyter_core 4.4.0 py37_0
kiwisolver 1.0.1 py37h0a44026_0
libcxx 4.0.1 h579ed51_0
libcxxabi 4.0.1 hebd6815_0
libedit 3.1.20170329 hb402a30_2
libffi 3.2.1 h475c297_4
libgfortran 3.0.1 h93005f0_2
libiconv 1.15 hdd342a3_7
libpng 1.6.35 ha441bb4_0
libsodium 1.0.16 h3efe00b_0
libtiff 4.0.9 hcb84e12_2
markupsafe 1.0 py37h1de35cc_1
matplotlib 3.0.0 py37h54f8f79_0
mistune 0.8.4 py37h1de35cc_0
mkl 2019.0 118
mkl_fft 1.0.6 py37hb8a8100_0
mkl_random 1.0.1 py37h5d10147_1
msgpack-numpy 0.4.4.1 py37_0
msgpack-python 0.5.6 py37h04f5b5a_1
murmurhash 0.28.0 py37h0a44026_0
nbconvert 5.3.1 py37_0
nbformat 4.4.0 py37_0
ncurses 6.1 h0a44026_0
ninja 1.8.2 py37h04f5b5a_1
notebook 5.7.0 py37_0
numpy 1.15.2 py37h6a91979_1
numpy-base 1.15.2 py37h8a80b8c_1
olefile 0.46 py37_0
openssl 1.0.2p h1de35cc_0
pandas 0.23.4 py37h6440ff4_0
pandoc 2.2.3.2 0
pandocfilters 1.4.2 py37_1
parso 0.3.1 py37_0
pcre 8.42 h378b8a2_0
pexpect 4.6.0 py37_0
pickleshare 0.7.5 py37_0
pillow 5.3.0 py37hb68e598_0
pip 10.0.1 py37_0
plac 0.9.6 py37_0
preshed 1.0.1 py37h0a44026_0
prometheus_client 0.4.2 py37_0
prompt_toolkit 2.0.6 py37_0
ptyprocess 0.6.0 py37_0
pycparser 2.19 py37_0
pygments 2.2.0 py37_0
pyopenssl 18.0.0 py37_0
pyparsing 2.2.2 py37_0
pyqt 5.9.2 py37h655552a_2
pysocks 1.6.8 py37_0
python 3.7.0 hc167b69_0
python-dateutil 2.7.3 py37_0
pytorch-nightly-cpu 1.0.0.dev20181014 py3.7_0 pytorch
pytz 2018.5 py37_0
pyzmq 17.1.2 py37h1de35cc_0
qt 5.9.6 h45cd832_2
qtconsole 4.4.1 py37_0
readline 7.0 h1de35cc_5
regex 2018.07.11 py37h1de35cc_0
requests 2.19.1 py37_0
scipy 1.1.0 py37h28f7352_1
send2trash 1.5.0 py37_0
setuptools 40.4.3 py37_0
simplegeneric 0.8.1 py37_2
sip 4.19.8 py37h0a44026_0
six 1.11.0 py37_1
spacy 2.0.12 py37h6440ff4_0
sqlite 3.25.2 ha441bb4_0
termcolor 1.1.0 py37_1
terminado 0.8.1 py37_1
testpath 0.4.2 py37_0
thinc 6.10.3 py37h6440ff4_0
tk 8.6.8 ha441bb4_0
toolz 0.9.0 py37_0
torchvision-nightly-cpu 0.2.1 pyh19dea27_0 fastai
tornado 5.1.1 py37h1de35cc_0
tqdm 4.26.0 py37h28b3542_0
traitlets 4.3.2 py37_0
typing 3.6.4 py37_0
ujson 1.35 py37h1de35cc_0
urllib3 1.23 py37_0
wcwidth 0.1.7 py37_0
webencodings 0.5.1 py37_1
wheel 0.32.1 py37_0
widgetsnbextension 3.4.2 py37_0
wrapt 1.10.11 py37h1de35cc_2
xz 5.2.4 h1de35cc_4
zeromq 4.2.5 h0a44026_1
zlib 1.2.11 hf3cbc9b_2
[2]from fastai import * # Quick access to most common functionality
from fastai.tabular import * # Quick access to tabular functionality
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-1-eb17ac0e42e6> in <module>()
----> 1 from fastai import * # Quick access to most common functionality
2 from fastai.tabular import * # Quick access to tabular functionality
ModuleNotFoundError: No module named 'fastai'
If I do exactly the same thing, in the jupyter notebook instead of lab, it works.
I am in a mac, with CPU versions of torch.
What other info I need to provide?
Thanks for the extra info. apply_tfms isnât defined in that module. Generally itâs easiest to simply from fastai import * and from fastai.vision import * (assuming you are doing a vision application) then youâll have everything you need. You can then type apply_tfms and jupyter will return <function fastai.vision.image.apply_tfms(... so you can see where it came from. Youâll also find it in the vision.image section of the docs:
Thanks for your quick and effective reply.
But allow me one more question: If my personal box shows the expected behaviour, why did the DL box at work behave differently?
Thanks!
Yes I was reading it right now, it is a pleasure to read such a well-written documentation. Job well done!
I checked, and reinstalling jupyter-lab solved the issue. There are many threads about conda enviroments not showing in jupyter,
I hope that when jupyter-lab is officially released this is solved.
You can create a new, empty enviroment in conda, and call jupyter-lab/notebook and it will open, calling the system jupyter. This should not be allowed.
@tcapelle , I documented a summary of that SO thread here:
https://docs-dev.fast.ai/troubleshoot#conda-environments-not-showing-up-in-jupyter-notebook
Since I canât reproduce this issue, please review and let me know if anything needs to be changed in that new section.
If someone can give me a set of commands to run including creating a new conda environment and indicate at which stage the problem shows up - that will help me to reproduce the problem - and solve it.
sorry if thatâs the incorrect thread to reply on⌠do you know when pytorch1.0 is going to be ready in windows (rather than building myself)?
i have tried a whole day, installing vs2017 and follow the steps in pytorch website but canât get it compiledâŚ
i need to do a trial and error on installing cmake, msbuild, vs2017, v141âŚetc but still canât get it done.
They said at the conference the v1 would really be out around NIPS though you can also ask that question on the pytorch forum 
Just wanted to post that I built a new Ubuntu 18.04 box today with a 1080ti and was able to get it all installed fairly straightforward(only 1 full OS re-install).
I mainly followed this guide - http://blog.jeffhaluska.com/adventures-in-installing-pytorch-in-ubuntu-18-04/
Instead of using ubuntu gui(first 2 steps) to install nvidia drivers I did it through apt install the ppa repo:
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update
sudo apt install nvidia-driver-410
I did conda install for pytorch with this:
conda install pytorch-nightly cuda92 -c pytorch
how install the latest version of fastai?
running conda update fastai -c fastai will only install version 1.0.6
UPDATE: manage to install the latest version after doing these steps:
conda update conda -yconda install anaconda -yIf you encounter any permission error, do this:
sudo chown -R user /home/user/src
where user is the username, /home/user/src is the folder the where anaconda is installed in Ubuntu
Iâm getting âImportError: libcuda.so.1: cannot open shared object file: No such file or directoryâ
As per the Troubleshooting thread, I my driver is
nvidia-396/xenial,now 396.54-0ubuntu0~gpu16.04.1 amd64 [installed]
, and nvidia-smi is working.
$ find /usr/ | grep libcuda.so
/usr/share/man/man7/libcuda.so.7
/usr/local/cuda-9.2/lib64/stubs/libcuda.so
/usr/local/cuda-9.2/doc/man/man7/libcuda.so.7
âŚthese are from the CUDA that I just installed tonight in trying to get FastAI working.
On this Ubuntu 16.04 system, previously I was using CUDA 9.1 with CUDNN7.0 with PyTorch and it worked fine with my GPU. The things that are broken tonight are a result of trying to follow the Fast.ai installation instructions.
Iâve created a fresh conda environment, ran the 3 conda install lines to install pytorch and the fastai packages, but when I run the test line, I get
$ python -c 'import fastai; fastai.show_install(1)'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/anaconda/envs/fastai/lib/python3.6/site-packages/fastai/__init__.py", line 1, in <module>
from .basic_train import *
File "/opt/anaconda/envs/fastai/lib/python3.6/site-packages/fastai/basic_train.py", line 2, in <module>
from .torch_core import *
File "/opt/anaconda/envs/fastai/lib/python3.6/site-packages/fastai/torch_core.py", line 2, in <module>
from .imports.torch import *
File "/opt/anaconda/envs/fastai/lib/python3.6/site-packages/fastai/imports/__init__.py", line 2, in <module>
from .torch import *
File "/opt/anaconda/envs/fastai/lib/python3.6/site-packages/fastai/imports/torch.py", line 1, in <module>
import torch, torch.nn.functional as F
File "/opt/anaconda/envs/fastai/lib/python3.6/site-packages/torch/__init__.py", line 84, in <module>
from torch._C import *
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory
Would appreciate any help! Thanks.
PS- I do need CUDA installed ânormallyâ on my system, because I also have a project that uses Keras that I need to run on this machine. (âŚOh geez, and I removed CUDA 9.1 in doing the Troubleshooting, but my Tensorflow was built from source using 9.1!)
There is an issue with the recent fastai conda packages if you have an outdated anaconda package.
This is caused by an outdated anaconda package, which wants a numpy < 1.15
do
conda install anaconda
which should install anaconda 5.3.0 or higher.
and then it should work.
I changed meta.yaml deps to not ask for numpy>=1.15, but rolled back to >=1.12 so the next release should take care of it.