I’m on ubuntu 18 and I don’t have any issue with it.
Perhaps switch to using a conda env? This is what I use. Somethink like: http://docs-dev.fast.ai/release#run-install-tests-in-a-fresh-environment
I think it’s an important recommendation because you don’t want to mess with system-wide packages. Earlier I did and broke something things. This time around I don’t touch system-wide python at all.
I will add this recommendation to docs.
I think you are exactly right. In my case, it’s probably less about fastai messing with my system, as it is my system having messed up settings, causing fastai’s installation to reveal the brokenness. I say this because when I follow the docs on a fresh vm without any modification, I have no issues. Thanks for your pointers re: conda.
Have you tried, with CUDA 9.2?
Just another gentle nudge to send you guys into creating a windows thread and relocate that discussion there if you will 
it will help others.
I have just upgraged to V1 on a paper specs machine and ran into cuda not available problems. In my case I got it working with just the latter part of these instructions, that is:
sudo apt-get —purge remove nvidia-387
sudo apt-get -f install
sudo reboot now
After this it was working ok and after changing the symlinks I was able to use the old library workbooks also
Many thanks
Hi, I am not sure if my current issue is with windows or the version of fastai I am using, can you confirm that the installed version 1.0.6 is compiled from commit 62091ed651fb8a07587fd6e3da805415bb6fd8e0?
I ask because I am getting an error with the data loader. If I run through the dogs_cats notebook
%reload_ext autoreload
%autoreload 2
from fastai import *
from fastai.vision import *
path = untar_data(URLs.DOGS)
data = ImageDataBunch.from_folder(path, ds_tfms=get_transforms(), tfms=imagenet_norm, size=224)
and then call
next(iter(data.train_dl))
I get the following error
PicklingError Traceback (most recent call last)
in
----> 1 next(iter(data.train_dl))d:\ssdbackup\dev\repos\fastai_v1\fastai\data.py in iter(self)
50 def iter(self):
51 “Process and returns items fromDataLoader.”
—> 52 for b in self.dl: yield self.proc_batch(b)
53
54 def one_batch(self)->Collection[Tensor]:D:\c_progs\Anaconda3\envs\test_fastai\lib\site-packages\torch\utils\data\dataloader.py in iter(self)
817
818 def iter(self):
–> 819 return _DataLoaderIter(self)
820
821 def len(self):D:\c_progs\Anaconda3\envs\test_fastai\lib\site-packages\torch\utils\data\dataloader.py in init(self, loader)
558 # before it starts, and del tries to join but will get:
559 # AssertionError: can only join a started process.
–> 560 w.start()
561 self.index_queues.append(index_queue)
562 self.workers.append(w)D:\c_progs\Anaconda3\envs\test_fastai\lib\multiprocessing\process.py in start(self)
103 ‘daemonic processes are not allowed to have children’
104 _cleanup()
–> 105 self._popen = self._Popen(self)
106 self._sentinel = self._popen.sentinel
107 # Avoid a refcycle if the target function holds an indirectD:\c_progs\Anaconda3\envs\test_fastai\lib\multiprocessing\context.py in _Popen(process_obj)
221 @staticmethod
222 def _Popen(process_obj):
–> 223 return _default_context.get_context().Process._Popen(process_obj)
224
225 class DefaultContext(BaseContext):D:\c_progs\Anaconda3\envs\test_fastai\lib\multiprocessing\context.py in _Popen(process_obj)
320 def _Popen(process_obj):
321 from .popen_spawn_win32 import Popen
–> 322 return Popen(process_obj)
323
324 class SpawnContext(BaseContext):D:\c_progs\Anaconda3\envs\test_fastai\lib\multiprocessing\popen_spawn_win32.py in init(self, process_obj)
63 try:
64 reduction.dump(prep_data, to_child)
—> 65 reduction.dump(process_obj, to_child)
66 finally:
67 set_spawning_popen(None)D:\c_progs\Anaconda3\envs\test_fastai\lib\multiprocessing\reduction.py in dump(obj, file, protocol)
58 def dump(obj, file, protocol=None):
59 ‘’‘Replacement for pickle.dump() using ForkingPickler.’’’
—> 60 ForkingPickler(file, protocol).dump(obj)
61
62 #PicklingError: Can’t pickle <function crop_pad at 0x000000A0A8516378>: it’s not the same object as fastai.vision.transform.crop_pad
I am posting here because I don’t know if this is an windows install error or if it is a bug in the commit I have used to install from?
This is specific to Windows and multiprocessing. If you put num_workers=0 in your DataBunch, you should be fine.
Fantastic, thank you, I am training dogs_cats without any issues.
I have noticed that the GPU utilization is significantly lower than with fastai 0.7, averaging only 30% compared to nearly 100% before, is this likely to be because I have reduced the number of workers and therefore the speed at which data is fed to the GPU?
Yup, you’re limited by the CPU now since you can’t multiprocess the data fetching.
And I’m guessing there is no easy fix to the multiprocessing issue on windows?
Hi Fellas. I was playing with V1’s augmentor, and had a nasty surprise. I then replicated the issue inside an Ipython notebook, just to leave out any jupyter-related issue.
Above you can see a python prompt from my personal machine. fastai.vision.transform is imported without any fuss, BUT the kernel does not see the function apply_tmfs() (Still, it is present in image.py).
Below, you can see a python prompt from a machine at my workplace upon which I installed V1. Same install method. The kernel sees apply_tmfs() without any complaint.
Any suggestion?
In the first one you’ve not spelt it correctly. Use tab-completion to help.
I am starting to move to jupyter-lab (instaed of notebook) and in the enviroment I created to use fastai, jupyter notebook imports fastai without problems, but jupyter-lab does not find the library.
Any idea?
We need a lot more info to help you.
I’ll merge this in to the installation thread.
Thanks boss. You are right, I didn’t spell it correctly since tab-completion didn’t work, and it didnt work since apply_tfms() is not seen. Look:
If I just import it in another way, the kernel does see apply_tfms()! This is madness…
I even wiped the whole environment, did a conda clean --all to flush any broken package in the cache repository, and reinstalled the whole env from scratch. Same behaviour.
Importing the whole fastai and calling vision.transform.apply_tfms() DOES WORK. It doesn’t like from fastai.vision.transform import *, but just for transform, since everything else imported this way does work as well…
From within the jupyter lab, I try to execute the tabular nb:
[1] !conda list
# packages in environment at /Users/thomascapelle/anaconda3/envs/fastai:
#
# Name Version Build Channel
appnope 0.1.0 py37_0
asn1crypto 0.24.0 py37_0
backcall 0.1.0 py37_0
blas 1.0 mkl
bleach 3.0.2 py37_0
ca-certificates 2018.03.07 0
certifi 2018.10.15 py37_0
cffi 1.11.5 py37h6174b99_1
chardet 3.0.4 py37_1
cryptography 2.3.1 py37hdbc3d79_0
cycler 0.10.0 py37_0
cymem 1.31.2 py37h04f5b5a_0
cytoolz 0.9.0.1 py37h1de35cc_1
dataclasses 0.6 py_0 fastai
dbus 1.13.2 h760590f_1
decorator 4.3.0 py37_0
dill 0.2.8.2 py37_0
entrypoints 0.2.3 py37_2
expat 2.2.6 h0a44026_0
fastai 1.0.6 py_1 fastai
fastprogress 0.1.10 py_0 fastai
freetype 2.9.1 hb4e5f40_0
gettext 0.19.8.1 h15daf44_3
glib 2.56.2 hd9629dc_0
icu 58.2 h4b95b61_1
idna 2.7 py37_0
intel-openmp 2019.0 118
ipykernel 5.0.0 py37h39e3cac_0
ipython 7.0.1 py37h39e3cac_0
ipython_genutils 0.2.0 py37_0
ipywidgets 7.4.2 py37_0
jedi 0.13.1 py37_0
jinja2 2.10 py37_0
jpeg 9b he5867d9_2
jsonschema 2.6.0 py37_0
jupyter 1.0.0 py37_7
jupyter_client 5.2.3 py37_0
jupyter_console 6.0.0 py37_0
jupyter_core 4.4.0 py37_0
kiwisolver 1.0.1 py37h0a44026_0
libcxx 4.0.1 h579ed51_0
libcxxabi 4.0.1 hebd6815_0
libedit 3.1.20170329 hb402a30_2
libffi 3.2.1 h475c297_4
libgfortran 3.0.1 h93005f0_2
libiconv 1.15 hdd342a3_7
libpng 1.6.35 ha441bb4_0
libsodium 1.0.16 h3efe00b_0
libtiff 4.0.9 hcb84e12_2
markupsafe 1.0 py37h1de35cc_1
matplotlib 3.0.0 py37h54f8f79_0
mistune 0.8.4 py37h1de35cc_0
mkl 2019.0 118
mkl_fft 1.0.6 py37hb8a8100_0
mkl_random 1.0.1 py37h5d10147_1
msgpack-numpy 0.4.4.1 py37_0
msgpack-python 0.5.6 py37h04f5b5a_1
murmurhash 0.28.0 py37h0a44026_0
nbconvert 5.3.1 py37_0
nbformat 4.4.0 py37_0
ncurses 6.1 h0a44026_0
ninja 1.8.2 py37h04f5b5a_1
notebook 5.7.0 py37_0
numpy 1.15.2 py37h6a91979_1
numpy-base 1.15.2 py37h8a80b8c_1
olefile 0.46 py37_0
openssl 1.0.2p h1de35cc_0
pandas 0.23.4 py37h6440ff4_0
pandoc 2.2.3.2 0
pandocfilters 1.4.2 py37_1
parso 0.3.1 py37_0
pcre 8.42 h378b8a2_0
pexpect 4.6.0 py37_0
pickleshare 0.7.5 py37_0
pillow 5.3.0 py37hb68e598_0
pip 10.0.1 py37_0
plac 0.9.6 py37_0
preshed 1.0.1 py37h0a44026_0
prometheus_client 0.4.2 py37_0
prompt_toolkit 2.0.6 py37_0
ptyprocess 0.6.0 py37_0
pycparser 2.19 py37_0
pygments 2.2.0 py37_0
pyopenssl 18.0.0 py37_0
pyparsing 2.2.2 py37_0
pyqt 5.9.2 py37h655552a_2
pysocks 1.6.8 py37_0
python 3.7.0 hc167b69_0
python-dateutil 2.7.3 py37_0
pytorch-nightly-cpu 1.0.0.dev20181014 py3.7_0 pytorch
pytz 2018.5 py37_0
pyzmq 17.1.2 py37h1de35cc_0
qt 5.9.6 h45cd832_2
qtconsole 4.4.1 py37_0
readline 7.0 h1de35cc_5
regex 2018.07.11 py37h1de35cc_0
requests 2.19.1 py37_0
scipy 1.1.0 py37h28f7352_1
send2trash 1.5.0 py37_0
setuptools 40.4.3 py37_0
simplegeneric 0.8.1 py37_2
sip 4.19.8 py37h0a44026_0
six 1.11.0 py37_1
spacy 2.0.12 py37h6440ff4_0
sqlite 3.25.2 ha441bb4_0
termcolor 1.1.0 py37_1
terminado 0.8.1 py37_1
testpath 0.4.2 py37_0
thinc 6.10.3 py37h6440ff4_0
tk 8.6.8 ha441bb4_0
toolz 0.9.0 py37_0
torchvision-nightly-cpu 0.2.1 pyh19dea27_0 fastai
tornado 5.1.1 py37h1de35cc_0
tqdm 4.26.0 py37h28b3542_0
traitlets 4.3.2 py37_0
typing 3.6.4 py37_0
ujson 1.35 py37h1de35cc_0
urllib3 1.23 py37_0
wcwidth 0.1.7 py37_0
webencodings 0.5.1 py37_1
wheel 0.32.1 py37_0
widgetsnbextension 3.4.2 py37_0
wrapt 1.10.11 py37h1de35cc_2
xz 5.2.4 h1de35cc_4
zeromq 4.2.5 h0a44026_1
zlib 1.2.11 hf3cbc9b_2
[2]from fastai import * # Quick access to most common functionality
from fastai.tabular import * # Quick access to tabular functionality
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-1-eb17ac0e42e6> in <module>()
----> 1 from fastai import * # Quick access to most common functionality
2 from fastai.tabular import * # Quick access to tabular functionality
ModuleNotFoundError: No module named 'fastai'
If I do exactly the same thing, in the jupyter notebook instead of lab, it works.
I am in a mac, with CPU versions of torch.
What other info I need to provide?
Thanks for the extra info. apply_tfms isn’t defined in that module. Generally it’s easiest to simply from fastai import * and from fastai.vision import * (assuming you are doing a vision application) then you’ll have everything you need. You can then type apply_tfms and jupyter will return <function fastai.vision.image.apply_tfms(... so you can see where it came from. You’ll also find it in the vision.image section of the docs:
Thanks for your quick and effective reply.
But allow me one more question: If my personal box shows the expected behaviour, why did the DL box at work behave differently?
Thanks!
Yes I was reading it right now, it is a pleasure to read such a well-written documentation. Job well done!