I’m getting the same thing.
Ok found the problem. Needed to update fastprogress, as the way it’s being interacted with in the fast.ai code has changed. @MicPie
Weird, how did you update your library? Normally conda would have updated fastprogress for you.
I used pip to upgrade fastprogress directly.
I’m embarrassed to say, I don’t know how to resolve an impasse I currently have between jupyter_contrib_nbextensions and pytorch. Until I can get that sorted out,
conda env update isn’t working.
I am using the developer install and I could fix it with
conda install -c fastai fastprogress to get the newest fastprogress version. Thanks for the fast help.
Not if you do
git pull. I have experienced this problem several times.
I think the best solution is to assert for a specific fasprogress version in the fastai code (and not just package dependencies). That way regardless of how people get their fastai updates, they will know right away that they need a newer version of fastprogress.
It doesn’t look this is common in python to perform that at the code level. Or at least I couldn’t find almost any references on how to do it, or even people asking about how to do it.
I found this suggestion:
import pkg_resources pkg_resources.require("fastprogress>=0.1.18") import fastprogress
And if the version is insufficient, you’d get:
python -c 'import pkg_resources; pkg_resources.require("fastprogress>=0.1.19"); import fastprogress' Traceback (most recent call last): File "<string>", line 1, in <module> File "/home/stas/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 898, in require needed = self.resolve(parse_requirements(requirements)) File "/home/stas/.local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 789, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.VersionConflict: (fastprogress 0.1.18 (/home/stas/anaconda3/envs/pytorch-dev/lib/python3.6/site-packages), Requirement.parse('fastprogress>=0.1.19'))
environment-cpu.yml belong to the old fastai (0.7).
conda env update is no longer the way to update your fastai-1.x environment. This is unfortunate, but we can’t remove these because the course-v2 video instructions rely on this setup. Eventually, once course-v3 p1 and p2 will be completed, they will probably be moved to where they belong - under
instead do a normal install (even if you’re updating a previous install):
conda install -c fastai fastai or
pip install fastai and everything that needs to be updated will be updated.
Thanks for the help, I’ll try this out in the next few days.
To give some detail about what brought me to this point, I followed the recommendations of the MOOC and got a paperspace machine with fast.ai preinstalled, and have been
git pulling since with monthly-ish
conda env update. (I eventually ran into the problem with nbextensions but really didn’t want to give it up!)
Somewhere in the middle of working my way through the MOOC I realized the lessons were being done with an old version of fast.ai, and every time I was
git pulling, none of the updates were hitting what I was using. Eventually I decided to try out 1.0.
I’m figuring it out, but it wasn’t real clear to me at first how to make the transition.
It can be very confusing indeed, especially given that fastai-0.7 files aren’t all in the same place, due to historic reasons I explained above. If you’re following the course-v2 MOOC you better stick to fastai-0.7, since the API has dramatically changed and it’ll be very difficult to make sense of the course-v2 videos using fastai-1.x. And if you do then, yes, you still need to use
conda env update exactly as in the past. But you won’t need fastprogress then.
It wasn’t clear from your original post that you were referring to the fastai-0.7 version, but we do know that now. And if you do have installation issues with fastai-0.7 - please post those here.
I was curious to see if i could get to generate a
train_ds w/ labels using the new API, but it appears to be very rough at the edges and failing in many ways if the prescribed way of train/valid split is not followed to the point, e.g.:
train_ds = ImageItemList.from_folder(path).random_split_by_pct(0) ... IndexError: index 0 is out of bounds for axis 0 with size 0
and it also has an edge problem of generating 1 item in valid ds, which shouldn’t be there, since train ds, already has the full number of available items.
and this fails too:
train_ds = ImageItemList.from_folder(path) train_ds train_ds = train_ds.label_from_folder() train_ds ... TypeError: 'NoneType' object is not subscriptable
Is this a work-in-progress, or is the new API set to not allow any non-fastai-way of creating _ds objects (i.e. w/o valid set together with train_ds)?
I was experimenting with fashion mnist, but mnist will do too if you’d like to try it (i.e. path = /path/to/mnist)
I was planning to feed
ImageDataBunch.create(train_ds...) as in
nbs/dl1/lesson7-wgan.ipynb, but the example there has no labels and w/o labels it works.
p.s. the 2 ways of using the test dataset w/ labels to validate against are now documented here.
Neither. It simply means you found a bug.
It looks like v1.0 has been released https://github.com/pytorch/pytorch/releases
FYI, @sgugger implemented this - so future
git pull updates will be fastprogress-progress aware.
FYI, a hotfix fastai-1.0.36.post1 pypi release was made (a fix in regex dependencies which was conflicting with spacy)
And it looks like
pip behaves strangely with
.postX releases. When I first run:
pip install fastai
it won’t install
fastai-1.0.36.post1 (installed fastai-1.0.34 instead, due to conflicts in 1.0.36 I suppose). But when I did:
pip uninstall fastai -y pip install fastai
it did install
fastai-1.0.36.post1. That’s a very inconsistent behavior.
I think that I found a mistake in the simplest CV example. Normalisation was done with imagenet_stats instead of the actual dataset stats (in our case - mnist_stats). I’ve made a pull request which was rejected. Can somebody look at the PR and express the third opinion?
When using a network pretrained on imagenet then you must use image-netstat to normalise - always
On this, added a
no_split method that will create a training set with everything and a validation set with nothing. Note that the empty validation set will still have a length of 1 otherwise the pytorch dataloader will complain (you have to test
len(valid_ds.items) to check if it’s 0 or not.
random_split_by_pct so that a 0 falls back to
no_split, but this should be the method used.