confirmed. when I moved the 50 000 train and 10 000 test files into nested subdirectories, dataloader worked. I don’t know why this is ‘new’
Excuse if the below code is awful python. I don’t like looping individual files. (note I haven’t put in code to delete the original files)
#note plane would work for airplane, car wouldnt for automobile
classes = ('airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
cd ./data/cifar10/train
#cd ./data/cifar10/test
for c in classes:
%mkdir $c
import fnmatch
allpng=fnmatch.filter(os.listdir(), '*.png')
for c in classes:
c_list = fnmatch.filter(allpng, '*'+c+'*')
for fn in c_list:
shutil.copy( fn, c)
sounds like creating a csv for this download would be the way to go then…
Same, I’m revisiting DL after finishing the ML stuff. I remember Jeremy saying he spends more time fitting datasets to what fastai wants, rather than writing more fastai dataloaders. I just found this odd, using the recommended download from within lesson 7
The information in @ecdrid’s reply above is incorrect. Please don’t use that approach to setting up Windows, but use the approach in the top post in this current thread instead.
We are using the pre-release version of Pytorch for Windows. It runs under Windows directly, not under the Linux subsystem.
I have GTX 1080ti on my windows box. I followed these instructions and I see that my Jupyter notebook runs slow. Is this common for Jupyter Notebook to run slow?
In fact, the learning is also slower.
Here’s a comparison (running first model from lesson1.ipynb)
Ubuntu Unix Box
RAM 32 GB
GPU GTX 970
Drive SSD
Example: Our first model (dogs vs cats) with no temporary files
Windows Box:
RAM 64GB
GPU GTX 1080ti
Drive SSD
I have two GPUs
Therefore, I set device as follows:
When I run the same piece of code as above, it not only runs slowly but gives warning for the other GPU (device 1).
Just wanted to share my experience with the install for anyone who has issues. I’m a huge novice to the command line and programming so these tips are targeted at that audience.
I followed Jeremy’s instructions but received a module not found error when inside of the lesson 1 running this: from fastai.imports import *
I believe I had two issues. The first was I had previously installed Anaconda but I had it installed for all users rather than just me like Jeremy had specified. I uninstalled my anaconda and reinstalled for just me.
My second issue was creating a symlink in the command line for step 8 of the install. (which was the first time I’ve ever done that so needed a bit more verbose instructions). What isn’t clear for noobs but I’m sure is totally obvious to people with command line experience is once you load the anaconda command line prompt in administrator mode, it loads you into an entirely different directory than the home directory you were working in for steps 1-7. Therefore you need to navigate to the home directory you were in for the previous steps using the change directory function “cd”. For me, I needed to navigate back to: C:\Users\Will\fastai\courses\dl1 but you’ll need to replace "Will’ with your home directory name. Once at this directory, which you can check by typing ‘pwd’ you can finish the install by following the instructions in step 8.
One final thing to note is that the instructions work as long as you are in the dl1 course. If you try to run a notebook from the ml1 course, you’ll need to change your directory within the jupyter notebook to courses/dl1 instead of courses/ml1. I’m sure there is a better way to do this but this is what got it to work for me.
Anyway, I’m sure this is obvious to most but hopefully helpful to the coding newbs in the group.
I am new to the course so my question might be silly. Is it impossible to implement fastai on CPU with AMD Radeon or it just very slow to work?
Also when I tried with the same I am getting an error for line: from fastai.imports import *
I thought there was a problem with directory path so tried
@jeremy - I am sorry to tag you in this post but I have not been able to find a solution to this yet.
Have you observed that CUDA 9 is slower tha CUDA 8? My windows box (GPU 1080i) is slower than my Ubuntu box (GTX 970) in that it takes twice as long to run the same code.
@vikasbahirwani How do the CPUs compare on these two boxes? Training seems to be heavily CPU bound, to the point that I and many others have initially thought it wasn’t using our GPUs at all.
A couple other things to verify would be disk I/O and RAM read/write rates. M.2 drives can be much faster than SATA SSDs.
@jeremy I tried for almost 3 hours and searched the forum for almost an hour. I wasn’t able to find anything other than this thread.
I can’t financially support for the cloud service.
I have a 2GB radeon enabled graphics processor.
I’ve been trying to do the above mentioned steps but have failed every time and these errors
Great videos. Can’t express it enough how easy you make this. Learning Deep learning from scratch and moving right along. Your setup for my windows install worked flawlessly.
Can’t wait to make it all the way through your series.
or do i need to go through all of the installation steps again? I’m afraid to experiment with this on my own as i have everything working and lack the skill to repair it if I break something.