Howto: installation on Windows

[muqsit25 (Abdul Muqsit Abbasi)]

I have been trying to find a way around the gpu. I’ll train my networks on another system. Is it possible to follow these tutorials without GPU?

[sam2 (Sam)]

@ecdrid,

It does use GPU…just that the activity seen at GPU-Z suggests that the GPU is starved of data. I think I should play a little with parameters like num_workers in the ImageClassifierData. I also think that a SSD would work better i/o a HDD.
Currently what I see is that GPU load shows a pulse with 90% load and is silent for a few seconds and again gets a spike and so on.

[Arghz (Dominique Gilleman)]

I had the same error.

The only way so solve it, i’ve re-created an Anaconda environment with Pytorch 3.1 and fastai as:
http://www.deeplearningessentials.science/installation/

Now it’s fine.

[nok]

Same error, no luck, looking for solution. I think the previous version was ok as I successfully built before, but since 1/2 weeks ago I rebuild it on other computer it fails.

[jeremy (Jeremy Howard)]

FYI I’ve done nearly all development of part 2 so far on my windows laptop (surface book 15 inch) and haven’t had a single issue so far. So Windows support in fastai/pytorch is looking really solid!

[Kasianenko (Igor Kasianenko)]

surface book 15 inch

Hi, do you use 16 GB laptop?

I run into MemoryError in lesson 2, with my 16GB laptop, and this post workaround solved this problem.

Memory Error (Not cuda out of memory error)

There is some kind of memory leak caused by using ThreadPoolExecutor in DataLoader (fastai/dataloader.py). A temporary fix is to disable multi-threaded execution altogether. For example the following diff will fix the issue but make loading data slower: def iter(self): - with ThreadPoolExecutor(max_workers=self.num_workers) as e: - for batch in e.map(self.get_batch, iter(self.batch_sampler)): - yield get_tensor(batch, self.pin_memory) + for batch…

[hafidz (Hafidz Zulkifli)]

Words can’t describe how glad I am to hear that

[gigaom (Om)]

No, Abhishek. I tried all way and did not get much help. I am using paperspace until I have working setup.

[nileshgarg]

NameError Traceback (most recent call last)
in ()
----> 1 from fastai.transforms import *
2 from fastai.conv_learner import *
3 from fastai.model import *
4 from fastai.dataset import *
5 from fastai.sgdr import *

\fastai\courses\dl1\fastai\transforms.py in ()
38 return np.clip((im-mu)*c+mu+b,0.,1.).astype(np.float32)
39
—> 40 def rotate_cv(im, deg, mode=cv2.BORDER_CONSTANT):
41 “”" Rotates an image by deg degrees
42

NameError: name ‘cv2’ is not defined.

I am getting this error when trying to run lesson1

I am using windows10 and zotac nvidia gtx 1060 card.

I tried with conda install -y opencv but no luck.

[ecdrid]

A quick google search will help as to how to set up opencv/cv2 on windows

[nileshgarg]

I tried several ways, still getting error. When I run conda list it shows opencv there. Any specific link you can refer.

[aravind]

Hello Windows users,

The latest version of Pytorch - 0.3.1, doesn’t offer support for GPU with CUDA capability less or than equal to 5.0 and this is the only version available for Windows from my understanding.

If anyone is open for dual boot with Ubuntu, then can follow this setup script with following additional steps.

1. After cloning the git, edit environment.yml in the fastai folder from pytorch >= 0.2.0 to pytorch <= 0.3.0

2. Instead of conda env update, use conda env create - not sure whether this is absolutely necessary but just to be safe I did this.

I was able to follow lesson1 without any trouble in my laptop. Good luck!

[rkd2000 (Keith Douglas)]

Failing at step 5: “conda env update”

"# >>>> ERROR REPORT <<<<
…
yaml.constructor.ConstructorError: could not determine a constructor for the tag ‘tag:yaml.org,2002:python/unicode’
in “C:\Users\antip\AppData\Local\ContinuumIO\binstar\config.yaml”, line 1, column 1
"

This thread suggests it’s a problem with the yaml creation?

Issue: yaml.constructor.ConstructorError: could not determine a constructor for the tag 'tag:yaml.org,2002:python/unicode'

Fresh install of Anaconda.

Suggestions?

[rkd2000 (Keith Douglas)]

Ok, nevermind.

conda update -all
cd fastai
conda env update

This works.

[Brad_S]

has the zip file below changed? did it use to have sub directories for the categories?

wget http://pjreddie.com/media/files/cifar.tgz

dl1/lesson7-cifar10.ipynb is throwing me an error in fastai\dataset.py line 48
for fname in os.listdir(os.path.join(full_path, label)):

NotADirectoryError: [WinError 267] The directory name is invalid: ‘data/cifar10/train\0_frog.png’

not sure what the listdir() is trying to do after join adds the filename to the path?
(I have done a git pull, maybe there’s some work going on here?)

I deleted the os.listdir call and this step ran, then errored on the next step
x,y=next(iter(data.trn_dl))
again errrored in dataset.py this time line 199

OSError: No such file or directory: data/cifar10/train\16187_horse.png\t

[ecdrid]

The problem is \ or /…
It specific to Windows…
(Have a look at the directory it’s telling, it’s definately not a directory)

[Brad_S]

The problem is \ or /…

I don’t think so. For that reason I put it in the Windows thread, (but I was too tired to think properly, then I edited my post, but didn’t communicate well enough)…
NotADirectoryError: [WinError 267] The directory name is invalid: 'data\\cifar10\\train\\0_frog.png'

now clearly thats not a directory, but it doesn’t look like a windows problem. I now presume fastai is expecting a directory structure like train\cats*.img and train\dogs*.img (here planes and frogs etc)

‘data\cifar10\train\frog\0.png’

but thats different to what I got by downloading the suggested cifar.tgz . I haven’t written something to move the files around yet, which will demonstrate if thats right, but I figured other people would have had this same download (recently?)

[ecdrid]

fast.ai does expects all images in different folders according to the classes or mapped by a CSV? Or out the directory name in f string in python within quotes
Pardon me haven’t used DL side of fast.ai for past 2-3 weeks…(so I may be wrong)

[Brad_S]

confirmed. when I moved the 50 000 train and 10 000 test files into nested subdirectories, dataloader worked. I don’t know why this is ‘new’

Excuse if the below code is awful python. I don’t like looping individual files. (note I haven’t put in code to delete the original files)

#note plane would work for airplane, car wouldnt for automobile

classes = ('airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
cd ./data/cifar10/train
#cd ./data/cifar10/test

for c in classes:
    %mkdir $c
    
import fnmatch
allpng=fnmatch.filter(os.listdir(), '*.png')
for c in classes:
    c_list = fnmatch.filter(allpng, '*'+c+'*')
    for fn in c_list:
        shutil.copy( fn, c)

And now I see a 2nd thread on this same thing… Not a Directory error in CIFAR10 exercise

[Brad_S]

sounds like creating a csv for this download would be the way to go then…

fast.ai does expects all images in different folders according to the classes or mapped by a CSV? Or out the directory name in f string in python within quotes

Same, I’m revisiting DL after finishing the ML stuff. I remember Jeremy saying he spends more time fitting datasets to what fastai wants, rather than writing more fastai dataloaders. I just found this odd, using the recommended download from within lesson 7