Yup the pickling error on windows is going to be tricky to fix. Might take a few weeks…
2 Likes
Q: Peter Piper pip-ed a peck of pickled pthreads.
A peck of pickled pthreads Peter Piper pip-ed.
If Peter Piper pip-ed a peck of pickled pthreads,
Where the heck is the pickled pets that Peter Piper python-ed?
A: Peter Pipers, please pause two weeks to pip a peck of pickled pthreads.
4 Likes
I’m able to get the lesson-1-pets network properly trained on Windows, with the latest fastai.
Thanks for the quick fix.
1 Like
Hello, I tried to update fastai from conda today, but I’m having issues with pytorch-1.0 linked torchvision. Seems as though there’s no torchvision-nightly in ‘-c fastai’ and the torchvision package there is linked against pytorch 0.4.1. Are you guys seeing the same or have I somehow screwed up my installation.
I’m on arch linux host, anaconda53 installed.
Kind regards,
Ville
Found this one, probably explains the situation better then my explanation above. And the ‘torchvision’ there pulls in pytorch 0.4.1 if I try to install is.
You didn’t get the pickle error?
PicklingError: Can’t pickle <function crop_pad at 0x00000283DBF51840>: it’s not the same object as fastai.vision.transform.crop_pad
I’m still getting that error, so can’t run the notebooks locally on Windows 10, although PyTorch and fastai newest versions installed.
I’m getting past that error with Python 3.7 and FastAI 1.0.37.
This is the steps I’m using for my Conda environment. Windows 10.
conda create -n fastai-3.7 python=3.7
conda activate fastai-3.7
conda install pytorch torchvision -c pytorch
conda install -c fastai fastai
conda install nb_conda_kernels
python -m ipykernel install --user --name fastai-3.7 --display-name “Python (fastai-3.7)”
conda install ipywidgets
5 Likes
Worked for me – Thank you!!!
Major breakthrough for Windows. Big thanks. Your commands have gotten me passed the pickle problem. Pets is in the process of running to end. So far so good.
Do you know what fixed the pickle issue?
I don’t @bsalita. I know it was related to multi-threading. Not sure if it was properly fixed or multi-threading avoided for now. It does feel on my Windows machine that the GPU stuff flies - but it is very slow in the CPU parts - but subjective - I keep meaning to go back to my Linux VM and do a proper comparison.
Thanks again for the fix, really appreciated 
I’ve got the exact same feeling. There’s these “pauses” while it is grabbing batches it feels like, that don’t happen on my ubuntu install. Could just be placebo though…
1 Like
Have you updated pytorch / fastai yet?
Any issues?
I haven’t recently. I think 1.0.37 is what I have currently on my home machine.
I upgraded to 1.0.39 and haven’t seen any issues (yet) - I did notice that more of the utilities are in FastAI.vision - so the notebooks have been updated to remove import FastAI.
I did some comparative timings too - Ubuntu v Windows (Azure NC6 v Win10 and GTX1080ti)
Lesson 1 Pets, the first 4 epochs took 3m 36s on Ubuntu - and 5m 26s on Windows. The pauses were also very obvious in that the GPU processing was probably 3 times faster on Windows - but long gaps both between epochs and even between train/verify. I assume it is loading the batches. I did see a thread that suggested a faster library for image handling for Windows - but I think the real answer is the multi-threading and the picking issue.
1 Like
“GPU 3 x faster on windows”
What now?
1 Like
To be clear this isn’t a direct comparison - just that the apparent GPU work happens 3x faster on my local Windows 1080ti than the same work in the Azure NC6.
Ahhh, ok, makes sense.
I have both on my machine, but can’t really compro them because the Ubuntu install is on NVME and the windows install is on SATA so that muddies the waters.
NVME vs SATA is insignificant compared to the speed difference between Windows and Ubuntu. Also, you could mount the NVME in Ubuntu and access the Windows version of the data.