I just got the new paperspace fast.ai template.
When i try to run the first notebook, i get an error saying the computer cannot locate bcolz module while trying to import utils.py.
I have tried both pip and conda installs which seemed to work, but the issue was not solved after running the code again.
Is there a solution for this?
I just got the new paperspace fast.ai template.
I had the same issue and I thought it just needed to be installed and I was planning on doing it, but I got sidetracked with a “work work” project, so I’m a little troubled to hear this. I know paperspace folks hang out at this forum (for example Daniel @dkobran from paperspace was super helpful in another thread re: paperspace.com) , so hopefully one of them will respond. Maybe try creating a helpdesk ticket and someone might look into it faster? It may very well be that their fast.ai template needs a tweak or two.
@crawftv Were you able to get this resolved? I was able to do a conda install but that is python3 and the notebook runs in a python2 kernel. I’m not sure how to install bcolz for python2 because a simple pip2 install fails due to insufficient privilege error and I can’t seem to be able to use sudo because I don’t have the password for the dummy paperspace account they run everything under.
It seems they took their ML template, copied the fast.ai stuff onto the desktop and called it a day. Don’t think the notebooks actually work in that template and that is kind of annoying because making a template available implies that it works out of the box. In fact, that would be the whole point of going through the trouble of making a template to begin with.
I tried installing it every way i could think of but nothing worked. Paperspace showed me these help links, but i haven’t been able to access them.
Time for me to invest in a half decent gpu I think
I managed to overcome this, here’s what I found:
- bcolz isn’t being imported due to some obscure jupyter kernel error. Fixed by running this command (with the appropriate conda env. already activated):
jupyter kernelspec install-self --user
apparently this command is deprecated, but it woked for me…
- theano will give you import errors due to an environment variable not set. You need to permanently set the environment variable like so:
sudo nano /etc/environment(add line: MKL_THREADING_LAYER=GNU) save file, then
sudo reboot now
Here’s the step-by-step process that got Lesson1 running on my paperspace:
Creating the basic paperspace account:
- Create basic fast.ai ubuntu image + static ip option
- Delete data folder from desktop (to free up necessary space)
- change default password using
1 - set jupyter pw:
jupyter notebook password
2 - Set inbound ip to ‘*’ and allow password certification in options (set to True):
sudo nano ~/.jupyter/jupyter_notebook_config.py
3 - Open firewall for jupyter:
sudo ufw allow 8888
4 - Set environment variable (for theano to import properly)
sudo nano /etc/environment (add line: MKL_THREADING_LAYER=GNU)
5 - reboot for environment variable to take effect (
sudo reboot now)
6 - Install dependencies (see below)
conda create -n fastai python=2.7 pip source activate fastai conda install matplotlib pillow tensorflow pandas scikit-learn bcolz sympy h5py conda install -c mila-udem -c mila-udem/label/pre theano pygpu conda install -c conda-forge blas pip install keras==1.2.2 conda install jupyter jupyter kernelspec install-self --user source deactivate source activate fastai jupyter notebook
To log into your notebook, point your browser (using YOUR computer, not paperspace) to:
Then log in using the jupyter password you set up in step 1.
Good luck, let me know if this works for you.
Also, if you don’t have a paperspace account yet, click this link for $10 of free credit (I get free credit too):
Hey Eric, these instructions are brilliant! it worked for me perfectly. I’m running a full batch right now. I just provisioned a P5000 machine so my ssh and 8888 ports are still not accessible from the local machine. BTW, which exact lines did you change in the jupyter_notebook_config.py file?
BTW, I noticed that in my fastai env, the default backend is tensorflow even though I installed keras 1.2.2 … also using a batch size of 64 I’m getting an ETA of about 16000 seconds (5 hours or so) which doesn’t seem right given that this is a p5000 machine LOL … I’m going to try with a smaller batch size (just FYI for anyone using p5000 machines with paperspace)
Mike! I just ran into that problem.
To get keras to use the theano backend:
There’s an option there for which backend to use - change it to “theano”
Strange that you can’t access the machine via ssh. Do you have a static ip? You’ll need that for SSH access. If you can SSH but cannot access the jupyter notebook, then it’s likely a jupyter configuration issue.
Can’t remember the exact changes (and don’t have the paperspace running atm) but I do remember it was two changes:
- changing the inbound ip from “” to “*”
- changing the password certification from “False” to “True”
Last thing (a bit of a dummy check, really), don’t forget to actually run
jupyter notebook to start the notebook server
haha! no I defonitly ran the notebook! that’s how I got the 17000s ETA which freaked me out tbh! Yeah that was my next item to check (keras.json) but I assumed it uses theano as default especially the fast.ai template but I’ve been playing with so many images, docker containers, local installs, jupyter in the cloud that my head is spinning right now
I opened an issue with paperspace because both my ssh and 8888 ports claim to be open but I can’t ssh to 22 and can’t get to jupyter:8888 … too late in the day maybe. I’ll follow that setup guide and report back how i made up.
Thanks for the instructions, I hope paperspace updates their template with this because this actually made the machine usable for me (I had given up on paperspace for the time being until I used your instructions to install everything properly)
Hi Eric, wonderful set of instructions.
Thank you for this workaround. It was brilliant solution.
Everything worked well except. Got error while loading Vgg16.
Something similar to
Following steps took care of issue as explained in above mentioned thread
pip uninstall keras
pip install keras==1.1.2 (instead of 1.2.2)