No problem. Daniel it seems like the notebook backed instance never ran. Thanks!
I am trying to ssh into my paperspace gradient machine. But when I enter username and password on my local machine terminal, I get ‘Permission Denied’ error.
I need to download the model file to local PC so that I can upload it to my Google Drive for deploying on Web.
Please help.
For those interested, we updated the Fast.ai VM template with the latest course material. You can access it on the Public Templates tab on the machine provisioning page:
I’m working on getting a script together so you can update an existing machine (instead of creating a new one).
@MadeUpMasters @adric
1 Like
I created a new machine, pulled from git and updated fastai library.
I’m trying to reproduce lesson5 in a jupyter notebook. I’m on this line:
learn = Learner(data, Mnist_NN(), loss_func=loss_func, metrics=accuracy)
And I get the same error as this guy did:
Something to do with the loss_func not recognizing the nn.CrossEntropyLoss() function
Which GPU notebook should I choose?
Is the K80 preemptible a good choice?
With resnet50 I still get 20 min learning times where I see our tutor has 4mins. Sadly slows things down quite a bit.
@dkobran Can we get auto-shutdown options of less than 8 hours? It seems trivial to add and most of us are using this for working with the fast.ai coursework which generally is a few hours of work.
Also is there a reason you don’t offer auto-shutdown for gradient-0 customers? There could be a proper justification but it comes off like you’re basing your business model around people forgetting to shutdown their instances…
1 Like
I was just going to post with the same issue. Also using K80 preemptible but it is running really slowly. I am going through the Planets example and taking me over 20 minutes to fit the model instead of the 4 minutes for the tutor. Is Jeremy using a very fast GPU or is the K80 a poor option? I had read good things about it.
I noticed that the default gradient notebook v3 is currently using ‘1.0.42’ version of fastai. Is there a way to update it? I did a git pull in the course-v3 directory but that doesn’t update the fastai library, correct? I notice that cnn_learner() isn’t part of this, so you have to use create_cnn() instead. So the pets notebook doesn’t work out of the box, for instance.
# git remote show origin
* remote origin
Fetch URL: https://github.com/fastai/course-v3.git
Push URL: https://github.com/fastai/course-v3.git
HEAD branch: master
Remote branches:
ImageDeleter tracked
gpu tracked
master tracked
revert-24-master tracked
revert-66-patch-1 tracked
sagemaker-ebs-volume tracked
Local branch configured for 'git pull':
master merges with remote master
Local ref configured for 'git push':
master pushes to master (up to date)
#
@LeonW @malanon According to Google, the K80 preemptible and regular on-demand K80 do not have different performance characteristics. They are are identical with the difference being that you are not guaranteed access to the preemptible type (it can be preempted anytime) and they only run for up to 24 hours at a time.
It could be something related to the GPU itself since the K80 is now several years old and not nearly as powerful as newer chips. I would suggest trying the P100 (also offered as on-demand or preemptible) and see if there’s an improvement. Alternatively, the P4000 and P5000 are also more recent GPUs.
Hope that helps.
@ab90 I don’t know the safest way to update these libraries since we just build a fresh template each time ie we don’t update ours per se. I can def inquire about replacing our tile with a newer version. In any case, if you find out how to update fastai, please do share here as that’s great info to have.
You can just open a terminal and run:
# conda install -c fastai fastai
This updated the library for me. Now I’m at 1.0.49 up from 1.0.42.
import fastai; fastai.__version__
'1.0.49'
Hope this helps!
Cheers,
Adam
1 Like
Hey I just want to leave a note here to anyone else left scratching their head by paperspace/gradient.
I was trying to run the imdb lesson three notebook. However the default /notebooks directory in paperspace was so unbelievably slow that it wasn’t possible to untar the imdb.tgz file in any sort of reasonable time. I believe the fast.ai library threw an exception and caught it silently or something, because the end result was that the “imdb” folder didn’t have its training data. I had to go into the terminal, where I tried to manually untar the file. Even piping output to /dev/null, the process was taking FOREVER.
Solution: I untared the file to /tmp/imdb and created a symlink to that directory. I was able to get the project running again after this, but overall this was very annoying to investigate. I am using a K80 preemptive instance. I would highly advise paperspace team to look into improving storage performance…
thank you! performance on the P100 is much better. seems to cost less than the K80 overall given its efficiency.
1 Like
I saw something similar and pinged paperspace support. You might try refreshing the browser. It then (for me) shows as ‘Stopped’ after a minute or two.
@ab90 can you (or someone else) expand on this? In what folder do you run the conda install command? I’m confused by the folder structure in paperspace/fastai. I tried to install the tree command but was unable to do so.
Thanks,
David
EDIT (SOLVED):
In python notebook I went to the Conda tab and saw this:
So I went to /opt/conda/envs/fastai in a new terminal window
From within that folder I ran:
conda install -c fastai fastai
After the installation I launched python with the python3 command. I was then able to run
import fastai; fastai.__version__
and the version which had been 1.0.42 is now showing as 1.0.50.
1 Like
I’m experiencing a weird (but persistent and replicable) bug on paperspace gradient. I had a notebook completely disappear from my /course-v1/nbs/dl1 folder where all the lesson notebooks are stored, costing me a full day’s work. With some experimentation I’ve found that it can be replicated, and only happens for notebooks that come alphabetically after the final lesson notebook “lesson7-wgan.ipynb”.
So, for instance, if I make a new notebook named “reorg.ipynb” or “loss.ipynb” and save it, it will disappear without a trace 100% of the time after I shutdown my instance. If I make one that comes before “lesson” alphabetically like “laws.ipynb” or “apples.ipynb” it will never disappear.
Seems like a really horrible bug. It is happening on a C7 instance and doesn’t seem to be affecting my other machines. I’ve had really bad experiences lately with paperspace. They don’t seem to ever reply here either.
Hi all
I Want to start part 2 of 2018 (I did part 1 of 2019).
Im using paperspace as my GPU.
how do I download the course content to paperspace? when I did course 1 (2019) it was automatically there in dl1 folder.
thank you!
The instructions for updating fastai on Gradient are here
https://course.fast.ai/start_gradient.html#step-3--update-the-fastai-library
1 Like
Hello
thank you for the answer. i continued the process and no I am not able do download data to paperspace.
what am I doing wrong? I am stuck with this for a long time!!
Offir
Hi. Could you please share the steps you followed to solve this? Thanks!