Introducing fastai-shell

cwerner · November 10, 2018, 8:16pm

I managed to get an instance running (P4) in us-west2-c. however, I cannot connect using the given IP. Do I need to change stuff in GCP settings?

nikhil_no_1 · November 12, 2018, 4:14pm

Thanks. This is very useful.
I followed all the steps but I am not able to access http://my-external-ip:8888. Even after running a new instance of jupyter notebook on port 8889, I cannot access it until I change the NoteBookApp.IP in jupyter_notebook_config.py from ‘localhost’ to ‘0.0.0.0’.

In lsof output I can see even port 8888 is bound on localhost.

(fastai-v1) nutane@fastai:~$ lsof |grep 8888
jupyter-n 1625 nutane 6u IPv4 22390 0t0 TCP localhost:8888 (LISTEN)

Am I doing anything wrong? BTW, why doesn’t this instance of jupyter notebook list under jupyter notebook list command?

arunoda · November 13, 2018, 12:11am

Did you try port: 8080 ?
In the latest script I changed it. Anyway, you’ll see the correct URL after it’s started in the CLI.

nikhil_no_1 · November 13, 2018, 4:17am

Port number was not the problem. Bound IP address was. I did not use the fastai shell. Rather I used the gcloud compute command as I was getting some error for fastai-shell.
For GCP instance, does it make sense to bind to ‘localhost’ in default jupyter config? I’d think 0.0.0.0 would be better option.

arunoda · November 13, 2018, 11:34am

If you bind to localhost you can’t access it from outside. Use 0.0.0.0.

nikhil_no_1 · November 13, 2018, 11:43am

Yup, that’s exactly my point. The default jupyter config should also have 0.0.0.0. Anyways, I am up and running with fastai-shell. Earlier in the day the commands where throwing weird errors because of which I tried manual options (which gave me quite a bit of pain). Good job. Thanks.

cwerner · November 13, 2018, 12:03pm

Hi again.

I finally managed to spinup an instance again. But now I do get a 403 error when I try to connect…

http://35.230.109.121:8080

Is there a way to check what is going on? Do I need to do some network setup in my project settings in GCP?

Could that be a conda thing on the instance ?

Thanks,
C

arunoda · November 13, 2018, 12:39pm

That’s pretty interesting. I’ll have a look.

cwerner · November 13, 2018, 12:43pm

thanks!

arunoda · November 13, 2018, 12:45pm

I had an issue with the official image.
Got the same issue.

So, I went back to custom image.
Could you try to update fastai-shell and try again.
See: https://github.com/arunoda/fastai-shell#stay-up-to-date

cwerner · November 13, 2018, 12:49pm

I just updated to 0.1 before tried it… Do you know if I should check project settings in GCP for network issues?

Tried it again: same result… 403 forbidden

arunoda · November 13, 2018, 12:55pm

I just updated to 0.1 before tried it

That’s interesting. Let’s me try it again.

arunoda · November 13, 2018, 12:59pm

One thing: You’ve to run fastai destroy and fastai create again.
You have to upload any crucial data before doing this.

Very sorry for this.
But I couldn’t rely on the official image since it’s getting changed and I have no control over that.
So starting scratch from the base ubuntu is method which will work for sure.

cwerner · November 13, 2018, 1:53pm

Ok, did that and finally have a connection to GCP! Man, I was getting desperate

Rares · November 13, 2018, 9:23pm

hello
i am looking to download some datasets straight to gcp, unzip them and use them in a fastai instance.
can you please suggest how to do this?
thank you very much

arunoda · November 13, 2018, 9:31pm

Once you start server. You can access your notebooks from https://ip:8080.
In that you create a new Terminal.
Then you can download them and unzip.

nikhil_no_1 · November 15, 2018, 3:02am

Hi @arunoda,

I am now using fastai-shell and it is working beautifully. Thanks again.
Some questions, does the image used here differ considerably from the official one?
I wanted to try using Jeremy’s .vimrc but ran into quite a bit of issues. The vim did not have the python support which I then compiled in but still getting other issues.

arunoda · November 15, 2018, 4:18am

It’s a different setup. But it should be the same.
Try following before you run any commands:

source activate fastai-v1

oguiza · November 15, 2018, 9:33am

I’ve been using fastai-shell and think it works great! Thanks a lot for putting it together @arunoda

There’s only one are where I struggle that is the nbextensions. I have used:

`conda install -c conda-forge jupyter_contrib_nbextensions`
`conda install -c conda-forge jupyter_nbextensions_configurator`

in the jupyter terminal, and have restarted the instance, but for some reason I cannot still see the jupyter_nbextensions_configurator tab.

I have checked the jupyter serverextension list:

oguiza@fastai-1:~$ jupyter serverextension list
config dir: /home/oguiza/.jupyter
    jupyter_nbextensions_configurator  enabled
    - Validating...
      jupyter_nbextensions_configurator 0.4.0 OK
    jupyter_notebook_gist  enabled
    - Validating...
      jupyter_notebook_gist 0.5.0 OK
config dir: /home/oguiza/anaconda3/etc/jupyter
    jupyterlab  enabled
    - Validating...
      jupyterlab 0.34.9 OK
    jupyter_nbextensions_configurator  enabled
    - Validating...
      jupyter_nbextensions_configurator 0.4.0 OK

and jupyter nbextension list:

    oguiza@fastai-1:~$ jupyter nbextension list
Known nbextensions:
  config dir: /home/oguiza/.jupyter/nbconfig
    common section
      jupyter-notebook-gist/common  enabled
      - Validating: problems found:
        - require?  X jupyter-notebook-gist/common
    notebook section
      nbextensions_configurator/config_menu/main  enabled
      - Validating: problems found:
        - require?  X nbextensions_configurator/config_menu/main
      contrib_nbextensions_help_item/main  enabled
      - Validating: OK
      jupyter-js-widgets/extension  enabled
      - Validating: OK
      jupyter-notebook-gist/notebook-extension  enabled
      - Validating: problems found:
        - require?  X jupyter-notebook-gist/notebook-extension
    tree section
      nbextensions_configurator/tree_tab/main  enabled
      - Validating: problems found:
        - require?  X nbextensions_configurator/tree_tab/main
      jupyter-notebook-gist/tree-extension  enabled
      - Validating: problems found:
        - require?  X jupyter-notebook-gist/tree-extension
  config dir: /home/oguiza/anaconda3/etc/jupyter/nbconfig
    notebook section
      jupyter-js-widgets/extension  enabled
      - Validating: OK
      nbextensions_configurator/config_menu/main  enabled
      - Validating: problems found:
        - require?  X nbextensions_configurator/config_menu/main
      contrib_nbextensions_help_item/main  enabled
      - Validating: OK
    tree section
      nbextensions_configurator/tree_tab/main  enabled
      - Validating: problems found:
        - require?  X nbextensions_configurator/tree_tab/main

I have also uninstalled and reinstalled it again, but unsuccessfully. Could anyone please help?

arunoda · November 15, 2018, 10:02am

When you run your terminal, activate the conda env like this:

source activate fastai-v1

I think I’m going to add this automatically.