Note that during these first few sessions we set up Paperspace from scratch. However we used this process to create a script to do it all automatically – it you’d rather use the script, here it is.
Where to find the right install command for pytorch? get-started-locally
What does this script mean mamba install pytorch torchvision torchaudio -c pytorch, in particular -c pytorch? 1:04:04
Will pip install mess up with my mamba kingdom? 1:05:06 yes, pip install does mess you up. So you should either pip uninstall or remove mambaforge folder and start install allover to be sure
How do we know pytorch is working? ipython and import torch and see
How to run jupyter lab without browser? jupyter lab --no-browser
How to find and check the content of .bash_history? cat .bash_history1:12:53
How to make the most out of .bash_history file?
How to search the .bash_history commands? ctrl + r and use delete tab to clear the search, use esc tab to exit search
How to run a command starting with ju? !ju
How to run the last command? !!
How to show the last command? echo !!
How to add alias in command line? alias jl="jupyter lab"1:15:37
How to move around the command line, start of a line, end of a line, move a word at a time? ctrl + e, ctrl + a, opt + left or right (to move a word at a time, esc + B and esc + F can do with my macpro)
How to write and save alias for good? 1:17:05 open .bash_rc and add the alias and save (on my macpro, I have to use .bash_profile)
How to edit files in terminal without vim? use open . to open a folder, and to edit a file with vscode
Radek mentioned some additional advantages of WSL
How to create a jupyter notebook in a folder from jupyter lab 1:21:28
How to install jupyter ipywidget? mamba install ipywidgets
Can we do gui stuff with jupyterlab, like click and delete?
Additional tricks without going too far
how to clear the screen of terminal? ctrl + l
How to clear a line of command in terminal? ctrl + u
How to clear a word of a line of command in terminal? ctrl + w
Why recommend not to use the latest version of python?
How to get back to your home directory? cd
57:05 Install other packages with mamba
What does mamba do for us about installing useful things? your own environment
How to install packages into python environment? 58:09
How to install python libraries into your python env? mamba, conda, pip
How conda and mamba differ? mamba is faster, but new so not all libs supported
Advice: only use conda, pip or mamba to install python libraries, most of time just use mamba
How to know more about mamba? mamba
How to install ipython with mamba? mamba install ipython
What does mamba ask and do for you? check which libs you need and ask for permission to install
How to close ipython or most programs on terminal? ctrl + d (or ctrl + z)
1:02:04 Install pytorch
Where to find the right install command for pytorch? get-started-locally
What does this script mean mamba install pytorch torchvision torchaudio -c pytorch, in particular -c pytorch? 1:04:04
Will pip install mess up with my mamba kingdom? 1:05:06 yes, pip install does mess you up. So you should either pip uninstall or remove mambaforge folder and start install allover to be sure
How do we know pytorch is working? ipython and import torch and see
1:09:12 Install Jupyterlab
How to install jupyterlab with mamba? mamba install jupyterlab
1:10:11 Create the first notebook
how to create a nbs folder? mkdir nbs
go inside this folder cd nbs
run command jupyter lab
Where to find the link to open jupyter lab?
How to run jupyter lab without browser? jupyter lab --no-browser
How to find and check the content of .bash_history? cat .bash_history 1:12:53
How to make the most out of .bash_history file?
How to search the .bash_history commands? ctrl + r and use delete tab to clear the search, use esc tab to exit search
How to run a command starting with ju? !ju
How to run the last command? !!
How to show the last command? echo !!
How to add alias in command line? alias jl="jupyter lab"1:15:37
How to move around the command line, start of a line, end of a line, move a word at a time? ctrl + e, ctrl + a, opt + left or right (to move a word at a time, esc + B and esc + F can do with my macpro)
How to write and save alias for good? 1:17:05 open .bash_rc and add the alias and save (on my macpro, I have to use .bash_profile)
How to edit files in terminal without vim? use open . to open a folder, and to edit a file with vscode
Radek mentioned some additional advantages of WSL
How to create a jupyter notebook in a folder from jupyter lab 1:21:28
How to install jupyter ipywidget? mamba install ipywidgets
Can we do gui stuff with jupyterlab, like click and delete?
Additional tricks without going too far
how to clear the screen of terminal? ctrl + l
How to clear a line of command in terminal? ctrl + u
How to clear a word of a line of command in terminal? ctrl + w
My expectations of walkthru 1 are very similar to @radek and nick:
I want to have Jeremy’s way of setting everything up and use Jeremy’s tricks to get things running without friction, and more importantly I can simply do the same setup online by opening a web-browser (like paperspace) without relying on a specific local machines with its own specifics.
So, here are my questions to @jeremy about the walkthru 1:
Why didn’t we apply this walkthrough directly to paperspace to setup python, mamba, jupyterlab and everything in the cloud rather than locally, as there are endless possibilities to mess up locally?
Will you show us how to do it on paperspace? or is it easy enough for us to apply what we learnt from walkthru 1 and directly apply it to paperspace?
I think doing all walkthrus in paperspace can really assure all of us that whatever Jeremy is demonstrating in the videos we can do the same online without too much trouble.
We’ll be doing paperspace soon. But I recommend having a working environment on your own machine too.
These walkthrus show you how to do things the best way, not the easiest way. And the best way is to be able to work directly on your own machine sometimes.
Creating a new environment.
My suggestion would be to create a new user on your system and use that to follow along. In that way if you have created scripts to setup an environment you can place them in a shared place for all users and then remove and recreate that user if it goes wrong, or make a ‘backup’ of the initial installation, although as things change that might not be useful. A new user is dependent on disc space that you have locally of course.
I note that users of linux based systems should be familiar with the available help systems these include man pages e.g ‘man some_command’ or info ‘info some_command’ indeed ‘info info’ is a good place to start.
The linux command ‘env’ lists all variable pointers available to the shell and this is where it can help to resolve issues. I note in my ‘condo’ system after I have activated it many ‘CONDA’ variables are defined, I haven’t got to install ‘mamba’ yet so not sure how or if they complement each other in the definition of variables that point to executables.
I experienced an issue installing Mamba 4.12.0 latest release, the issue being that the embedded packages in the shell script file ’preconda.tar.bz2’ failed to extract, why, maybe because my hardware could becoming out dated, MacBook Pro mid 2009 OSX 10.11.6. However I managed to install Mamba 4.11.0. You may ask why install there as I have other more recent Apple and linux platforms, well simply because Anaconda was giving me problems with installing later releases of python there and would hang for long periods causing abortion of the script.
I’m trying to replicate everything in the Walk-thru. I use a local computer Ubuntu 20.04 that already has a conda environment. So I created a new user for the mambaforge installation. I switched to the new user as Jeremy does in the video, there is no file or folder in the home folder, no python too, but when I check jupyter, it is there:
Yes it’s a problem. Try which jupyter to see where it’s coming from. I’m guessing you’ve accidentally installed jupyter into your system python. Type sudo pip uninstall jupyter from your terminal with no conda env activated to remove it.
That’s another conda install, and shouldn’t be in the path of a new user. Something odd has happened there.
That’s in your system python.
So you need to uninstall whilst logged in as niyazi and with conda activated, and you also need to uninstall with no conda activated. You’ve sure installed jupyter a lot of times and places!
Here are my notes on Walkthru 1, there are two parts, first part is about the tips and shortcuts mentioned by Jeremy and the second is my terminal timeline. It may be handy when you are watching it.
Tips & Shortcuts from Walkthru 1 :
To move things, use mv, the first is the source and the second is the target, if there is no target then it works like a renaming, here is an example:
mv miniconda miniconda.old
To remove files & Folders : remove recursively force r recursive and f forced.
rm -rf
Some list (ls) options:
ls -l # -l is long-form ls
ls -h #h human-readable form, for example, file size as MB or GB but not bytes.
# When you need to use multiple flags then no need to put two times
ls -lf
ls -a # for listing all files (hidden)
Aliases
# Aliases
# up and down arrows work as shortcuts because of .bash_history
# it is handy to create your scripts by copying from the history.
# CTRL-r is great for searching the history in the terminal
# also !ju runs the last one starts with 'ju'
# !! runs the last command
alias jl="jupyter lab" # no spaces and if you want make the aliases persistent then
# to them put in the .bashrc
My command timeline for the installation with some comments.
Do not use the python that comes with the system installation.
Download mambaforge and work install it and work on the ‘base’ environment
# .sh file is a script and to see it
less Mambaforge-Linux-x86_64.sh
# to install
bash Mambaforge-Linux-x86_64.sh
# remove the mamba for a automatical setup
rm -rf mambaforge/
.bashrc # automatically runs when you run the terminal so,
# edit the file for removing the conda initialisation, just delete the conda part.
# that will remove the [base] from the command line
# fastsetup arranges everything for a new computer
https://raw.githubusercontent.com/fastai/fastsetup/master/setup-conda.sh
./setup-conda.sh #no need for bash at this time because it is already in the script's first line
# doesn't help on the first try, because We need to change permission for execution
chmod u+x setup-conda.sh
# then again:
./setup-conda.sh
# in my case I've got this error
# ./setup-conda.sh: line 22: curl: command not found
# and I install curl:
sudo apt install curl
# then pytorch from here: https://pytorch.org/get-started/locally/
mamba install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
# -c pytorch means install things from pytorch channel
# then Jupyter or JupyterLab
mamba install jupyterlab
I am still catching up with the walk thrus but here are some questions:
Did anyone have an issue with ‘—no-progress-meter’ in the setup shell script from the repo? I removed the option and the script ran fine but I’m wondering what’s different about my setup that doesn’t recognise the flag.
Is there any reason why we are installing Jupyter Lab and not Jupyter Notebook?
