Thanks for your answer. It’s help a lot.
I also noticed that paperspace has mamba now pre-installed. I checked out your blog (awesome!) and found that I have set things up almost identical.
However, when trying to install any other (python based) package through mamba (so not
universal-ctags) Mamba still wants to install way too much stuff, e.g. also a python version together with a huge number of other stuffs.
It looks as if:
- the system is using python 3.9.13 in
python --versionin the terminal)
- all main dependencies such as torch and fastai are all installed in
ipythonfrom the terminal and import for example
fastaiand then run
- this is the stuff that is used by ipython and jupyter lab
- installing python stuff with
pip -U --useris integrated with that (I tested it with a couple of libraries)
- mamba is a totally separate installation with a separate python (version 3.10.6) installed in
mamba listin the terminal)
- this mamba is and can only be used to install (
mamba install -p ~/conda <package>) none python specific stuff such as
unverisal-ctags, for anything else, it will try to install a completely fresh installation of python as well… and such packages will also not be integrated with the system python (
Long story short, we should still install any python specific additional libs with
pip for it to work in any jupyter notebook.
Another way to see this, is by creating a notebook (an actual notebook…) and run
import sys; sys.path which doesn’t mention anything on
['/notebooks', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '', '/root/.local/lib/python3.9/site-packages', '/usr/local/lib/python3.9/dist-packages', '/usr/lib/python3/dist-packages']
Is this also your experience? Or am I doing something wrong?
Thank you for reading my blog. I really appreciate it.
I also get the same output as yours. I got this: (starting from
['/notebooks/paddy', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '', '/root/.local/lib/python3.9/site-packages', '/usr/local/lib/python3.9/dist-packages', '/usr/lib/python3/dist-packages']
I also have the same experience as yours, and I noticed many of your observations. I could not execute packages in notebook that I installed with mamba. I could use mamba installed packages on terminal, but not in the notebook. So, I just installed packages I needed using
pip -U --user command. So far, I have not run into any issue from using pip.
Does this answer your question?
Great, thanks for confirming. I also just found this post from Jeremy:
Which seems also to be aimed at the new setup, where mamba comes pre-installed.
Thanks so much for these videos, it’s great to spend so much time on the setup and get comfortable with it
There is one last thing I would like to understand better, the
.bashrc and .bash_profile scripts.
The background is that I noticed that when we enter the terminal in jupyter lab, the aliases set up in the
.bashrc script are not working (
ls etc), as well as the coloring of folders/files when executing
ls (which is configured in
.bashrc) is not working.
However, when executing
bash in the terminal all this stuff seems to work all of a sudden [Also, the conda environment is being activated, as shown by the
(base) on the start of each line, which is also part of the
.bashrc script]. So somehow entering a new bash ran this
I read somewhere that the
.bashrc script is executed whenever we are entering a “non-login shell” and the
.bash_profile script whenever we enter the “login shell”.
I found out here that we can run
shopt -q login_shell && echo 'Login shell' || echo 'Non-login shell'
to find out whether we are in a login or non-login shell.
Running this command in the jupyter terminal gives that we are running in login shell, so that indeed the
.bash_profile script should be executed and not the
However, I also spin up a totally empty machine (without fastai or anything) on paperspace and found that a very similar
.bashrc is created, so with all the coloring and aliases included. However, opening a terminal on that machine seems to run this script since coloring and aliases are working directly. I guess this is because there exist no
bash_profile script by default, so instead it executes the
bashrc. And indeed upon
touch .bash_profile and opening a new terminal the coloring and aliases are gone, because now it is executing the (empty)
.bash_profile and no longer the
.bashrc for login shell’s.
Question is: can somebody shed some light on this, what’s the reason for having a login and non-login shell? Why does only the non-login shell activate the
base conda environment? And last but not least, how does this relate to Jeremy having coloring in the terminal upon opening a terminal from (paperspace) jupyter?