Having video timestamps would help to locate the problem at hand actually.
I need to go back and watch the micromamba part. I think at one point Jeremy removed a bunch of large files because we don’t want the file sizes to get too large and removal of libicudata was part of that.
I don’t see how typing /storage/ would get one an equivalent of ls command, but having a timestamp would help locating what might be going on?
I dowloaded it using wget as I described in my post and installed it to the ~/conda folder. I did everything else as you described and it seemed to work fine.
For some reason, bash_history is not in my home folder. Any tips how to locate that file, or how to find where it defaults to? Not a big deal, but I haven’t been able to create persistence for that.
Why is what we did in this walkthrough better than just creating a persistent virtual environment as described here and then installing ctags? Less disk space?
Another fantastic session. Thanks Jeremy! And now we are ready to explore the fastai library in Paperspace optimised for fastai!
6:36 - Creating a persistent environment in Paperspace
13:08 - Conda install mamba with -p to control directory location
13:30 - Install universal-ctags using micromamba
(but I used conda install -p ~/conda -c conda-forge universal-ctags because I didn’t have micromamba at the time and it seemed to work fine)
14:50 - Clean up conda directory
18:30 - Fixing path to universal-ctags and mamba
20:20 - Create a bash.local file in /storage
23:30 - Install micromamba into conda folder
cd cd conda curl -Ls https://micro.mamba.pm/api/micromamba/linux-64/latest | tar -xvj bin/micromamba
24:00 - Remove mamba and move conda folder into storage
24:40 - Edit pre-run.sh file with symlinks to conda
25:20 - Preserving .bash_history file
30:00 - Test setup on new machine
34:30 - Clone forked copy of fastbook
42:30 - Adding git config file to persistent storage
45:00 - Discussion about making contributions to repos with pull requests
48:00 - Comparing different versions with nbdime on Paperspace
48:20 - Start fastbook chapter 1 and tips for navigating and understanding a library
51:20 - __all__ is pronounced “dunder all”
52:50 - A nifty trick for navigating source files: Place cursor on object and press Shift + 8 ‘*’
57:30 - Optimising storage use on Paperspace (/storage/data# du -sh *)
59:40 - Move fastai config.ini into storage and symlink
1:05:45 - The Path.BASE_PATH variable trick
1:09:00 - The fastai L class: a dropin replacement for a list
Try redirecting to a text file `ls conda/bin > condabin.txt. each item is on a separate line. What are you trying to achieve perhaps just a mindful moment
Each command has an input stdin and an output stdout
so the ls command has input conda/bin
the ls command stdout is returned to the terminal and formatted for display
intercept ls stdout by redirecting to an txt file shows what is return to the terminal view.
your search string return exactly the 2 lines containing mamba
Not sure if that is clear
Thank you for your reply! Indeed when I redirect the output to a text file things appear line by line!
I think I understand this now. The terminal (or my shell?) is doing the formatting so that it is displayed how it is. Makes sense, thank you for the explanation
The usual* behaviour for linux tools when redirecting or piping is to drop all the decorations(formatting+colours), so that only compact cleartext is passed forward.
Similarly, ls can determine if the output stream is the usual stdout(defaults to the terminal) or a redirection(>,>>,…), or a pipe(|), and behave accordingly.
In the case of ls
when piped it behaves similar to ls --format=single-column --color=never
you can override this (though I wouldn’t) by forcing a column formatting using ls --format=vertical
(There could be some more options I’m missing above here)
Try running the second command with a pipe and you’ll get what you wanted
ls --format=vertical | grep mamba, which will be similar to the output of
ls --format=vertical --width=80 --color=never | grep mamba
So yeah, it’s all about what ls considers as sane defaults in various cases for what the output stream looks like. And, of course you can override them.
wow, I didn’t know if info!!! There is a bit more information on what that is on Wikipedia here, if anyone might be interested.
In practical terms, it seems some entries for man and info are the same, but her for example in case of ls they are very much different, with info being much more extensive!
Thank you very much for telling me about this, Jeremy
I managed to get my persistent environment. in paperspace working. The last final detail is the .bash_history file, it does not show up in my root dir, so I cannot make it persistent. How to solve this?
You can create an file with touch .bash_history and just use that. Or open a terminal, type one command, close the terminal, then reopen, and the history will be there.