Live coding discussion

Part 1 2022
122

This is incredible. I am so excited to share that I have successfully configured my own PC from Ubuntu installation to installing all the tools for doing ML via the fastai. While I knew few bits before but it is first time that after watching these walkthroughs, I know exactly what I am doing. I can figure out reasons behind issues like cuda 10 was not working with pytorch on RTX 3090 GPUs. I was able to uninstall pytorch (without worrying to break stuff) and then reinstall the right version to make it work. Everything has been working perfect so far including the terminal, git, tmux, ctags, and most importantly vim. I loved exploring the codebase like @jeremy :slight_smile: using ctags in vim. So excited to watch the recorded sessions as it is quite early in the UK. Thanks Jeremy for all this.

5 Likes
123

Jeremy showed in one of the walkthroughs that once we install a new env, conda creates a separate folder in the envs subfolder. I was wondering if I could move this folder to persistent storage and create a soft link as we did for the .local folder. Then I can maintain an environment where I can use mamba and keep our installations current without relying on a paperspace container. I tried and it worked.

Here are the steps I took:

  1. conda create -n dl python=3.9. Here dl is the name of my new env.
  2. conda init.
  3. conda activate dl
  4. conda install mamba -c conda-forge
  5. mamba install fastbook -c fastchan. You can install more packages if you want or you can install packages later on.
  6. mv opt/conda/envs/dl /storage/envs/dl.
  7. cd opt/conda/envs/
  8. ln -s /storage/envs/dl

We need to make changes to the pre-run.sh file as well. We also need to make sure that we activate the new env before we launch the jupyter lab. Keeping this in mind here is how my pre-run.sh looks :

#!/bin/bash
cd
rm -rf .local
ln -s /storage/cfg/.local/

rm -rf .ssh
ln -s /storage/cfg/.ssh/

rm -rf /opt/conda/envs/dl
cd /opt/conda/envs/
ln -s /storage/envs/dl

conda init
conda activate dl

cd /notebooks

When I launched jupyter lab, it was using the new env that I created. To update packages, I can use the terminal. In the terminal,I still need to use conda activate new_env and then use mamba to update/install packages. Hopefully, I am not making a rookie mistake. One downside of this method is that the env folder in the storage can get bigger as we add more packages.

2 Likes
124

Very smart - nice work! The only downside of this is the storage space it’s using.

BTW you can put conda activate in
.bashrc.local

125

Is there a walk thru meeting for today 6/6/22 ?

EDIT: Nevermind, I see it’s already in progress :smiley:

126

Can someone remind me which video exactly was it where we created a pre-run.sh file that failed to cd back to the home dir at the end, as a result of which folks couldn’t see their notebooks any more? I’d like to add a note to the top of the description and to the walkthru thread about that.

(apologies originally I posted this in a locked thread! since that time I got the answer from @brismith - it’s walkthru 4)

1 Like
127

Are these two places what you look for?

128

Mainly for my future reference, here’s the plan for walk-thrus:

About these sessions

The focus of these sessions will be on the actual steps we need to take to complete all the things we do in a lesson, rather than the underlying machine learning theory. Things we’ll be covering along the way:

  • Using the terminal and bash shell (both basic getting-started stuff, and productivity tips for using them effectively)
  • git, GitHub and version control
  • Using vim
  • Intermediate and advanced Python stuff used in the course (list comprehensions, broadcasting, args and kwargs, etc)
  • More detail on each step of the modeling process.

These sessions will be suitable for curious and tenacious beginners who want to really understand how to get everything working properly, and patient experts who don’t mind sitting thru some stuff they already know to pick up some nifty coding and machine learning tips and tricks. @n-e-w and @radek will be joining me on these journey. :smiley:

Please reply below if you have any questions/comments/requests/etc.

Things to cover

I’ll use this section to list things we come across during the sessions that we want to look into further later:

  • How to blog
  • Running different pythons in notebooks
  • how do you “reload” bash after making changes to .bashrc
  • how do you create an “alias” that can take a parameter?
  • fastchan
  • how to create a git repo locally
  • nbdime and nbdev
  • set up home gpu server to be accessible from outside of local (home) network
1 Like
129

These sessions are great, It is sort of peeking at your thought process behind lessons, witnessing how you approach the problems that appear suddenly. I joined live sessions only 4 times because of my time zone (GMT 3+) but I watched them carefully a couple of hours later. Looking forward to nbdev and home GPU server sessions.
Thank you, Jeremy.

2 Likes