Live coding 2

This topic is for discussion of the 2nd live coding session.

<<< session 1session 3 >>>

Links from the walk thru

What was covered

  • Understanding github and git
  • Creating a git repo
  • Setting up ssh keys
  • Cloning a repo
  • Using tmux for better terminal productivity
  • Installing fastai and fastbook
  • Committing and pushing to git

Video timeline - thank you @Daniel

00:00 - People intro and questions to address later

03:27 - Switch users in Linux

  • How to switch users in terminal with Linux machine? sudo -u username -i
  • Why or when may you need to have multiple users in your machine?

04:48 - Introduction to git and Github

  • How to create a notebook and put it in the github?
  • What is a git repository? 05:19
  • Can github repo store all versions of your documents?
  • How to tell organisations/individuals and their repositories?
  • What kind of files are stored in folders? how large can the files be?
  • What is the file README.md? What’s special about it?

08:32 - Build a website with github

  • How to create a new repository?
  • Why should you always make public repository?
  • What to do with .gitignore, and why? 10:52
  • What license should I choose and why?
  • What inside the .gitignore and what exactly does it do? 12:37
  • What’s the raw view of the file?
  • How to actively choose specific file or folders to include or exclude? (later)
  • How to create a little website by editing the README.md on github? 14:50
  • What is the markdown? How to make different levels of headings? How to make another paragraphs? How to make a list?
  • How to save a file in github with commit with version control? 16:33
  • Where to check with all the commits or changes of specific file and check the differences (diff) in history? 17:56
  • How to go back in history for the whole repository? 18:11

19:00 Clone your repo with SSH
19:00 - Setting up and using ssh keys

  • How to put the repository on your computer?
  • Why and how to clone the repo with SSH?
  • Why should I make a folder to store all my git content?
  • How to clone the repo with SSH into your local computer? 20:11 git clone url-copied
  • What are the security measures?
  • How to clone others’ repo normally? using HTTP copy not SSH copy above
  • How to configure your git when you try to commit/save changes in the repo? 22:11
  • How to find the .gitconfig file and add your username and password for saving changes in repo?
  • How to switch to the latest directory you used? cd ~
  • How to remember a directory and get back to it fast? pushd ~ to remember, popd to get back 24:02
  • Why Jeremy almost never use HTTP for git clone but use SSH? 24:14
  • How we don’t have permission for cloning with SSH?
  • What are the private and public key of the repo? How they are useful?
  • How to create the key pair (private and public)? ssh-keygen and enter a bunch of times
  • How to find the private and public keys from the output? 26:35
  • How to display and copy our public key in terminal? cat ~/.ssh/id_rsa.pub and select and copy it
  • Why it is safe to lease this public key on public?
  • How to give this public key to the github to allow you to make commit? 27:12
  • more discussion on how easy and safe it is to use SSH
  • Can we push repo to different destinations? 34:28
  • Should you ever put private files in github public repo? 35:31

39:01 - Using tmux for better terminal productivity

  • How to install tmux in your terminal in linux windows? sudo apt install tmux
  • How to install homebrew for installing tmux on mac?
  • How to split the terminal screen horizontally by tmux? ctrl + b + % 42:43
  • How to split a subscreen vertically? ctrl + b + "
  • How to close a screen? ctrl + d
  • How to move around different screens or panels? ctrl + b + arrows
  • How to zoom into a screen/panel (to get more room)? ctrl + b + z and do it again to zoom back 43:44
  • How to get back to the previous terminal-tmux setup when you have some long running tasks? as long as no shut down computer, ctrl + b + d to detach, and close all terminals, then open terminal and type tmux a 45:21

48:54 - Create a notebook in jupyter lab

  • Two links to enter jupyter lab
  • How to create a notebook in the lab?
  • How to launch the classic notebook? 50:10
  • Where to find the keyboard shortcut for the lab? 51:13
  • How to change between md and code mode? m for markdown, y for code
  • How to make headings? use 1, 2, 3 for levels of headings
  • How to show a function parameters and usages after typing the function? shift + tab
  • How to close a notebook properly? ctrl + shift + q or close from kernel/terminal panel 53:39

54:26 - Committing and pushing to git

  • How to find out the unchecked files in git? git status
  • How to add a file into git? git add the-file-name
  • How to commit the changes? git commit
  • How to edit the commit message with nano editor? 56:00
  • By now, git has version controlled your repo locally
  • How to push the commit to remote github? git push
  • Share your works through repos on github

58:55 - Fork a repo

  • Why or when to use fork when you can clone with HTTP or SSH?
  • How to fork and then SSH clone?

1:01:31 - Installing fastai and fastbook

  • How to install fastai? mamba install -c fastchan fastai
  • Where to find installation info?
  • What does -c fastchan fastai mean? 1:01:59
  • Why Jeremy dislike virtual environments but only use base? 1:03:20
  • What does this line of code ! [-e/ content] && pip install -Uqq fastbook mean? only for colab environment to update fastbook
  • How to install fastbook? mamba install -c fastchan fastbook 1:04:36
  • What is inside fastbook and how it is different from fastai?
  • How to install sentencepiece to get rid of warnings? mamba install -c fastchan sentencepeiece?
  • How to git add all unchecked files and commit at once? git commit -am 'message' 1:08:01
  • How to see the change you made which is different from the fastai/master?
14 Likes

Github doesn’t support password based authentication for some time. They introduced PAT (personal access tokens). This would be alternative method of authentication. Would there by any pros / cons of using SSH vs PAT for github authentication?

Found the walk-thru 1 video to be very helpful and everything worked great! One thing I had to do was install wget for my Mac Terminal which was made easier using Homebrew (https://brew.sh/). Was sharing in case if it helps other mac users, assuming it is an okay approach.

3 Likes

I prefer SSH because it’s a single thing which works everywhere. As long as I’m working somewhere which has my SSH keys (which is everywhere!) I don’t have to think about it.

But either works fine really.

2 Likes

Hi,

I’m not an advanced Github user, but just wanted to share that yesterday happened to me exactly what Jeremy said in the session about having a key in a private repository and then changing it to public.

I wasn’t aware of the key, but after changing it to public I received an email from GitGuardian with the following message:

image


The key was written in a txt file like this:

I din’t knew about GitGuardian but I setup an account and now I can see detailed information from a scan the application did on my Github account.

Here the plans…

1 Like

What is good practice for committing and pushing commits?

How often should I commit to my local repo and how often push to the Github repo?
Is there good practice for this? Committing/Pushing to little could make you lose useful work an commiting/pushing to much might make tracing back an overwhelming task. Any ideas in this?

I always push when I commit. And I commit any time I reach a point where I’ve got something that’s kinda finished.

1 Like

Walkthrough 2

00:00 People intro and questions to address later

03:27 Switch users in Linux

  • How to switch users in terminal with Linux machine? sudo -u username -i

  • Why or when may you need to have multiple users in your machine?

04:48 Introduction to Github

  • How to create a notebook and put it in the github?

  • What is a git repository? 05:19

  • Can github repo store all versions of your documents?

  • How to tell organisations/individuals and their repositories?

  • What kind of files are stored in folders? how large can the files be?

  • What is the file README.md? What’s special about it?

08:32 Build a website with github

  • How to create a new repository?

  • Why should you always make public repository?

  • What to do with .gitignore, and why? 10:52

  • What license should I choose and why?

  • What inside the .gitignore and what exactly does it do? 12:37

  • What’s the raw view of the file?

  • How to actively choose specific file or folders to include or exclude? (later)

  • How to create a little website by editing the README.md on github? 14:50

  • What is the markdown? How to make different levels of headings? How to make another paragraphs? How to make a list?

  • How to save a file in github with commit with version control? 16:33

  • Where to check with all the commits or changes of specific file and check the differences (diff) in history? 17:56

  • How to go back in history for the whole repository? 18:11

19:00 Clone your repo with SSH

  • How to put the repository on your computer?

  • Why and how to clone the repo with SSH?

  • Why should I make a folder to store all my git content?

  • How to clone the repo with SSH into your local computer? 20:11 git clone url-copied

  • What are the security measures?

  • How to clone others’ repo normally? using HTTP copy not SSH copy above

  • How to configure your git when you try to commit/save changes in the repo? 22:11

  • How to find the .gitconfig file and add your username and password for saving changes in repo?

  • How to switch to the latest directory you used? cd ~

  • How to remember a directory and get back to it fast? pushd ~ to remember, popd to get back 24:02

  • Why Jeremy almost never use HTTP for git clone but use SSH? 24:14

  • How we don’t have permission for cloning with SSH?

  • What are the private and public key of the repo? How they are useful?

  • How to create the key pair (private and public)? ssh-keygen and enter a bunch of times

  • How to find the private and public keys from the output? 26:35

  • How to display and copy our public key in terminal? cat ~/.ssh/id_rsa.pub and select and copy it

  • Why it is safe to lease this public key on public?

  • How to give this public key to the github to allow you to make commit? 27:12

  • more discussion on how easy and safe it is to use SSH

  • Can we push repo to different destinations? 34:28

  • Should you ever put private files in github public repo? 35:31

39:01 Using tmux for better terminal productivity

  • How to install tmux in your terminal in linux windows? sudo apt install tmux

  • How to install homebrew for installing tmux on mac?

  • How to split the terminal screen horizontally by tmux? ctrl + b + % 42:43

  • How to split a subscreen vertically? ctrl + b + "

  • How to close a screen? ctrl + d

  • How to move around different screens or panels? ctrl + b + arrows

  • How to zoom into a screen/panel (to get more room)? ctrl + b + z and do it again to zoom back 43:44

  • How to get back to the previous terminal-tmux setup when you have some long running tasks? as long as no shut down computer, ctrl + b + d to detach, and close all terminals, then open terminal and type tmux a 45:21

48:54 Create a notebook in jupyter lab

  • Two links to enter jupyter lab

  • How to create a notebook in the lab?

  • How to launch the classic notebook? 50:10

  • Where to find the keyboard shortcut for the lab? 51:13

  • How to change between md and code mode? m for markdown, y for code

  • How to make headings? use 1, 2, 3 for levels of headings

  • How to show a function parameters and usages after typing the function? shift + tab

  • How to close a notebook properly? ctrl + shift + q or close from kernel/terminal panel 53:39

54:26 Committing and pushing to git

  • How to find out the unchecked files in git? git status

  • How to add a file into git? git add the-file-name

  • How to commit the changes? git commit

  • How to edit the commit message with nano editor? 56:00

  • By now, git has version controlled your repo locally

  • How to push the commit to remote github? git push

  • Share your works through repos on github

58:55 Fork a repo

  • Why or when to use fork when you can clone with HTTP or SSH?

  • How to fork and then SSH clone?

1:01:31 Installing fastai and fastbook

  • How to install fastai? mamba install -c fastchan fastai

  • Where to find installation info?

  • What does -c fastchan fastai mean? 1:01:59

  • Why Jeremy dislike virtual environments but only use base? 1:03:20

  • What does this line of code ! [-e/ content] && pip install -Uqq fastbook mean? only for colab environment to update fastbook

  • How to install fastbook? mamba install -c fastchan fastbook 1:04:36

  • What is inside fastbook and how it is different from fastai?

  • How to install sentencepiece to get rid of warnings? mamba install -c fastchan sentencepeiece?

  • How to git add all unchecked files and commit at once? git commit -am 'message' 1:08:01

  • How to see the change you made which is different from the fastai/master?

6 Likes

Yep, I wanted to raise this in class, what can happen if one shares their API key like that, but thought that probably no one was using API keys in their scripts just yet and that this was unlikely to happen to anyone…

The bottom line is this – if not careful, this is a very good way to lose a lot of money.

Just as there exists GitGuardian, so are there apps deployed by black hat hackers looking through every repo that turns public, looking at every commit, and looking for API keys they can use. Every now and then you will see someone post to social media about how they woke up one day to a message that over the last couple of hours they racked up thousands of dollars in instance costs.

Someone came across their API key that they pushed to a public repo and spun up all the instances that they could to power botnets or mine cryptocurrencies.

Now, generally GCP (and I would imagine other providers) have quotas, which should limit the damage, but even in such a scenario, this is not great. It is a bit of an exaggerated story, and generally making stuff public comes with a much lower risk than people assume (most likely very few people will notice you shared something and yet fewer will care, which can be seemed as unfortunate). But the point is – you need to be careful around storing API keys :slight_smile:

So that is a good story that you shared @fmussari! :slight_smile:

6 Likes

I love the way you’ve included keyboard shortcuts and command details here!

2 Likes

Hi Jeremy.

I don’t really know how to overwrite the gitconfig file to include my email and name - do I type something to save at the end?

Thanks!

You open the ~/.gitconfig file in the editor of your choice, and then type in your name and email over the top of the ones that are there.

BTW, if you’re able to join our sessions while they’re running, it would be amazing if you could share your screen and ask any questions you have about things you’re not sure about. It’s totally fine if they’re about older sessions – it’s really helpful to have opportunities to revisit them,

2 Likes

Sorry Jeremy. I am slowly catching up with the lessons I missed due to work. I did what you have listed in the message. I am assuming you are pressing Ctrl + something to save and exit the screen as below.

It let me get out this screen using :p! but it doesn’t save the details I have entered.

Thanks!

I got it!

It was esc :x !!!

Oh you’re using vim! Heh sorry I didn’t realise. In a couple of walkthrus time we’ll have a whole session focused on vim. To save and close vim it’s: esc :wq

2 Likes

At 49:44 Jeremy’s screen has only one option to create a Notebook…
image

but mine has two options…

Not an issue, since I can just choose the “ipykernel” one, but curious how the “conda envroot” alternative might have arisen, and when it might be preferred.

Few unrelated questions that I got as part of Walkthrough 2:

Do we need to install Homebrew and Tmux on the Paperspace Terminal? If we do so, does that count against storage? Also should that be installed while pwd is root?

Another question I had was I couldn’t get git to remove a file which I had added to the repo, until I did a git push in terminal in addition to the git commit…? But in lesson 2, it does not look like git push was needed. Why am I still seeing the file after doing rm and also commit; the file disappears from repo only when I do git commit. This was in paperspace terminal.

Also a different question: I was able to use SSH to connect the git repo with Paperspace terminal. But when I do cat .gitconfig, its not there automatically. I had to configure it using git config --global user.name “username” (similar to the instructions in https://linuxize.com/post/how-to-configure-git-username-and-email/). In Lesson 2, I think it appears that .gitconfig file should have been automatically created. But after I configure it manually, it works fine–no problems.

Thanks.

There’s a thing called nb_conda_kernels or something like that, which adds all conda envs as options when starting a notebook. I guess you got that installed.

You don’t need either. In a later walkthru we’ll see how to install software on Paperspace instances.

We’d need more info to help. When you say you couldn’t do something, please be sure to paste your command and what error message you received.