Beginner: Python, git, bash, etc ✅

suvash · September 23, 2022, 11:12am

Hmm, that could be because you’ve been using https url for cloning. Could you try running the following commands and paste the output here.

ssh -T git@github.com
cd /tmp && git clone git@github.com:fastai/fastai.git

If the first command succeeds without interaction, then the second should work accordingly as well. If not, then it might be something very odd going on. Let us know.

Daniel · September 23, 2022, 2:10pm

After running the first line ssh -T git@github.com I got the following

I just wonder could changing IP address really help from the post above ?

Daniel · September 23, 2022, 2:13pm

I run the command and got the following result

(base) 22:11 ~ > ssh -Tv git@github.com
OpenSSH_8.6p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/Natsume/.ssh/config
debug1: /Users/Natsume/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to github.com port 22.
debug1: Connection established.
debug1: identity file /Users/Natsume/.ssh/id_ed25519 type 3
debug1: identity file /Users/Natsume/.ssh/id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.6
debug1: compat_banner: match: OpenSSH_7.6 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug1: Authenticating to github.com:22 as 'git'
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-rsa SHA256:rEmlJenVMSL5GVemSY0Gk8WGw6B4ege4J85M+vup8R0
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'github.com' is known and matches the RSA host key.
debug1: Found key in /Users/Natsume/.ssh/known_hosts:1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /Users/Natsume/.ssh/id_ed25519 ED25519 SHA256:UFKLp/rN0o7czWwT5DukRK6TL2GyxKzfX8alHvlIimc explicit agent
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering public key: /Users/Natsume/.ssh/id_ed25519 ED25519 SHA256:UFKLp/rN0o7czWwT5DukRK6TL2GyxKzfX8alHvlIimc explicit agent
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: password
git@github.com's password:

Daniel · September 23, 2022, 2:16pm

I have run ssh -Tv git@github.com several times with different locations, and got the same result

(base) 22:14 ~ > ssh -Tv git@github.com
OpenSSH_8.6p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/Natsume/.ssh/config
debug1: /Users/Natsume/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to github.com port 22.
debug1: Connection established.
debug1: identity file /Users/Natsume/.ssh/id_ed25519 type 3
debug1: identity file /Users/Natsume/.ssh/id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.6
debug1: compat_banner: match: OpenSSH_7.6 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug1: Authenticating to github.com:22 as 'git'
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-rsa SHA256:rEmlJenVMSL5GVemSY0Gk8WGw6B4ege4J85M+vup8R0
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'github.com' is known and matches the RSA host key.
debug1: Found key in /Users/Natsume/.ssh/known_hosts:1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /Users/Natsume/.ssh/id_ed25519 ED25519 SHA256:UFKLp/rN0o7czWwT5DukRK6TL2GyxKzfX8alHvlIimc explicit agent
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering public key: /Users/Natsume/.ssh/id_ed25519 ED25519 SHA256:UFKLp/rN0o7czWwT5DukRK6TL2GyxKzfX8alHvlIimc explicit agent
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: password
git@github.com's password:

bencoman · September 23, 2022, 2:30pm

Not quite at that point yet. It only worked direct to a known IP address, not to the domain “github.com”

Awesome. Thats a really good step. This proves your keys are good and you are hitting a legitimate github server.

$ ssh -Tv git@github.com
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password,keyboard-interactive

So “password” being available here as an “Authentication method” hints this server doesn’t belong to github, thus you should suspect your DNS resolution from “github.com to IP address” has been compromised. This could be either upstream or on your machine, so we need to discriminate between those.

Could setting IP address manually be the solution?

As last resort yes, but first should examine your DNS. If your DNS resolution is compromised, then that affects the security of your whole machine, so you should try to fix that first. The general scenarios are:

The DNS servers of your ISP have been compromised/poisoned.
The network packets sent to DNS servers are being intercepted and modified.
Something on your local machine has been compromised.

These are in order of easiest to examine and isolate. The first case can be tested by changing which DNS servers do your DNS resolution. To use google’s DNS servers rather than your ISP’s DNS servers, follow instructions here…

kurianbenoy · September 24, 2022, 10:50am

Thank you @bencoman for pointing out the difference. I have been using SSH merely as user, yet it now looks I should put more effort to understand this.

Daniel · September 24, 2022, 2:07pm

Great news! Right inside this page, lies the solution to my git problem!

Hello guys @bencoman @kurianbenoy @suvash, thank you all so much for helping me out!

Being cut out from using github is like being cut off from learning and sharing codes with the world, and most importantly from fastai community, which is unthinkable.

Luckily, I have wonderful support and help from all you guys and finally @suvash has been amazingly kind and offered me a video call to help me troubleshoot the git problem. Thank you so so much, Suvash!

The source of the problem (I will try to describe it as clear as I could) is that in some places of the world, people are blocked from google and now github completely (blocking port 25). The solution Suvash provided is that if people choose to connect to github on the port (port 443) where the rest websites are based, then the blocking won’t work.

You guys are amazing, thank you!

suvash · September 24, 2022, 2:24pm

No worries Daniel ! I had some free hours during the day, and figured that I could try to help and sort this out a bit. Great that we were able to figure this out together.

Yeah, I haven’t really seen this in a while, but some ISP firewalls still like to ban/intercept/drop packets on SSH(22) ports. Good that Github allows connecting over the HTTPS(443) port, which solved your problem.

sdCarr · September 29, 2022, 11:10am

¡ Hello everyone! I’m doing JH’s tutorial to install nbdev and create a sample card package. I have a recurring git problem that I have “suffered” before and I don’t quite know how to get out of it. When I want to clone with the terminal the repository just created on Github, I get the following error message: Cloning in 'my-nbdev-repo'... git@github.com: Permission denied (public key). fatal: could not read from remote repository. I have tried to create new public and private ssh keys, but I can’t clone the repository. Does anyone know how to deal with this problem?

bencoman · October 2, 2022, 7:45am

Read the several posts between myself and Daniel, starting here.

xt0r3 · December 20, 2022, 12:47pm

I am writing here because I saw you mention that you recommend that we use mamba to install packages because PIP doesn’t generally install packages in a way that works on GPUs.

I just wanted to flag that I’d used pip as my package manager when installing the course and everything seems to work fine for me.

My environment is Windows 11 running WSL, and my GPU is an RTX 3070Ti.
I’ve attached the results of the nvidia-smi monitoring as an image to make sure that the GPU is actually working and it’s not just me mistaking something for GPU use.

erin · May 15, 2023, 9:02pm

Hello,
I just started this course and I’m a little confused on how you did this.

It looks like you are cloning into your windows Document folder. But then it says that you cloned into a git folder. And then you go into your WSL and open it up directly in VS Code but when I cloned it, it didnt automatically create a folder in git/ in my WSL.

erin · May 15, 2023, 9:04pm

heres the other image, it doesn’t let me post more than one.

ForBo7 · May 16, 2023, 4:40am

Could you upload a screenshot of your folder’s directory structure?

Also, are you cloning into a folder which already has Git initialized?

erin · May 20, 2023, 12:30am

sorry, no I didnt clone it into a folder with git already initialized. I’ve been watching the live codings on youtube. I don’t have the problem anymore