Always encountering errors running the course code

Hi!

I’m on Lesson 2 and have encountered several issues in both Lesson 1 and Lesson 2 so far. These issues are errors encountered whilst trying to get the notebooks to run.

My local computer is a MacBook Pro and my DL server is a AWS server running Ubuntu as per the setup guide for this course.

To get the Lesson 1 notebook to work for me, I had to make the following edits:
C2

from fastai import *
from fastai.vision import *

C9

np.random.seed(2)
pat = r’/([^/]+)_\d+.jpg$’

C10

data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=224, bs=bs, padding_mode=‘border’)
data.normalize(imagenet_stats)

C?? - Resnet50

data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(),
size=299, bs=bs//2, padding_mode=‘zeros’).normalize(imagenet_stats)

With Lesson 2 I encountered errors right off the bat. I had to add:

from fastai.vision import *
from pathlib import Path

I them spent a lot of time trying to figure out how to upload files as there is no Upload button on my Jupyter Notebook. After more time spent doing non-DL stuff, I managed to find out how to do it.

(for others the command is:
scp -i ~/.ssh/[your key file] [local file path] [username]@[aws ip]:[destination file path]
)

So now I have the text files uploaded but the code throws another error:

path
"PosixPath(’~/data/bears’)

download_images(path/file, dest, max_pics=200)
“[Errno 2] No such file or directory: ‘~/data/bears/urls_black.txt’”

Screen%20Shot

My frustration is that I have spent the past 90 mins dealing with errors not associated with the subject matter, as what I experience in front of me as a student, does not match what is displayed on the video. Its like trying to follow a teacher who is reading a different text book to what the student has. I can eventually figure it out but it takes 10 times longer and its incredibly frustrating!

I’m also mindful that I may have missed something or doing something wrong that is plain obvious for a software developer. Any advice will be greatly appreciated.

Hi !

First off let me reassure you : you are not alone ! Many fast.ai students go through the same frustrating steps of setting up the environment and painfully learning about stuff that is loosely related to DL. I certainly did, and I still do quite frequently. If you didn’t read it yet, I recommend Sylvain’s blog post “One year of deep learning” about how he went from no experience at all in all that (DL and the computer stuff) to where he is now.
So please keep at it. It’s frustrating now, and it will still be frustrating the next few times you’ll encounter issues. But I assure you’ll figure this out, you’ll become better and better with practice, and you’ll get better at solving those kind of issues.

For your error : it’s difficult to diagnose exactly what’s going on with the information you gave so far but here’s the step you can take to find out. You can call path.ls() to list the files inside the path directory. If you see the files like you see them on the screenshot you put on your post, there’s indeed something weird going on (and I’ll need more info to help you out). If not, try to navigate around to understand where you are and point the path to the right directory.

Good luck !

PS : you choose to use AWS. It’s a great service, used a lot in the industry so it’s great to be familiar with it. However, it’s also one of the hardest option to run fast.ai out there. In the fast.ai course v3 website, it’s listed as an “advanced” option.
If you find it frustrating right now you can move to another easier option (like Crestle or Gradient), and maybe go back to AWS later on ?
That being said the issues you mentioned doesn’t seem to be very AWS-related.

4 Likes

Getting up and running on a server, importing data takes time for anybody. also i agree that while aws is awesome in the longer run, there are far easier ways to get started such as paperspace unless you are already familiar with aws.

I would start with the easy solutions, because many of those that choose to continue studying orworking with AI after 6 months ends up bying a stationary to keep the cost of experiements down

1 Like

I tried Crestle but it timed out when launching a notebook. I’m in Perth, Western Australia so its about as remote a city as you can get so I assume the latency killed it. I also have AWS credits so I’m keen to use them before they expire.

I launched the lesson 2 notebook again and did some testing.

!more ~/data/bears/urls_grizzly.txt
https://thumbs-prod.si-cdn.com/H31KVJSPQiOenWdUV5_QlzKpPwQ=/800x600/filters:no_u
pscale()/https://public-media.si-cdn.com/filer/ca/67/ca67cdf9-e061-4aa1-be4c-8ae
0589b60fa/4trout-lake-bear-landagildart.jpg
https://upload.wikimedia.org/wikipedia/commons/thumb/1/1a/Grizzly_Bear_Family_in
_Glacier_National_Park.jpg/1200px-Grizzly_Bear_Family_in_Glacier_National_Park.j
pg
https://ichef.bbci.co.uk/news/660/cpsprodpb/DFB2/production/_103566275_b05fb6f1-
5594-409b-8b26-79786f7e2c4f.jpg
https://upload.wikimedia.org/wikipedia/commons/thumb/3/33/Jasper_Dwayne_Reilande
r-4.jpg/220px-Jasper_Dwayne_Reilander-4.jpg
https://www.biologicaldiversity.org/assets/img/species/mammals/GrizzlyBearRobinS
ilver.jpg

This suggests that the notebook can read the text file, so that’s good.

Next …

path/file
PosixPath(’~/data/bears/urls_black.txt’)

The path seems ok…

I tried using the path/file and manually entering the file path … same result for both

###download_images(path/file, dest, max_pics=200)
download_images(’~/data/bears/urls_black.txt’, dest, max_pics=200)

FileNotFoundError Traceback (most recent call last)
in ()
1 #download_images(path/file, dest, max_pics=200)
----> 2 download_images(’~/data/bears/urls_black.txt’, dest, max_pics=200)

Here is the ls -l of the directory.

drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 29 22:49 black
drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 29 22:49 grizzly
drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 29 22:49 teddys
-rw-r–r-- 1 ubuntu ubuntu 10010 Jan 29 22:45 urls_black.txt
-rw-r–r-- 1 ubuntu ubuntu 11098 Jan 29 22:46 urls_grizzly.txt
-rw-r–r-- 1 ubuntu ubuntu 10007 Jan 29 22:47 urls_teddys.txt

I’m thinking maybe permissions or something? Do these flags look ok?

Any ideas anyone?

I did some more experimenting and I copied the data from ~/data/bears to a sub-folder in the directory where the fastai dl1 is installed. The notebook worked first go. I don’t know why it couldn’t load files from a directory outside of the fastai directory but at least it’s working now.

Great job getting it working ! Based on what you’ve said I think that was what happening is that the path variable was relative to where the notebook is located, so to get to the data you could have put some ../../ in there.

Next …

path/file
PosixPath(’~/data/bears/urls_black.txt’)

The path seems ok…

When you typed path/file it didn’t actually show you the file or the directory. It was a variable, just like it would be if it had been a string.

1 Like