Early draft notebooks available

You’ll find there’s now a ‘courses/dl2’ folder available in github fastai. We’ll be starting on Monday with the ‘pascal.ipynb’ notebook.

You’ll see that there is little if any prose in the notebooks - in general there’s a lot less “hand holding” in part 2, compared to part 1, although I’ll try to provide all the information you need during class (and if course we’ll all help each other out here on the forums too!)

47 Likes

In case people need to know the commands to fire up an AWS, here’s the wiki link for the same

Whats the download link for pascal dataset?

From this link?

After registering on the website of Pascal, I cant find the 2007 data?

I supposed is from here: https://www.kaggle.com/c/carvana-image-masking-challenge

1 Like

Just to elaborate on the data question. That link above, https://pjreddie.com/projects/pascal-voc-dataset-mirror/, is a mirror of both the 2007 and 2012 data, but the annotations are all in an xml format. The json files referenced in the notebook look more like those from the COCO API. I found a detectron thread that seems to shed some light: https://github.com/facebookresearch/Detectron/issues/6

2 Likes

Glad to see Caravana challenge in the lecture notebook since I worked hard on this competition :slight_smile:

For quick overview of this competition, check 1st place winner’s solution:


and http://slides.com/vladimiriglovikov/kaggle-deep-learning-to-create-a-model-for-binary-segmentation-of-car-images

And I attached a short summaries of other solutions that I made for the local study group months ago. It was interesting to see different approaches for seemingly simple problem. There were many more different approaches that are not mentioned in the slide.
Hope you guys find this helpful.

carvana_challenge.pdf (1.7 MB)

11 Likes

Thanks for replying,
I figured it out after downloading the whole dataset…

You are correct but it seems from the nbs that two cool datasets are used

  • Carvana
  • Pascal for SSD
    I thought it would be good if we could have our machines setup already as downloading won’t take time, but extracting might…
3 Likes

@ecdrid I am slightly confused about the datasets.

the first notebook (pascal.ipynb) uses just the pascal dataset correct? Where should we download that from?

And there is a separate notebook for carvana right? And the dataset for the same comes from Kaggle ?

For Json:

**For the dataset:**Q
https://pjreddie.com/projects/pascal-voc-dataset-mirror/

5 Likes

Nice! Will be interesting to learn about object detection and segmentation.

1 Like

It seems working on my end.

I am having XML files, my bad…
Will confirm

Run this in dl2 folder, somehow I fail to pack it into a bash file, the wget command fail when it is in a bash script but works fine when I paste to terminal. Would love some help to pack it into one file…

git clone https://github.com/noklam/fastaipart2v2helpder.git

cd fastaipart2v2helpder
mv * …/
cd …/
cd data/pascal

wget --header=“Host: pjreddie.com” --header=“User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36” --header=“Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8” --header=“Accept-Language: en-HK,en-US;q=0.9,en;q=0.8” --header=“Cookie: __utma=134107727.377515788.1521354594.1521354594.1521354594.1; __utmc=134107727; __utmz=134107727.1521354594.1.1.utmcsr=forums.fast.ai|utmccn=(referral)|utmcmd=referral|utmcct=/t/early-draft-notebooks-available/13418/5; __utmt=1; __utmb=134107727.2.10.1521354594” --header=“Connection: keep-alive” “https://pjreddie.com/media/files/VOCtrainval_11-May-2012.tar” -O “VOCtrainval_11-May-2012.tar” -c
wget --header=“Host: pjreddie.com” --header=“User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36” --header=“Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8” --header=“Accept-Language: en-HK,en-US;q=0.9,en;q=0.8” --header=“Cookie: __utma=134107727.377515788.1521354594.1521354594.1521354594.1; __utmc=134107727; __utmz=134107727.1521354594.1.1.utmcsr=forums.fast.ai|utmccn=(referral)|utmcmd=referral|utmcct=/t/early-draft-notebooks-available/13418/5” --header=“Connection: keep-alive” “https://pjreddie.com/media/files/VOCtrainval_06-Nov-2007.tar” -O “VOCtrainval_06-Nov-2007.tar” -c
tar -xvf VOCtrainval_06-Nov-2007.tar
cd VOCdevkit/
mv VOC2007 …/
cd …/
tar -xvf VOCtrainval_11-May-2012.tar
cd VOCdevkit/
mv VOC2012 …/
cd …/

2 Likes

Just tested on Google Cloud Platform Ubuntu 16.04 instance, run pascal.ipynb successfully before the Bbox part with no issue. Note that you may have some different output as this Path object depends on your OS.

Window (My local computer, window 10)
image

Ubuntu 16.04:

1 Like

Thanks

We love these early treats XD.

I have an error in section
Bbox only

in line
x,y=next(iter(md.val_dl))

The error is
IndexError: index 383 is out of bounds for axis 0 with size 298

I think the reason is tfm_y=TfmType.COORD argument that is passed in tfms_from_model() function, becasue when I comment it, the error gone away.

I did not look at the source code yet, just want to report about a possible bug in the raw notebook.
I will report about solving that problem here if I will succeed.

Here is a link to the full error traceback if somebody interested - https://gist.github.com/anonymous/9140946b47097b0a8ca8bdd65ec09eaa

P. S. Does anybody have this error too?

3 Likes

I created a Gist to download pascal data based on @nok’s https://github.com/noklam/fastaipart2v2helpder

21 Likes

Had same error the coordinates may specify a region that does not exist in the 224 x 224 augmented image. Not sure if the code attempts to scale the image or coordinates

Yes, I also get the error in pascal.ipynb. This error we are seeing could be because of bounding box defined is outside of the image dimensions. Not sure if this is bad data or the bb definition not scaled in transformation. We may need to wait for @jeremy to upload his data files to see if its an issue there as well -