Early draft notebooks available

I am having XML files, my bad…
Will confirm

Run this in dl2 folder, somehow I fail to pack it into a bash file, the wget command fail when it is in a bash script but works fine when I paste to terminal. Would love some help to pack it into one file…

git clone https://github.com/noklam/fastaipart2v2helpder.git

cd fastaipart2v2helpder
mv * …/
cd …/
cd data/pascal

wget --header=“Host: pjreddie.com” --header=“User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36” --header=“Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8” --header=“Accept-Language: en-HK,en-US;q=0.9,en;q=0.8” --header=“Cookie: __utma=134107727.377515788.1521354594.1521354594.1521354594.1; __utmc=134107727; __utmz=134107727.1521354594.1.1.utmcsr=forums.fast.ai|utmccn=(referral)|utmcmd=referral|utmcct=/t/early-draft-notebooks-available/13418/5; __utmt=1; __utmb=134107727.2.10.1521354594” --header=“Connection: keep-alive” “https://pjreddie.com/media/files/VOCtrainval_11-May-2012.tar” -O “VOCtrainval_11-May-2012.tar” -c
wget --header=“Host: pjreddie.com” --header=“User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36” --header=“Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8” --header=“Accept-Language: en-HK,en-US;q=0.9,en;q=0.8” --header=“Cookie: __utma=134107727.377515788.1521354594.1521354594.1521354594.1; __utmc=134107727; __utmz=134107727.1521354594.1.1.utmcsr=forums.fast.ai|utmccn=(referral)|utmcmd=referral|utmcct=/t/early-draft-notebooks-available/13418/5” --header=“Connection: keep-alive” “https://pjreddie.com/media/files/VOCtrainval_06-Nov-2007.tar” -O “VOCtrainval_06-Nov-2007.tar” -c
tar -xvf VOCtrainval_06-Nov-2007.tar
cd VOCdevkit/
mv VOC2007 …/
cd …/
tar -xvf VOCtrainval_11-May-2012.tar
cd VOCdevkit/
mv VOC2012 …/
cd …/

2 Likes

Just tested on Google Cloud Platform Ubuntu 16.04 instance, run pascal.ipynb successfully before the Bbox part with no issue. Note that you may have some different output as this Path object depends on your OS.

Window (My local computer, window 10)
image

Ubuntu 16.04:

1 Like

Thanks

We love these early treats XD.

I have an error in section
Bbox only

in line
x,y=next(iter(md.val_dl))

The error is
IndexError: index 383 is out of bounds for axis 0 with size 298

I think the reason is tfm_y=TfmType.COORD argument that is passed in tfms_from_model() function, becasue when I comment it, the error gone away.

I did not look at the source code yet, just want to report about a possible bug in the raw notebook.
I will report about solving that problem here if I will succeed.

Here is a link to the full error traceback if somebody interested - https://gist.github.com/anonymous/9140946b47097b0a8ca8bdd65ec09eaa

P. S. Does anybody have this error too?

3 Likes

I created a Gist to download pascal data based on @nok’s https://github.com/noklam/fastaipart2v2helpder

21 Likes

Had same error the coordinates may specify a region that does not exist in the 224 x 224 augmented image. Not sure if the code attempts to scale the image or coordinates

Yes, I also get the error in pascal.ipynb. This error we are seeing could be because of bounding box defined is outside of the image dimensions. Not sure if this is bad data or the bb definition not scaled in transformation. We may need to wait for @jeremy to upload his data files to see if its an issue there as well -

Yes, I’ve realized this coords problem too, but can’t solve it yet.
Don’t understand for what we need a random_px_rect() function

Agreed.
The output crops if I comment out that particular part are:

@ramesh - very helpful thanks.

@naman-bhalla same results

Yes, encounter the same error, thanks for the solution, it works without the tfm_y parameter specified.

It is not the solution, I think.
Results of prediction of the fitted net are very poor which is expected because of wrong labels.

Yes, the problem definitely appears to be a result of the bounding box, I modified the random_px_rect function and changed the last two lines from:

Y[rows, cols] = 1
return Y

to

try:
    Y[rows, cols] = 1
except:
    print("Error for Image size: {} bounding box: {:.0f}:{:.0f} , {:.0f}:{:.0f}".format(x.shape, y[0],y[1],y[2],y[3]))
return Y

and got the following results when it runs:

Error for Image size: (224, 298, 3) bounding box: 109:106 , 268:198
Error for Image size: (224, 298, 3) bounding box: 86:287 , 374:499
Error for Image size: (224, 298, 3) bounding box: 208:252 , 374:499
Error for Image size: (224, 282, 3) bounding box: 25:25 , 371:474
Error for Image size: (224, 298, 3) bounding box: 17:163 , 374:499
Error for Image size: (276, 224, 3) bounding box: 68:16 , 465:301
Error for Image size: (224, 298, 3) bounding box: 34:22 , 374:499
Error for Image size: (224, 298, 3) bounding box: 156:131 , 374:480
Error for Image size: (298, 224, 3) bounding box: 383:198 , 482:374
Error for Image size: (224, 298, 3) bounding box: 2:49 , 374:499
Error for Image size: (224, 277, 3) bounding box: 69:60 , 347:499
Error for Image size: (224, 231, 3) bounding box: 104:55 , 382:399
Error for Image size: (224, 298, 3) bounding box: 12:74 , 374:499
Error for Image size: (224, 291, 3) bounding box: 40:35 , 350:442

In the bounding box passed into the function, the bottom/right row exceeds the image rows…

1 Like

if you replace the print statement with “pass” the function will run and your results will correlate pretty well (loss 30.671 to jeremy’s 201.5) results.

1 Like

This is awesome! Thanks a bunch

Hey all!

I was getting this error when using pathlib:
OSError: Error handling image at: data\pascal\VOC2007\JPEGImages\000012.jpg

in the pascal notebook when running:
im = open_image(IMG_PATH/im0_d[FILE_NAME]).

I fixed it by using an f-string.
im = open_image(f'{IMG_PATH/im0_d[FILE_NAME]}')

Hope this helps someone.

If I figure out why pathlib isn’t working properly, I’ll report back.

UPDATE: I updated fastai library to the latest and this fixed my issues.

1 Like

If you are using a remote server like Im where you can’t copy/paste/write directly onto the http-ssh-client then do something like this.

  1. Download the file directly to your computer.
  2. Run python -m SimpleHTTPServer 8000 where you downloaded the sh file
  3. Run ngrok http 8000 (you can get it from ngrok.com)

Download it with a more writable reference to your user wget http://random.ngrok.io/pascal_download.sh, then chmod +x pascal_download.sh and ./pascal_download.sh.

1 Like