Early draft notebooks available

Part 2 & Alumni (2018)
20

I have an error in section
Bbox only

in line
x,y=next(iter(md.val_dl))

The error is
IndexError: index 383 is out of bounds for axis 0 with size 298

I think the reason is tfm_y=TfmType.COORD argument that is passed in tfms_from_model() function, becasue when I comment it, the error gone away.

I did not look at the source code yet, just want to report about a possible bug in the raw notebook.
I will report about solving that problem here if I will succeed.

Here is a link to the full error traceback if somebody interested - https://gist.github.com/anonymous/9140946b47097b0a8ca8bdd65ec09eaa

P. S. Does anybody have this error too?

3 Likes
21

I created a Gist to download pascal data based on @nok’s https://github.com/noklam/fastaipart2v2helpder

21 Likes
22

Had same error the coordinates may specify a region that does not exist in the 224 x 224 augmented image. Not sure if the code attempts to scale the image or coordinates

23

Yes, I also get the error in pascal.ipynb. This error we are seeing could be because of bounding box defined is outside of the image dimensions. Not sure if this is bad data or the bb definition not scaled in transformation. We may need to wait for @jeremy to upload his data files to see if its an issue there as well -

15%20AM
Screen Shot 2018-03-18 at 10.15.15 AM.png1426×1056 135 KB

25

Yes, I’ve realized this coords problem too, but can’t solve it yet.
Don’t understand for what we need a random_px_rect() function

26

Agreed.
The output crops if I comment out that particular part are:

download
download.png829×558 584 KB

27

@ramesh - very helpful thanks.

28

@naman-bhalla same results

29

Yes, encounter the same error, thanks for the solution, it works without the tfm_y parameter specified.

30

It is not the solution, I think.
Results of prediction of the fitted net are very poor which is expected because of wrong labels.

31

Yes, the problem definitely appears to be a result of the bounding box, I modified the random_px_rect function and changed the last two lines from:

Y[rows, cols] = 1
return Y

to

try:
    Y[rows, cols] = 1
except:
    print("Error for Image size: {} bounding box: {:.0f}:{:.0f} , {:.0f}:{:.0f}".format(x.shape, y[0],y[1],y[2],y[3]))
return Y

and got the following results when it runs:

Error for Image size: (224, 298, 3) bounding box: 109:106 , 268:198
Error for Image size: (224, 298, 3) bounding box: 86:287 , 374:499
Error for Image size: (224, 298, 3) bounding box: 208:252 , 374:499
Error for Image size: (224, 282, 3) bounding box: 25:25 , 371:474
Error for Image size: (224, 298, 3) bounding box: 17:163 , 374:499
Error for Image size: (276, 224, 3) bounding box: 68:16 , 465:301
Error for Image size: (224, 298, 3) bounding box: 34:22 , 374:499
Error for Image size: (224, 298, 3) bounding box: 156:131 , 374:480
Error for Image size: (298, 224, 3) bounding box: 383:198 , 482:374
Error for Image size: (224, 298, 3) bounding box: 2:49 , 374:499
Error for Image size: (224, 277, 3) bounding box: 69:60 , 347:499
Error for Image size: (224, 231, 3) bounding box: 104:55 , 382:399
Error for Image size: (224, 298, 3) bounding box: 12:74 , 374:499
Error for Image size: (224, 291, 3) bounding box: 40:35 , 350:442

In the bounding box passed into the function, the bottom/right row exceeds the image rows…

1 Like
32

if you replace the print statement with “pass” the function will run and your results will correlate pretty well (loss 30.671 to jeremy’s 201.5) results.

1 Like
33

This is awesome! Thanks a bunch

34

Hey all!

I was getting this error when using pathlib:
OSError: Error handling image at: data\pascal\VOC2007\JPEGImages\000012.jpg

in the pascal notebook when running:
im = open_image(IMG_PATH/im0_d[FILE_NAME]).

I fixed it by using an f-string.
im = open_image(f'{IMG_PATH/im0_d[FILE_NAME]}')

Hope this helps someone.

If I figure out why pathlib isn’t working properly, I’ll report back.

UPDATE: I updated fastai library to the latest and this fixed my issues.

1 Like
35

If you are using a remote server like Im where you can’t copy/paste/write directly onto the http-ssh-client then do something like this.

  1. Download the file directly to your computer.
  2. Run python -m SimpleHTTPServer 8000 where you downloaded the sh file
  3. Run ngrok http 8000 (you can get it from ngrok.com)

Download it with a more writable reference to your user wget http://random.ngrok.io/pascal_download.sh, then chmod +x pascal_download.sh and ./pascal_download.sh.

1 Like
36

Well, it seems taht there exist TfmType.NO, TfmType.PIXEL and TfmType.COORD, and the previous argument, the only one that show no error is TfmType.NO wich is the default.

For CropType there is RANDOM, CENTER ,NO.

So the line for me is tfms = tfms_from_model(f_model, sz, crop_type=CropType.NO, tfm_y=TfmType.NO)

37

It is actually expected that No won’t show any error as the dimensions of images weren’t affected. But, often it is necessary to get images to same dimension, so I feel it is a library bug that the annotations aren’t being changed so as to reflect change in image.

38

Assuming that’s your own github handle, where did you get / how did you build the json files corresponding to the dataset ?

EDIT: Ah, never mind. I found it mentioned in the external datasets for COCO here. http://cocodataset.org/#external

1 Like
39

Great stuff coming :slight_smile: but is there any chance we’ll see deep learning applied to problem other than computer vision or NLP? I see a lot of papers being released on time series and dealing with ts is what profesional Data scientists do most of the time. I would like to see more stuff like rossmann or what this guy did :no_mouth: .

1 Like
40

Regarding time series and structured data, I don’t really have anything to add beyond what’s in the Machine Learning (preview) course and the Rossmann lessons. Other than the idea of feature generation using an auto-encoder (which may only be useful for anonymized datasets - I’m not sure, but I haven’t seen it applied usefully elsewhere yet) I don’t know what else I’d say!..

4 Likes