DeepLearning-Lec8-Notes

After working through the week’s notebook ones after going through my notes, and then a second pass along with your notes always helps in bolstering the conceptual understanding.

@timlee really a big thanks to you from us all for compiling and explaining the lessons again in such detail! :smile: :ok_hand:

Thanks for the notes @timlee. As usual, it’s detailed and useful.

Thanks for the feedback. I’ve updated the text according!

It’s all greek to me.

Posted this here because I’m not sure where the wiki is for this lesson. But anyway ,here’s a pretty good link for both Greek and Mathematical notation.
https://www.rapidtables.com/math/symbols/Basic_Math_Symbols.html

You can edit the ‘in class discussion’ post.

did that, should I withdraw the post?

No need - it’s nice to have both I think.

Thank you for putting it together @timlee! That’s very helpful.

Does anyone know where predict_batch is defined?

It’s used in these notes where the model is tested under “Largest item classifier” in

probs = F.softmax(predict_batch(learn.model, x), -1)

I tried jupyter notebook’s shift-tab and ?predict_batch, as well as googling and also just trying grep -R for it in the fastai dir in bash, but no luck.

1 Like

Its a function in fastai.model. Try out using VS Code as Jeremy was showing in the lecture, very useful for digging around in the source code.

You probably haven’t updated(git pull) fastai repo.
Also, easily searchable on Github. https://github.com/fastai/fastai/search?utf8=✓&q=predict_batch&type=

3 Likes

You can try this out !

1 Like

In the following code from the pascal.ipnb notebook,

trn_anno = collections.defaultdict(lambda:[])
for o in trn_j[ANNOTATIONS]:
    if not o['ignore']:
        bb = o[BBOX]
        bb = np.array([bb[1], bb[0], bb[3]+bb[1]-1, bb[2]+bb[0]-1])
        trn_anno[o[IMG_ID]].append((bb,o[CAT_ID]))
        
len(trn_anno)

Instead of using collections.defaultdict(lambda:[]) expression in the first line, we could have used collections.defaultdict(list). It would have the same behavior - When each key is encountered for the first time, it is not already in the mapping; so an entry is automatically created using the default_factory function which returns an empty list.

Thank you @timlee ! Could you turn the post in a wiki so I can copy the markdown, please? Thank you in advance! :slight_smile:

Done!

1 Like

I already have a lot of images annotated in the old Pascal/VOC XML-format, after some searching I found this way to convert into JSON (I started to build it my self until I realized that someone must have done it already):

I could’t find anywhere on the forum how to do the translation and Jermey only referencing to a already translated version:

Remember to change the ID’s because they gets prefixed based on your filenames, and will give you errors because the defaultdict approach expect the ID’s to increment from 1 (and not a prefix).

3 Likes

I have one question for according to the course, now I wanted to train the customized image sets for beverage brands, how I can do the annotation to get the same PASCAL VOC data formats? I used labelimg and labelme to do the annotation, I have tried searched on the google but can’t find a solution, anyone can give me the similar case or advice would be helpful, thanks!

hi Punnerud, I just saw what u posted, so use that script can transform the datasets to the PASCAL VOC json file and can use the same as the course 8 tutorial one right?

Thats is right.

I recommend doing it on the original files first to have a working example, then on your own annotated output afterward. If I remember correctly there is some fields you have to fix. It was a small “search and replace” before running the code. Two other folks at my team also figured out the same with no help, so I think you fill figure it out.

1 Like

thanks @punnerud I have already use this script to get the json file and run the original files first to get understand what’s going on and then played around with my customized image set, still get one matrix bug ValueError: Expected more than 1 value per channel when training, got input size [1, 1024], so I need change my training data size and figure out how to solve this.