A walk with fastai2 - Vision - Study Group and Online Lectures Megathread

fastai
1183

Thanks @muellerzr

1184

Hi,

How can I approach a keypoint regression problem, when the number of points is not the same in every image? (in fact, there could be 0 points of interest).

Watching the tutorial on keypoint regression from this forum made me think that a possible way to do it would be to encode the output “no point” as a constant (-1,-1) and use as output layer 2 times the maximum number of points found in an image from the training set. However, that does not seem a very promising approach.

Does anyone have any ideas/examples of how to better do this?

Best!

1185

There’s models like HRNet and other pose detection models that could be good. Main issue is figuring out how to build the databunch (something I tried and failed on doing as I brought up briefly in the course). Essentially each point becomes a heatmap that you can then regress for. But our standard models won’t cut it for that as we need a maximum number of outputs wanted for our final linear layer.

1186

Interesting! You mention regressing heatmaps…that made me think that maybe I could transform the problem into a segmentation, where the ys are black pixels everywhere, except for white pixels in the points of interest. Does that make sense? On the other hand, are there any implementations of pose detectors in fastai2 that you are aware of?

Thanks!!!

1187

Besides the one I tried, no. If you search around v2 vision I asked Sylvain for a lot of help setting up the data. I can try to find it again but I’m afraid I don’t have time to revisit it myself, as much as I would have liked to.

While it does, Unets are super expensive and have a long inference time (compared to say a standard resnet34). But you can try it and tell us what happens :slight_smile:

2 Likes
1188

Thanks! Btw, I plan it to use it for an upcoming competition that looks quite interesting for the use of AI in space engineering!
https://kelvins.esa.int/spot-the-geo-satellites/

1189

How can I specify in which device the inference is done with learner.predict?

1190

@WaterKnight

How can I specify in which device the inference is done with learner.predict ?

I believe this is based on learner.dls.device.

1 Like
1191

It makes sense. I will give it a try and tell you if it worked.

Thank you @msivanes

1193

Can someone please explain me what is the meaning of ’ L ’ in the line
‘’‘split_list.append(L(range(len(train_imgs), len(train_imgs)+len(tst_imgs))))’’’, which is present in lesson-3 cross-validation notebook?
Maybe its a stupid question to ask, but I’m not getting any answers on Google.
Thanks.

1194

L comes from fastcore. It’s a special kind of list that better utilizes Python. To learn more about that specifically I’d recommend jeremy’s source code walkthroughs

1 Like
1195

Yes I got it. Thanks for the help.

1196

Any advice on whether I should start with fastai-v3 or instead start with this course and wait for fastai-v4? Thanks in advance! :slight_smile:

1197

I’d recommend both if possible :slight_smile: v3 shows the ideas more in depth, and my course is a jump into the v2 code with some of the background knowledge Jeremy touches on. It’s not as in depth as his, but I tried to cover (most) of the same grounds (minus NLP) and then with more examples than is shown in the regular course.

Also you can do v3 and work with fastai2, there’s a course section in the fastai2 repo with each notebook recreated with fastai2

2 Likes
1198

Thanks so much for your advice! I would like to do both as well, but I’m just wondering which one to start with?

1199

Personally I’d start with Jeremy’s course for the foundations of how it all works as a whole, then mine to see how to integrate it in the new library :slight_smile:

1 Like
1200

Quick question about Colab: for training models, is it recommended to work with data in the root directory or can we work with data on a mounted Drive? The mounted Drive approach seems to be much slower, but I’m just wondering what the recommended approach is? Thanks! :slight_smile:

1201

I’d recommend moving your items from the drive to the main (copy it) while training, but yes it will be considerably slower. It used to not be but Colab adjusted it’s security preferences so now it takes longer

1 Like
1202

Loss functions in PyTorch have an ignore_index which is convenient if you’ve got samples in the dataset whose labels need to be ignored. It’s unclear to me if this functionality can be used with CategoryBlock in the Data Blocks API.

Imagine a Multi-Modal scenario like the 06_Multimodal_Head_and_Kaggle.ipynb notebook but where not all samples are labelled. For example:

	image_id	gr_root	   vowel_di	   consonant_di  	grapheme
==================================================================
0	Train_0	       15		   9	       -100	            ক্ট্রো
1	Train_1	       159		  -100	         0	            হ
2	Train_2	       22		   3	         5	            খ্রী
3	Train_3	       -100		   2	         2	            র্টি
4	Train_4	       71		   9	         5	            থ্রো

-100 is different from an NA label, because I’d like the loss function to ignore it during computation i.e. return a loss of zero if the index is -100. (By default, ignore_index=-100 in PyTorch loss functions).

Is this possible to do with the Data Blocks API?

Thanks!

1203

You’d do this inside the loss function :slight_smile: Simply pass in your ignore_index to CrossEntropyLossFlat as you would any loss function, then pass this loss_func to Learner, cnn_learner, etc (This is separate from the DataBlock API)