BIWI Facial landmark dataset

Hi everybody,
Recently I thought I’d work on the biwi face landmark dataset, and try to create a video version that keeps track of the face.
However after training, I found that model generalized extremely poorly to images of me. Mostly it predicted the centre or a little off centre, If I had my head tilted towards the edge of the image, the model completely missed my face. I was training only a resnet18 as I did not have a big enough gpu.

Is this only becuase of the small model? or is there some other issue? I followed the fastai 2020 version of the lectures. Both train loss and valid loss were extremely low.

Hi Akash, looks like a good idea for a fastai project! Generalization of models is indeed an important topic, and generalization problems may be caused by many factors. I don’t think there’s enough info provided here to respond to your question - maybe if you share a notebook with your analysis, it would be easier to give you some hints. Good luck!

Hi darek,
Here’s a link to my training notebook. I Used the BIWI dataset.

https://drive.google.com/file/d/1Sug59PYZy8nvmvMYEpegBMDKDIIgHeM8/view?usp=sharing

Do let me know what you think.

Try removing your aug_transforms. Doing regression with transforms can be very buggy and do a “center clustering”, where it will essentially just point to the center of your image regardless. And transforms seem to be the cause.

2 Likes

So you’re saying even with just resnet18 I should get good inference results? I will definitely try it.
I was initially worried that the network trained on BIWI wouldn’t generalise well as biwi contains very few “NEW” faces. Most faces are repeated. And I thought new faces could be out of domain. What do you think about that? Could the minimal faces seen during training affect the inference on my face?

Resnet18 is one of my first go-to architectures for point regression, no I don’t believe it will have a problem with that.

I mean that’s also a potential, how different is your face from theirs? But I would expect it to be veering in some degree towards the center of yours, as NN’s are good generalizers.

First though definitly try modifying the transforms to just a Resize and Normalize and see where that gets you

Hi @muellerzr,
I have removed aug trasnforms, however i still had to resize as my gpu couldn’t handle full sized images. So i’m using only resize. Also I am brown (indian) and see that there are no faces in the dataset that represent this. Most faces (23 if i recall correctly) are white and 1 is asian. The model still generalizes extremely poor to my face.

Also could you expand a bit more on center clustering?

here is the link to updated experiments.

Thanks