Project: hand keypoints detection

Hi fastai people,

I tried using fastai v1 in my project, this is how it looks like at the moment

Want to share with you some points that were missing in the library for me

  1. In case ImagePoints (which is a label) is out of range [-1;1] after transformation (on data augmentation step) it is just discarded and an error is raised at some later stage by dimensions mismatch. Instead, I think the correct thing to do here is to try generating an image again. Moreover, at least in my project, I regenerated image again also in cases where the transformed points were too close to the edges. I wanted my points to be in range [-0.95;0.95].

  2. Recent cutout transformation is great, but doesn’t work (crash) for points labels when tfm_y=True.

  3. More transformations are needed, specifically beyond affine and perspective transformations, with efficient implementations, can be really helpful

  4. Is there automatic tracking of losses and metrics, and running times, across the whole training process? Can be great to have it with some plotting utilities.

Thank you,

For 1. you can normally pass the flag remove_out=False to keep the points that disappear after transformation.
For 2. cutout needs a coord implementation (like pad or crop have) to be able to work on ImagePoints (any PR doing that would be welcome)
Same for 3, we are completely open to PRs with more transforms
As for 4. everything (except the time) is stored in learn.recorder which has a few plotting utilities.

Hi Sylvain, thanks for the prompt answer.

Regarding point 4, isn’t it only the last cycle recording?
For example learn.recorder.plot_losses() plots only the last cycle, is there a different function for the whole training? (or may be this is only me running training 40 cycles, and it is designed to run only once?)

Point 1, thanks for mentioning this parameter, I didn’t know it existed. But besides that, I guess the desired functionality boils down to a callback which gets transformed image and label and can say “no, please generate another one instead, I don’t like what you have just generated”.

It’s only recording the pas training you’re right. If you want to do multiple cycles and record them all together, you should design it with TrainingPhase and the GeneralScheduler callback, so that it’s all done in one call to fit, otherwise it’s true the Recorder state is reset at each new training.