Hey Walter - thanks for the commits. I merged them into the master with your latest so no worries.
I finished my work on activations (FTSwishPlus was the winnner) so hopefully that may be of use for us.
I also did some indirect research/reading that may be helpful - there are two recent papers on using new loss functions. The idea is the loss function enforces regularization between the classification categories and that improves both validation accuracy and also generalization:
1 - W-Loss:
2 - Centroid loss:
and specific to medical image segmentation:
And a way to check for ‘memorization’ in the CNN - by monitoring this, you can apparently achieve the optimal stopping point during training:
There is also a new convolution layer proposed by Microsoft Research that is designed to capture better generality:
Finally, I found the github code for a nice GAN setup for super-resolution:
Their paper was very slick - they showed that teaching a GAN to decompose images first, then learn to upscale produced SOTA results vs cycleGAN, etc.
I don’t know if super resolution is something that will help us but at least it’s there should we decided we need to add it into the processing.
I’m going to get the data setup on floydhub and after I finish some deadlines today can hopefully start actively coding/testing tomorrow and will start with Walter’s basic implementation initially, then run it with a simple XResNet for comparison.