In many of the Inception networks, the models were trained with auxiliary classifiers, where small CNNs were inserted in between layers for training time, and the loss from these auxiliary classifiers were added to the main classification loss.
How could we implement something like that in Fastai. I would need to add a couple conv. layers and get their output to calculate auxiliary loss and update them during training time. So effectively I need a multi-output classifier, no? How would I be able to do this?
You can create a custom_loss class which can take all your outputs from the model return the final loss by calculating individual losses separately. I am working on a similar thing right now with YoloV3 gives 3 different outputs from different CNN layers.
How can you return different outputs from different parts of the CNN though? I understand that you can have a custom loss, and it is trivial to have multiple outputs if they are coming from the same layer, but if you have multiple layers, I am unsure how this would work.
That’s the part of the architecture you will create. In the forward method just return all outputs coming form your different CNN layers. You need to break your CNN architecture into smaller sub-networks to do so.
Your custom loss function will handle it. Fastai will send all the outputs from your model to your loss forward method and you need to handle them. Link
Something like:
class CustomLoss(nn.Module):
## your __init__ and code...
def forward(self, output, target1, target2...):
# incase you have 3 outputs
out1, out2, out3 = output[0], output[1], output[2]
## code to calculate your loss...
The Auxiliary Classifier GAN, or AC-GAN for short, is an extension of the conditional GAN that changes the discriminator to predict the class label of a given image rather than receive it as input.