In the Dynamic UNET, why are we creating the middle conv and unet block in eval mode?

Here is the code I am referencing, inside the init method of the dynamic unet, we have this two sections:

middle_conv = nn.Sequential(ConvLayer(ni, ni*2, act_cls=act_cls, norm_type=norm_type, **kwargs),
                                    ConvLayer(ni*2, ni, act_cls=act_cls, norm_type=norm_type, **kwargs)).eval()


unet_block = UnetBlock(up_in_c, x_in_c, self.sfs[i], final_div=not_final, blur=do_blur, self_attention=sa,
                                   act_cls=act_cls, init=init, norm_type=norm_type, **kwargs).eval()

So I was digging through the code and was confused by this. Why are we calling the .eval() method after creating them?
Don’t we want them in training mode?

I believe this is because dummy data is being passed through the model at the time it is initialized to determine input/output shapes the layers and calling .eval() prevents any updates to batch norm stats during this process.

1 Like

Hello :wave:
That seems like a plausible reason. However I can’t see the code that switches it back to training mode. Or will this happen automatically during the training loop?

I am assuming training on eval mode will not enable us to update the batchnorm which is undesirable behavior.

I believe that switch is automated during the training loop because eval mode should be turned on as well when your validation set is evaluated every epoch.

1 Like

Awesome. Thanks for the help!