In-Place Activated BatchNorm - 50% less GPU memory usage

Hi!

Has anyone tried to use this approach with fast.ai library yet?
Paper: https://arxiv.org/abs/1712.02616
PyTorch implementation: https://github.com/mapillary/inplace_abn

@jeremy do you think it might be useful?
I remember people having memory issues with language model but it doesn’t use BN anyway.

2 Likes