TTA Cannot allocate memory Error

I face the memory error when i do the TTA. I guess memory dsnt gets cleared up after each fold of predictions so soon after completion of first iteration,memory gets full.
Any one has any fix to overcome the issue ?