About TTA overfitting problem

Would it be a problem of overfitting if we even train our dataset on the validation set with data augmentation. Because if we do that our model is just memorizing the whole dataset and just because of that we are seperating validation and testing set .

Please correct me if iam wrong.