Test time augmentation results in higher error

Well was implement an ConvNeXt on a shoes classification model
without TTA , the error was TensorBase(0.0760)

However after using TTA the error was TensorBase(0.0800)

What might be the reason ? and should I now proceed with TTA being implemented with the model?