I joined the “Digit Recognizer” competition at Kaggle after reading all the chapters (1-6) on image recognition in the book by Jeremy. I thought I achieved great on the first try(trained only 3 epochs and scored 0.981): see my “victory” here!, and then I ventured to make some changes to my code such as normalization and label smoothing.
Then the weird thing happened: the performance of the model suddenly decreased rapidly, and the accuracy dropped to about 60% even after 4 epochs of fine-tuning. I tried to reverse the changes, and found that even if returning to the original code that I had written, the model failed to perform well.
The current notebook at here: https://www.kaggle.com/code/jasoncoderjia/submission-for-mnist
I’m exhausted. How did this happen?