To test the model of lesson 1 I chose images of pickleball and tennis. If you’re not familiar with either sport, it might be difficult even for a human to differentiate between images of these two sports. I had at first only 5 images of both sports in train and valid. The result after running 5 epochs (running more epochs did not improve the results) with a learning rate of 0.01 was an accuracy of 90%-100% (the accuracy varied on different runs with the same parameters), where the probability for images of tennis was in all cases correctly close to 1. The results for pickleball images were a lot more random, though, with one image incorrectly classified as tennis and others with probabilities between 0.03 and 0.3 correctly classified as pickleball.
To try to improve the accuracy of the model in recognizing pictures of pickleball I added 10 more pickleball pictures to the training data. Somewhat disappointingly, this did not improve the the overall probabilies of the validation images being correctly classified as pickleball. The model produced accuracies between 0.7 and 1 after running it for 5 epochs with the same parameters and the probabilities of validation images being classified as pickleball did not really improve, they just varied somewhat between different runs, with similar results as before.
I then tried a learning rate of 0.001 and ran the model several times for 10 epochs, but that only resulted in generally worse probabilities for images as being classified correctly.
The only major difference I noticed in all of these different runs was which image out of the 5 validation images of pickleball was incorrectly classified as tennis. In all runs it was one or the other of the same two pickleball images, but never both. I can easily understand why these images were difficult to classify. In one image there were only 2 pickleball players on the court (a singles match) while the images of tennis were also all singles matches with only 2 players on the court. In these runs the model seemed to have learned that 2 players on the court implies that tennis is being played.
The other image which was on other runs incorrectly classifies as tennis was an animated image with a view as seen slightly from above the court. All of my images of tennis were TV images which were all recorded from a camera which provided a view from above the court, so it seems to me that in these runs the model was able to learn that a horizontal view angle meant pickleball and a slightly vertical meant tennis.