[Asking a question for homework 3]
I find a validation accuracy more informative and easier to interpret than a log loss, from Kaggle. I understand that this would only apply to classification-type competitions, but is it possible to ask Kaggle to for that result rather than a log loss?
If the competition has loss metric I am not sure you can get it directly, but if you really want it and can spare the submissions to get it you can try this:
Submit fixed/polarized predictions, for example 0.99 and 0.01 respectively for each row.
Now taking the loss from Kaggle you should be able to calculate the accuracy.
Something like this: If there are m test-set examples:
loss = -1/m(correct * log(0.99) + (m-correct)*log(0.01))
Substituting loss and m should let you calculate the ‘correct’ variable and your accuracy is correct/m