Kaggle NLP Competition - Toxic Comment Classification Challenge

~0.5 sounds like you might be shuffling your test set or may have some other problem with creating the submission. Exactly 0.5 would be random