Thinking about this further, if you generate embeddings for the words in the dataset for your different models, and plot them with a dimensionality reduction technique, we should see decent clustering in both cases. I think this is probably most true for simple problems like sentiment classification were there are only a couple classes, and we know what types of words are positive and negative. So your embedding approach might also work.
Yes… That’s exactly what I am trying to prove…
I got feedback that I am giving out dense of information, therefore I am trying to compare things in the domain they are most comfortable with…
As you mentioned, in the simple 0,1 case, we probably just see a very decent clustering distribution which might not extend to any further use cases… But if I can use it to let people start to accept what I said might be worth a try?
Explain things to non-technical people is what I have heard a lot… and clearly I failed last Friday being asked what is AWD_LSTM, AWD stands for… (Now I know…)