This is an interesting publication by S. Merity about a single headed attention LSTM with promising results for such a small model:
„Single Headed Attention RNN: Stop Thinking With Your Head“
4 Likes
This is an interesting publication by S. Merity about a single headed attention LSTM with promising results for such a small model:
„Single Headed Attention RNN: Stop Thinking With Your Head“