Lesson 6 LSTM recurrence dropout

Deep Learning
1

Hi all, in lesson 6 we talked about lstm cell.In keras there seems to be dropout option on both non-recurrence and recurrence paths.But according to this paper https://arxiv.org/pdf/1409.2329.pdf, they suggest to apply dropout only on non-recurrence paths as it may make lstm to forget/remove vital information otherwise.Any justification as to why Keras included recurrence Dropout ?

Thank You

2

Good question! Keras’ implementation is based on this (later) paper: https://arxiv.org/abs/1512.05287

1 Like