Explanability of RNNs

There’s been significant progress in the domain of “explanability” of various neural networks. For example, Lasso Regression, LIME, LEMNA and FairML all try to give insight to the decision making process of a trained network.

However, none of these techniques seem to apply to RNNs use for regression or forecasting of time-series. I’m going to do a dive using Semantic Scholar, but I thought I would ask here first.