Feature importance in deep learning

Was the subject of SHAP assumptions about the feature independence discussed?
In tabular data it is quite an issue.

Have any of you read papers behind GradientExplainer and/or DeepExplainer and come to clear conclusion that we can use them without the independence assumption?

In Some Baselines for other Tabular Datasets with fastai2 we slit the topic to model interpretation, so I am bringing the conversation here, to make it easier to find and maybe more people will express their thoughts on the subject.

This is a response to the question How to conduct feature importance without assumption of feature independence?

We can extract FI without assuming feature independence with the attention from TabNet, SHAP explainers which do not use interventions (which might be GradientExplainer - sorry I haven’t checked that yet), @nestorDemeure’s idea or in general methods learning the feature importance during the training. In the future maybe the next update of SHAP will be resistant to the problem, because in the last paper (1911.11888) they describe improvements, but it’s not clear to me.

Any other thoughts?

1 Like