I’m a bit confused about this concept. On ML lesson 1 jeremy explains that curse of dimensionality is a stupid concept. But after looking at lesson 3 where @jeremy shows that removing columns actually improve our models by removing some cardinality I got lost.
I thought about this curse of dimensionality as follow (which is now obviously flawed):
The more columns/features you add to your dataset, the better it is as these new “meta-data” won’t have any negative impact on the model predictive performances. The price to pay being to need more compute power to process the data.
But now I can clearly see from lesson 3 that having too many features that doesn’t matter much relative to the dependent variable actually reduces models (or at least RF) predictive power. So what should I think about it? Was I completely wrong about the curse of dimensionality meaning? Should I care about this only for RF?