Deep Learning on structured data

(Rahul Raj) #1

A noob question, but may I ask how a deep learning model is better than classic machine learning model in the case of structured data?


universality theorem


Lesson 2 at 01:46:51 Universal Approximation Theorem

(Rahul Raj) #4

Thanks for the response.
But how does universal theorem answers my question? It just talks about how a neural network can solve any function.

(Paul) #5

afaik, deep learning approach can be used to find hidden dependencies that couldn’t be found via machine learning approach


RFs are universal function approximators as well.

Porto Seguro (kaggle competition) 1st place solution description is an interesting read on this subject :slight_smile: Seems much of the feature engineering can be automated away via DNNs.

But there is more. With NNs you can use embeddings - they are a way of inferring the meaning behind a categorical variable and encoding what can be learned in a high dimensional space. This sentence probably doesn’t do embeddings justice and by meaning I only refer to it in the context of what is in the data. There is a great paper on this by Geoffrey Hinton and in the context of structured data you can read more about this in a paper on a solution to the Kaggle Rossman competition.

I am providing the above information as a reference but I do not want to make a general statement as to DNNs being better or worse than other ML techniques for structured data. I think this will be very situation / dataset / requirements dependent. For practical applications, one also has to consider the availability of tools and ease with which they can be applied to the data. Getting a good enough answer in 30 minutes vs getting a slightly better one but after 4 days of hacking on a problem might not necessarily be worth it in certain contexts.