Deep Learning on structured data


(Rahul Raj) #1

A noob question, but may I ask how a deep learning model is better than classic machine learning model in the case of structured data?


#2

universality theorem


#3

Lesson 2 at 01:46:51 Universal Approximation Theorem


(Rahul Raj) #4

Thanks for the response.
But how does universal theorem answers my question? It just talks about how a neural network can solve any function.


(Paul) #5

afaik, deep learning approach can be used to find hidden dependencies that couldn’t be found via machine learning approach


#6

RFs are universal function approximators as well.

Porto Seguro (kaggle competition) 1st place solution description is an interesting read on this subject :slight_smile: Seems much of the feature engineering can be automated away via DNNs.

But there is more. With NNs you can use embeddings - they are a way of inferring the meaning behind a categorical variable and encoding what can be learned in a high dimensional space. This sentence probably doesn’t do embeddings justice and by meaning I only refer to it in the context of what is in the data. There is a great paper on this by Geoffrey Hinton and in the context of structured data you can read more about this in a paper on a solution to the Kaggle Rossman competition.

I am providing the above information as a reference but I do not want to make a general statement as to DNNs being better or worse than other ML techniques for structured data. I think this will be very situation / dataset / requirements dependent. For practical applications, one also has to consider the availability of tools and ease with which they can be applied to the data. Getting a good enough answer in 30 minutes vs getting a slightly better one but after 4 days of hacking on a problem might not necessarily be worth it in certain contexts.