Deep Learning on structured data

Part 1 (2018) Beginner (2018)
1

A noob question, but may I ask how a deep learning model is better than classic machine learning model in the case of structured data?

2 Likes
2

universality theorem

3

Lesson 2 at 01:46:51 Universal Approximation Theorem

4

Thanks for the response.
But how does universal theorem answers my question? It just talks about how a neural network can solve any function.

5

afaik, deep learning approach can be used to find hidden dependencies that couldn’t be found via machine learning approach

1 Like
6

RFs are universal function approximators as well.

Porto Seguro (kaggle competition) 1st place solution description is an interesting read on this subject :slight_smile: Seems much of the feature engineering can be automated away via DNNs.

But there is more. With NNs you can use embeddings - they are a way of inferring the meaning behind a categorical variable and encoding what can be learned in a high dimensional space. This sentence probably doesn’t do embeddings justice and by meaning I only refer to it in the context of what is in the data. There is a great paper on this by Geoffrey Hinton and in the context of structured data you can read more about this in a paper on a solution to the Kaggle Rossman competition.

I am providing the above information as a reference but I do not want to make a general statement as to DNNs being better or worse than other ML techniques for structured data. I think this will be very situation / dataset / requirements dependent. For practical applications, one also has to consider the availability of tools and ease with which they can be applied to the data. Getting a good enough answer in 30 minutes vs getting a slightly better one but after 4 days of hacking on a problem might not necessarily be worth it in certain contexts.

7 Likes