Lesson 7 - Official topic

[steef (Steef van Winkel)]

Jeremy’s final words during today’s course: Next week NLP & Computer Vision

[victor.vargas]

Thanks Jeremy and Rachel

[ilovescience (Tanishq)]

Thanks for another great lecture! Interesting to learn more about some “traditional” ML techniques like random forests!

[steef (Steef van Winkel)]

Hahaha - yes I have

Excited it’s coming next week

[MaheshKhatri]

Thanks Jeremy, Rachel & Sylvain.

[ganesh.bhat]

Can someone please point to what entity embedding mean from the last bit of the lesson? Is it the one hot encoding that we discussed or something else?

[username_not_found (unknown)]

Thanks, see you all next week!

[jcatanza (Joseph Catanzarite)]

The enhancements that make Random Forest such a powerful Decision Tree model are:

1. Bootstrap sampling, which is selecting a random subset of the data (i.e. rows in the data table) to construct each decision tree, and
2. Selecting a random subset of the features (i.e columns in the data table) to make a ‘split’ at each ‘node’ in a decision tree.
3. Ensembling, which is contructing a group of models (in this case, a ‘forest’ of trees) and averaging their votes to make each final classification.

The first two of these enhancements are analogous in their application and their effect to the Dropout technique in Neural Networks.

[muellerzr (Zachary Mueller)]

@muellerzr

Everything from attention to self normalization to unsupervised pretraining were attempted or discussed

For those interested in exploring said paper, I’ve opened up a thread here:

Deep Learning for Tabular Data: An Exploratory Study - By Jan Andre Marais

I wanted to bring to light a fantastic thesis by Mr. Marais here. It’s 144 pages long and explores fastai's tabular model in a way I’ve never seen done before. Here is a link to it. I wanted to open up this thread as a discussion for it as it’s well deserved in my opinion. I’ll highlight here important parts/chapters worth noting and exploring. Hopefully those of you interested in the tabular field can join in on the conversation! GitHub: https://github.com/jandremarais/tabularLe…

[geoffpidcock]

Thanks for the lesson Rachel, Jeremy, and team! As Kaggle seems to not have many open competitions using tabular data at present, I’m sharing the SIOP ML Competition as another place to find opportunities in putting chapter 9 through it’s paces!

[zevarela (José Varela)]

Thanks FraPochetti, I really liked your post!

[barnacl (Rekil Prashanth)]

Thanks @FraPochetti

[vijayabhaskar (Vijayabhaskar J)]

Entity encoding is replacing categorical columns with learnable embeddings.

[ganesh.bhat]

In the notebook, the categorical columns were already replaced by numerical values when doing random forest and we used the same for NN. How do we get these values?

[muellerzr (Zachary Mueller)]

@vijayabhaskar

Entity encoding is replacing categorical columns with learnable embeddings

Also for more on that and where it came from @rachel wrote a wonderful article: Redirect

TL;DR? Took inspiration from Word2Vec

[muellerzr (Zachary Mueller)]

@ganesh.bhat

How do we get these values?

They’re relational to the particular cardinality of that variable. An easy example is say we have Jack, Queen, King, and Ace. Now we can’t simply pass that in as they’re not numbers! Instead let’s represent each separate option as an integer number. So we could say Jack = 0, Queen = 1, etc. etc. This scale up is done the same way and is the main method for utilizing categorical data

[vijayabhaskar (Vijayabhaskar J)]

The tabular_learner internally inserts the embeddings layer to the columns which you specified as categorical columns.

[ganesh.bhat]

Thanks @muellerzr and @vijayabhaskar

[zevarela (José Varela)]

Great lesson! Thanks and kudos to all the fast.ai team

[jmete (James Mete)]

Thanks for the lecture! This is probably one of the most practically useful lessons for myself, and I’m guessing a lot of others who work regularly with tabular data.