Can anyone give me a practical example of this? When we want to start creating random forest from scratch. How do we find this split?
This thing is very confusing to me. Video at that time: https://youtu.be/blyXCk4sgEg?t=2534
How do we decide the variable to what variables we split on? Either we should split by price, area of house. What should be the split on?
What does he mean by trying every single variable? What did he compare MSE to? How did he decide he will split on Coupler System?
I am not exactly sure if I am right or not but first of all we need to split the data into two parts so the algorithm look at every variable or feature and its all possible values and decide the value at which we can split it into two parts. And for the bulldozer it happens to be the coupler system.
The computer are very fast now a days so checking every variable and its possible values is not a problem.
For other dataset for eg some house data set .it may split it first on the base of price, area or any other feature.
To understand this, first you need to know that a simplest model in ML using “average” as prediction. So if I have 10 rows I can take the average of the rows and use it as prediction.
RF works by trying every column in the dataset. And even within the column it will every possible split. Then it takes the average of the split and calculates the training error. The split with the least error will be taken as the correct split.
Lets take a dataset with only one row - Coupler System. RF will try to split based on each data point. Coupler System > 1 and Coupler System > 2, so and so forth. So for the split Coupler System > 1 it will split the data, which let us assume gives us a dataset made of 15 and 20 rows respectively. Now for all the 15 rows, it will use the average of the 15 rows as prediction and then for the rest of 20 rows it will use average of those rows. Both of these are models in their own right and have their own MSE.
RF will take the weighted average error which is calculated as - (MSE of the 20 row model * 20) + (MSE of the 15 row model * 15)
RF will then repeat the process for next split - Coupler System > 2. Let’s say the split gives us 10 rows and 25 rows. The weighted average error will be taken again. So on and so forth
Out of all the possible splits the one which has the least weighted average error is taken as the correct split.
Jeremy does talk about the weight calculation here: