Another treat! Early access to Intro To Machine Learning videos


(Kieran) #720

Use the train_cats() function which takes your data frame and changes the categorical variables to data type (dtype) category. Then when we call proc_df() these categorical variables will be changed to numeric variables. Its important that they all have the same spelling etc!

Check out 7:58 on Machine Learning 1 lesson 2 video.

Hope that helps


(Kieran) #721

My guess is that its one of these on this scholar search.
Problem is they tend to be uber technical and in general you have to pay for them.

https://scholar.google.ca/scholar?hl=en&as_sdt=0%2C5&as_ylo=2016&as_vis=1&q=training+with+imbalanced+data+set+oversampling&btnG=

Let me know if you find anything interesting :slight_smile:


#722

Hey, I’m also facing the very same issue. My kernel gets restarted when trying to run df, y, nas = proc_df(df_raw, ‘SalePrice’). This issue is seen with a dataframe size of 100. Where you able to resolve this issue?


(Sanyam Bhutani) #723

Cross Posting here for visibility:

I’ll host weekly discussions starting on the 12th of August. (ML MOOC)

Please take a second to vote for timings if you’re interested .

Sanyam


#724

Additionally to those, I had to install:

  • pip install isoweek
  • pip install pandas-summary

To find out which dependencies you might be lacking, it is useful to start an interactive python session just by typing “python” on the command line prompt and then running
from fastai.imports import *
at the python “>>>” prompt until you get no errors. Any errors there should show the missing dependencies.


(Axel Straminsky) #725

In one of the lectures Jeremy showed a library for interpreting random forests, but if I remember correctly he said that he didn’t know of a library that did the same for Neural Nets. A few days ago I came across a new library called SHAP, that apparently is not only for interpreting RF, but any ML model. Has anyone tried it?

Repo: https://github.com/slundberg/shap


(sashank) #726

can anyone help me on this ?


(Kevin Bird) #727

I am listening to lesson 5 and I am not sure I understand the extrapolation section. So you try to predict your validation set records (in my case, I have a holdout set) Then you take the feature importance of those and try to drop each of them and run the model like that. At that point I would expect you to keep the columns that would make the score worse if it weren’t in the model and drop anything that makes it better, but Age which when dropped doesn’t hurt but you still keep it in. Why is this not also dropped? I have tried implementing this in a real world scenario and I am not getting any of my columns that are making the model better when they shouldn’t, but when I predict a previous month and remove all the data after that point, I get fairly decent results, but when I try to predict the following month, I am not getting as good of results. I suspect data leakage of some sort, but I haven’t tracked it down yet.


(Kevin Bird) #728

I have used it and from what I can tell it does a pretty good job. I don’t think it is directly interpreting the model though so it is taking a simplified version of the model to make it’s assumptions I think. So sometimes the Feature importance list from the model will be different from the SHAP library. Overall I definitely think it has a lot of potential. I’ve been using it with XGBoost with pretty decent success.


(sashank) #729

Yes the issue is with creating the feather file . Not sure of internal issue but i stopped creating feather file and it solved the issued for me


(SA) #730

i do not have any ML experience. Should I watch these ML lectures or do DL1 & 2. Is this ML course complete? like are all the materials on the jupyter notebook or is it in a state of progress? I saw there are only 5 notebooks in the ML course repository.
Also, i have almost completed the first lecture. should i be knowing all the details of scikit,pandas etc.?
I have done a bit of matplotlib. My problem with libraries is that I keep forgetting the module/library specific commands bcoz there are so many of them. Similarly, in the lecture, there are a lot of attributes,dot notations, but how does one remember all that. Documentation is there but how will I use features if I don’t know/remember them at the right time?
Also, after watching 1 lecture how much time should I dedicate to go through the notebook/kaggle datasets before watching other lectures?

I am thinking of going through this MOOC to gain better understanding of pandas,scikit ,data cleaning etc. (https://courses.edx.org/courses/course-v1:UCSanDiegoX+DSE200x+1T2018/course/)
Is that needed or should I just go through the notebook?


#731

I’m using the paperspace fastai server template, but for lesson 1 the data folder isn’t there. And I haven’t been able to get the data downloaded using the methods explained in the videos (I can’t get the correct link).
If someone can help me out with the correct link to use, or an alternate way of getting the data there it would be really helpful!


(Sumit) #732

Hey @spock,

First of all, don’t worry just start doesn’t matter whether you know everything or know nothing.

Here I’ll quote as @jeremy & @rachel said learn things on as needed basis, don't try and learn everything that you might need first otherwise you'll never get around learning the stuff you actually want to learn

Although I had a little bit of ML exp, still I went through ML videos first and it helped me a lot.

Yes, it is.

Don’t worry most of us face the same issue. But in time you’ll create your own style of recalling/finding of whatever you required.

One more thing, I believe in the community here, I and others are here to help.

Cheers !!
Sumit


(SA) #733

Thanks. The reason i was along whether ML course is complete or nott is because it had only 5 Notebooks.
Also, is doing the ML part necessary or can I skip to DL 1,2 because before ML was launched people were doing DL 1,2 in the beginning by default, right? I guess doing ML part provides better foundation for DL 1,2?
Also, how much time should I dedicate to each lecture,and apart from reading the notebooks , experimentation what else are we supposed to do? When should i assume that my 1st lecture material is complete ?


(Ali) #734

Thank you for the link


(Kyle Nesgood) #735

Just throwing in my two cents, but I don’t consider this a “must-do” course. As an example, I listened to all of the DL course 1 lectures straight through before going back and dissecting each lesson. When I went back, I started with whatever lesson I was interested in, listened for a while / took flashcards, then used it to refine something I had been working on (a Kaggle competition, solo research, etc.).

Each of us will have a unique style that teaches ourselves “the best”. Just jump in and enjoy the ride - don’t become the roadblock to your own enjoyment of the material!


(SA) #736

Thanks


(Sumit) #737

Hi,

Can anyone help me to resolve this issue.

Thanks !!


(Gerardo Garcia) #738

Are you in the fastai folder?
Did you activate fastai?

Follow the steps here
FASTAI


(Sumit) #740

Hi @gerardo,

I’m running it in my local computer. Yes, I have followed the steps which are mentioned.
But still i’m facing the same issue.

Thanks,
Sumit