Another advantage is that due to the structure of neural nets, one always has access to a high dimensional space to group training instances. i.e. categorical variables are converted automatically into a continuous space in dense layers, allowing outlier detection. And then mixing these observations. Challenging but doable?
Active learning can help both with class imbalance and in selecting examples that need evaluation. It sort of provides a cycle to guide learning if you will.
By the way: thanks for making available the course. I thoroughly enjoyed it.
I read this interesting paper on difficulties with active learning this morning http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.188.3020 . I think it’s only really an issue for things like documents and sentences that you can’t see “at a glance”, but it’s still an interesting point.
The active learning with dropout paper is a cool idea - thanks for the link!
maybe we can get together starting last week of January, @sravya8 ? This is assuming the registration for part 2 goes through, I think it hasn’t opened yet …
@jeremy, @rachel I’ll be submitting for an international fellowship. Thank you so much for offering these. I’m extremely interested in the applications of deep learning for structured data and time series as it makes up the majority of data that I work with. I’ve used XGBoost a number of times and I’m very curious to see the comparisons.
In terms of topics, as an artist I’m also very interested in generative deep learning architectures like deep dream and the like and it would be amazing to cover those. I understand the principles but have yet to be able to put it into practice. I’d also really love to see something on ensemble learning and the combining of models, since that seems to be a very common practice among the top kaggle winners. Maybe even a project where you collaborate with another set of students to build an ensemble model?
One thing that’s not on the list, but would love to see covered in some way: hyperparameter search. It seems that you can use Keras models with scikit-learn? Is that how it’s really done? Are there other ways? Which methods work best, or when to use what?
I think it would be super interesting and useful to learn about models that generalize on little data, like Vicarious’ models and the Ilya Sutskever model.
Can we qualify for the international fellowship even if we are not international? (Finishing training as a radiologist in NYC and unable to be in SF for the course.)
I have just started Part 1 two weeks ago. It is SO great! I would be unable to catch up in time for the fellowship. Have a great class and I look forward to the MOOC in May.
Thank you for including it.