Deep Learning Certificate Part II plans

A bit more on active learning in CNN’s. I’ve heard that active learning can be integrated into neural nets quite seamlessly.

On top of my head these three methods allow active learning:

  • Detect examples with largest impact on network parameters
  • Committee method
  • Outlier detection in dense layers.

The committee method for active learning can be realized using dropout layers:

https://arxiv.org/abs/1511.06412

Another advantage is that due to the structure of neural nets, one always has access to a high dimensional space to group training instances. i.e. categorical variables are converted automatically into a continuous space in dense layers, allowing outlier detection. And then mixing these observations. Challenging but doable?

Active learning can help both with class imbalance and in selecting examples that need evaluation. It sort of provides a cycle to guide learning if you will.

By the way: thanks for making available the course. I thoroughly enjoyed it.

1 Like

I read this interesting paper on difficulties with active learning this morning http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.188.3020 . I think it’s only really an issue for things like documents and sentences that you can’t see “at a glance”, but it’s still an interesting point.

The active learning with dropout paper is a cool idea - thanks for the link!

+1 for GANs

1 Like

I really like Brendan’s first option. I would like to get my skills to the point where I can implement the latest and greatest.

First of all I gotta say this course is fantastic. I really like the intuitive approach to this topic. It fits how I learn very well.

If your still considering topics for future course,

+1 GAN’s
Restricted Boltzman Machines / DBM’s I think these are the same as GANs.
Hierarchal Coordinate Frames.

Again, fantastic resource here.
Good job.

1 Like

would love to see a lecture on GANs and style transfer.

1 Like

maybe we can get together starting last week of January, @sravya8 ? This is assuming the registration for part 2 goes through, I think it hasn’t opened yet …

1 Like

Is the syllabus up somewhere?

Yes, that works for me. Will message you directly to figure out time and place logistics.

The syllabus is up!
http://www.fast.ai/2017/01/17/curriculum2/

6 Likes

Wow, the second part looks both awesome and challenging, a lot of stuff to digest.

@jeremy @rachel By “Letter” do you mean statement of purpose?.

@jeremy, @rachel I’ll be submitting for an international fellowship. Thank you so much for offering these. I’m extremely interested in the applications of deep learning for structured data and time series as it makes up the majority of data that I work with. I’ve used XGBoost a number of times and I’m very curious to see the comparisons.

In terms of topics, as an artist I’m also very interested in generative deep learning architectures like deep dream and the like and it would be amazing to cover those. I understand the principles but have yet to be able to put it into practice. I’d also really love to see something on ensemble learning and the combining of models, since that seems to be a very common practice among the top kaggle winners. Maybe even a project where you collaborate with another set of students to build an ensemble model?

Thanks so much! I’m really excited for Part 2!!

1 Like

I’m curious for the Reinforcement Learning. :tada: Thank you for including it.

1 Like

Hello Jeremy,

I am interested to learn how i can apply deep learning in my work context ( Telcom)

Example

  1. Time series type of data sets.
  2. How can i apply i context where data is binary format

Thanks,

I see the plans for part II are up - looks great!

One thing that’s not on the list, but would love to see covered in some way: hyperparameter search. It seems that you can use Keras models with scikit-learn? Is that how it’s really done? Are there other ways? Which methods work best, or when to use what?

1 Like

I think it would be super interesting and useful to learn about models that generalize on little data, like Vicarious’ models and the Ilya Sutskever model.

See Vicarious papers:



1 Like

Can we qualify for the international fellowship even if we are not international? (Finishing training as a radiologist in NYC and unable to be in SF for the course.)

@davecg yes - although applications have closed, we can try and squeeze you in if you get your application in today.

I have just started Part 1 two weeks ago. It is SO great! I would be unable to catch up in time for the fellowship. Have a great class and I look forward to the MOOC in May.