Part 3 - What would you like to learn?

Cesare.montresor · October 16, 2018, 7:44am

Hello everyone,

I’m approaching the end of part 2 with my study group and I need to make a plan for what to study next. So here I am crowdsourcing ideas, the question is simple:

If you could make a wish for a “part 3” what architectures/algorithms you would like ?

Here is my list:

Attention-based models
Training of Complex Convnets (one body, many heads)
OpenPose (https://github.com/CMU-Perceptual-Computing-Lab/openpose)
DensePose (http://densepose.org/)
WaveNet ( https://deepmind.com/blog/wavenet-generative-model-raw-audio/ )
Anomaly detection with NN
Times series for prediction of rare events
Syntenic Gradients
Neural Arithmetic Logic Units (https://arxiv.org/abs/1808.00508)
Spike NN with SGD (can’t find back the article)

Thanks

Hugues1965 · October 16, 2018, 4:24pm

Deep reinforcement learning, that would be cool.

Cesare.montresor · October 16, 2018, 6:12pm

hehehe, that’s my idea too and in case I run out of ideas that what I’m going to do. But it’s not a natural extension of DL course, it should be a course by itself as it requires a whole new theory and the game is played at a different level

By the way, check this out:

hwasiti · October 16, 2018, 6:24pm

Nice list!
What is a Complex Convnets (one body, many heads)?

Cesare.montresor · October 16, 2018, 6:47pm

Complex Convnets, is for example in robotics, when you have a single source of images and you want object detection, depthmap, segmentation, etc heads using a single CNN body.
I’ve have been reading that is not so straightforward and if is not done correctly every additional head reduces the overall performance, but instead, if done correctly, every head can reinforce the other.
Something to do with heteroskedasticity

hwasiti · October 16, 2018, 8:34pm

How about Capsule Networks…
Although I think it lost its popularity?

Cesare.montresor · October 16, 2018, 9:12pm

Sure, why not!