Yeah, I’ve read that before. 
Was particularly wondering if he had any tips in the context of deep learning - there aren’t as many references on that.
imblearn is useful and has SMOTE etc, but haven’t been able to use those techniques with imbalanced data successfully with neural networks.
I’ve mainly oversampled with augmentation and undersampled so far, but wanted to know if he had any additional tips.
I’m having some trouble figuring out where to focus across the spectrum of Torch, Keras, Tensorflow, Theano, mxnet. Could you discuss the pro’s and con’s you see in each and your expectations for where they will be in a year? five years?
Will Google look to build out Keras more? Make TensorFlow more intuitive? Will corporate sponsorship of languages become more common?
How about we use stock trading data for TimeSeries? Essentially trying to predict the price of stocks. Hopefully we will end part 2 with more money than we started
Any best practices around Neural networks for structured datasets ?
What is the best way to go from intermediate programmer to advanced?
I’ve started seriously programming in python around a year ago, and I’ve feel I’ve reached some kind of a glass-ceiling - I know all basic and some advanced concepts, I know how to write reasonable code that works with OOP and all, but i don’t know how to take a step forward an doing correct programming architecture, and using more advanced concepts. what do you suggest, except practicing?
What is the future of machine learning chips?
In the last year, Nvidia stock tripled it’s value. it seems there is currently no competition in the GPU field for nvidia. what is the reason other firms c an’t catch up and make GPU of themselves? what do you think about the future of this field? are GPUs will continue ruling, or there will be other domain specific processors like googles tpu?
How to extract topics from long text?
How can we parse or predict things on long text such as articles, e.g topics, sentiment, keywords, intentions etc? in images, we can easily trim, resize etc. but text is not easy to auto summarize…
See:
If you consider a GPU, it has a lot of unnecessary hardware for graphics processing that deep learning doesn’t need. However the gaming industry has funded (and keeps funding) GPU innovation which is why we can dual-purpose GPUs for DL.
It’s clear though that the trajectory of DL will far outpace gaming. It’s only a matter of time before we see DL-specific chips w/o GPU-specific components.
You mentioned, that you might not be able to cover segmentation. That is a topic which interests me very much, because most publications on biomedical imaging deal with segmentation (see review).
I will investigate on my own anyways, but with your guidance I’d be 100x faster. If segmentation won’t make it into part 2 I’d love to see it in part 3, which I for sure hope is going to happen.
Very nice and clear. If you could add to that a code-centric example that would be much appreciated. How about an example using keras.js or similar?
Jeremy mentioned in the last class that Kaggle practitioners historically used U-Net, but have started to switch to DenseNet:
U-Net (https://www.kaggle.com/c/data-science-bowl-2017#tutorial)
DenseNet (https://github.com/SimJeg/FC-DenseNet)
Thanks for mentioning it. I definitely am looking at these. U-Net was a break-through in biomedical image segmentation.
@jeremy No I hadn’t. Thank you for sharing. Must say, a very clear framework to think around. I especially liked the inclusion of levers as the second step. We (the startup I work at) often tend to go from objective to data (3rd step).
I somewhat disagree with the name of this approach - “drivetrain”. The name indicates a step by step process. However we have often found circling back and iterating on previous steps or even the entire process altogether to produce desirable results.
Q1. Are the simulator and optimizer pieces essentially cross validation testing of the model ensembles? Or is there something more that I am missing.
Q2. Does this approach work equally well when you are building products for external users that may not be data/analytics centric? I believe steps 1 and step 2 become much harder.
This is covered in https://keras.io/getting-started/functional-api-guide/ , and also part 1 of the course. Let me know if you have any further questions after you’ve taken a look.
I’m not sure I agree with this statement, at least in the near term; the major graphics card developers are focused on general purpose gpu computing and have been for over a decade now and are very aware of the deep learning market. The amount of extra hardware isn’t that extraneous and amortizing the cost between all the people who want gaming hardware and all the people who want to do ML substantially drives down the cost.
Custom ASIC deep learning chips may outperform GPUs but they’ll be several orders of magnitude more expensive, and I think that will keep them out of the market for a long time. If anything I could see GPU manufacturers adding hardware specific to deep learning before we see custom deep learning chips. The other approach could be programmable logic (FPGAs) but they haven’t taken off to the level of GPUs so I don’t see that happening either.
Intel is getting into the DL chip space. We’ll see how fast they ramp up their chip development+production.
Cool! It’ll be interesting to see if it trickles down to small scale deep learning usage or if it will be limited to big companies like google. I wasn’t aware that Intel was moving into that space, although it makes sense that they are.
I ran across this paper this morning and thought you would appreciate it.
https://arxiv.org/abs/1508.03422v3
Haven’t read it yet, but it seemed relevant.
@jeremy after your course ends, what do you recommend as next steps for those interested in pursuing a career in deep learning?