Is Octavio’s video publicly available? (if so, link please <3)
5 Likes
I don’t think so, It was unlisted.
where’s the Octavio video? Is it available? Tried searching and couldn’t find it.
3 Likes
@yinterian
How do we arrive at the filters?
Are these optimised as well (via gradient descent or other methods?)
1 Like
Yes, the filters are optimized with Stochastic Gradient Descent (SGD) or a version of it.
2 Likes
I have a Keras implementation that looks a lot more like the fast ai lib. forum thread is here and github here.
Right now it gets about 98.5% acc on dogs and cats. I am still looking for the places where fastai (pytorch) and my lib differ to get it up to 99%
6 Likes
If @jeremy would have 3channel input in his excel spreadsheet, then the filter in first hidden layer would have dimension [3,3,3]?
Why using a simple sum of previous convolutions instead of weighted one?
Intuitively every layer captures either edges, shapes of the image. As information passes through these kernels and activations, isn’t there progressive information loss? Is information preserved or does this not matter?
2 Likes
Are there any situations where a non-square maxpool might be useful?
1 Like
@zaoyang It is not so much loss than the data is compressed. Deep Learning is about reducing the noice inside the data and getting its essential characteristics.
Their is a lot of loss happening in the max pooling layer. This is something people do not like especially Hinton (one of the godfather). New research is coming in to solve that.
But we keep using them because it just works in practice.
4 Likes
Can we please pick our own seats next time? I can’t see with where my group is placed.
It’s actually weighted. It’s the 3x3 subset coming from your layer weighted by your filter. It’s the weighted sum. (was that your question?)
Regarding the spreadsheet - I have a different set of values (see below) in filters 1/1 and 2/1, and then in the conv2 layer, I guess the spreadsheet might be out of sync?
Any good resources for the fully convolutional layers?
@jeremy you’ve briefly mentioned that you used network pre-trained on 3-channel input on a 4-channel input (RGB+NIR), by adding additional dimension to all pertained filters (initialized to 0 or perhaps random). Did you then ‘unfreeze’ these additional new filter weights along with the ones for RGB channels and train them?
7 Likes
We’re getting sound
Hm ok thanks guys.
Got it. I feel like my video breaks every time we go on break, lol