Lesson 6 In-Class Discussion ✅

Is the bias value also picked at random in the start and updated through the training or does it remain same all through? If the latter, then how do we determine what the bias value should be?

2 Likes

My bad, by switch I meant to make models in Pytorch directly without using fastai

Right, but to submit you have to use a Kaggle kernel. Did you submit one? and if so, I would love to see (I think they are all public, just let me know the uRL) Thanks!

opps not really without weather information you get stuck in lesson 6 notebook “experimenting with a sample”.

Anyone knows how to get the additional data supporting rossmann competition? store_states, state_names, googletrend, weather

Is it good or bad to apply our own filters to the image before we send it to the network? i.e. gaussian filters

Additional note on that (since that rule is stupid): pretraining your model takes a night on a p3 if you want, or a day on another hardware. I’ve posted a notebook recently that shows a pretraining a QRNN, but if you remove all the options in language_model_learner (that are there for the QRNN), you can replicate the pretraining of WT103.

12 Likes

Once we unfreeze and train a CNN more, are the kernels weights changed too? I mean, are the kernels learnt or pre-designed in Fast AI?

what is making some portions of image bright and dark during conv

I didn’t use a kernel. I just used the dataset to practice lesson 3. I used full pretraining and all that. Consequently I didn’t submit to the competition.

1 Like

It’s bad if you use a pretrained model, as it won’t know them.

Are the kernels learnt or is it a set that is decided for different aspects like vertical edges and such ?

1 Like

The kernel weights are the thing you are training when you call fit (or fit_one_cycle).

2 Likes

Are you referring to this viz: http://setosa.io/ev/image-kernels/

It may be helpful to try it out yourself some to see how the values are calculated (it is easier to read up close)

1 Like

Got the answer

We take the exponentially weighted average of the gradients at all the update steps.

1 Like

How do you decide how many kernels to use ?

1 Like

You can still get benefit from fastai helpers like create_cnn when making custom Pytorch models, see the resnet50_basic_datablocks notebook in my Protein Atlas competition repo here: https://github.com/wdhorton/protein-atlas-fastai

3 Likes

Another hyper-parameter to tune, although if you’re using a pretrained model, you have to use their numbers.

1 Like

How does one kernel combine the 3 inputs into 1 output that is 5x5? Is it just the average of the red, blue, green input multiplied against the weights in the kernel?

3 Likes

How does a network handle images of different sizes?

2 Likes