(Jeremy Howard) #41

Yes it is. I believe we should do np.exp before we take the mean. Or use a different kind of mean.

Well spotted!

This function is defined in dataset.py (lower end)

``````def split_by_idx(idxs, *a):
``````

What happens to that *a parameter when itās called as such for example(from column_data.py)?

``````def from_data_frame(cls, path, val_idxs, df, y, cat_flds, bs, test_df=None):
((val_df, trn_df), (val_y, trn_y)) = **split_by_idx(val_idxs, df, y)**
return cls.from_data_frames(path, trn_df, val_df, trn_y, val_y, cat_flds, bs, test_df=test_df)
``````

is `*a` equivalent to a list of `[df, y]`

(Jeremy Howard) #43

(Anup) #45

This is precious !! Thanks a ton !!

(Santhanam Elumalai) #46

@jeremy
I got this code from fast.ai library āfitā function

avg_loss = avg_loss * avg_mom + loss * (1-avg_mom)
debias_loss = avg_loss / (1 - avg_mom**batch_num)

I figured it out it (avg_loss) is linear interpolation, but what it exactly does and why we are using it? Also what is debias_loss how it differs from regular loss response?

(khan) #47

Some more Urlās I found useful reading the code.

https://www.programcreek.com/python/example/4684/collections.Iterable

(Jeremy Howard) #48

Itās an exponentially weighted moving average, to make the reported loss more stable.

Take a look at the Adam optimizer paper to learn about debiasing in this way.

(Santhanam Elumalai) #49

https://arxiv.org/abs/1412.6980 Is this the paper you referring to ??

(Jeremy Howard) #50

Yes.

(Santhanam Elumalai) #51

Gotcha !!

Is it possible to explain this two line, just like the RELU, dropout, and back-prob?

(Santhanam Elumalai) #52

In fast.ai, this.opt or layer_opt implies optimizer and layer optimizer, but donāt know why I am kind of reading it as option and layer options Very tough to agree with the optimizer why canāt we have it as optim instead of opt

(Jeremy Howard) #54

Seems reasonable - PRs welcome (but please grep the full source to ensure all usages are updated, including notebooks!)

(Santhanam Elumalai) #55

Changing variable name is fine for me, but testing functionality after name change is the critical part. How can I validate it?

(Santhanam Elumalai) #56

@jeremy is this below code is just to map the respective lrs and wds to the layers ??
params = list(zip(self.layer_groups,self.lrs,self.wds)) Interesting way of avoiding nested loops !

(Santhanam Elumalai) #57

Any reason why we have `def split_by_idxs(seq, idxs): last, sl = 0, len(seq) for idx in idxs: yield seq[last:idx] last = idx yield seq[last:]` yield , Since it is has been converted into a list on the receiver side, any other advantage in considering yield ?