Code reading

(Jeremy Howard) #41

Yes it is. I believe we should do np.exp before we take the mean. Or use a different kind of mean.

Well spotted!

(Aditya) #42

This function is defined in (lower end)

def split_by_idx(idxs, *a):
    mask = np.zeros(len(a[0]),dtype=bool)
    mask[np.array(idxs)] = True
    return [(o[mask],o[~mask]) for o in a]

What happens to that *a parameter when it’s called as such for example(from

def from_data_frame(cls, path, val_idxs, df, y, cat_flds, bs, test_df=None):
        ((val_df, trn_df), (val_y, trn_y)) = **split_by_idx(val_idxs, df, y)**
        return cls.from_data_frames(path, trn_df, val_df, trn_y, val_y, cat_flds, bs, test_df=test_df)

is *a equivalent to a list of [df, y]

(Jeremy Howard) #43

This is described here:

(Anup) #45

This is precious !! Thanks a ton !!

(Santhanam Elumalai) #46

I got this code from library “fit” function

avg_loss = avg_loss * avg_mom + loss * (1-avg_mom)
debias_loss = avg_loss / (1 - avg_mom**batch_num)

I figured it out it (avg_loss) is linear interpolation, but what it exactly does and why we are using it? Also what is debias_loss how it differs from regular loss response?

(khan) #47

Some more Url’s I found useful reading the code.

(Jeremy Howard) #48

It’s an exponentially weighted moving average, to make the reported loss more stable.

Take a look at the Adam optimizer paper to learn about debiasing in this way.

(Santhanam Elumalai) #49 Is this the paper you referring to ??

(Jeremy Howard) #50

Yes. :slight_smile:

(Santhanam Elumalai) #51

Gotcha !!

Is it possible to explain this two line, just like the RELU, dropout, and back-prob?

(Santhanam Elumalai) #52

In, this.opt or layer_opt implies optimizer and layer optimizer, but don’t know why I am kind of reading it as option and layer options :blush: Very tough to agree with the optimizer :slight_smile: why can’t we have it as optim instead of opt

(Jeremy Howard) #54

Seems reasonable - PRs welcome (but please grep the full source to ensure all usages are updated, including notebooks!) :slight_smile:

(Santhanam Elumalai) #55

Changing variable name is fine for me, but testing functionality after name change is the critical part. How can I validate it?

(Santhanam Elumalai) #56

@jeremy is this below code is just to map the respective lrs and wds to the layers ??
params = list(zip(self.layer_groups,self.lrs,self.wds)) Interesting way of avoiding nested loops !

(Santhanam Elumalai) #57

Any reason why we have def split_by_idxs(seq, idxs): last, sl = 0, len(seq) for idx in idxs: yield seq[last:idx] last = idx yield seq[last:] yield , Since it is has been converted into a list on the receiver side, any other advantage in considering yield ?