Lesson 6 In-Class Discussion


#42

Do I sense a competition with Jeremy again?


(Arvind Nagaraj) #43

How does an Autoencoder differ from a Variational-Autoencoder?


(Louis Guthmann) #44

Would you have any advise for modifying the library on the fly ?


(Ben Eacrett) #45

was reading this earlier today - gives a quick summary:


(Vikrant Behal) #46

When are we expecting part 2?


(yinterian) #47

I would copy what you want to modify to a Jupyter notebook and you can modify it there.


(yinterian) #48

There is no date yet but it should be around march 2018.


(Louis Guthmann) #49

Unfortunately, I had to deep dive in multiple layers of the library and it quickly became a real rabbit hole


(Vikrant Behal) #50

Good. That gives buffer time for ML course. :slight_smile:


(yinterian) #51

What are you trying to do?


(Ankit Goila) #52

I’ve seen TensorFlow using Xavier initialization. How’s it different from PyTorch’s/fastai’s He’s initialization?


(Pavel Surmenok) #53

What does kaiming_normal function do?


(ecdrid) #54

https://keras.io/initializers/


(ecdrid) #55

https://arxiv.org/abs/1502.01852


(yinterian) #56

Initializing the weights.


(Vaidas) #57

One thing that would make easier to read the notebooks is to use names for the parameters that you pass to the class and/or function.
Now it is a bit confusing what the values you pass stands for…


(Louis Guthmann) #58

I tried to debug why my GPU is starving and to implement another loss function. To do so, I had to modify StructuredLearner to take my new loss and starting to look what was under.
I did not manage to make it work for now :slight_smile:


(Arvind Nagaraj) #59

Basically, VAEs try to recreate something ‘similar’ to itself as opposed to the ‘exact thing as the input’


(yinterian) #60

Ask in the forums. To change you loss functions.

Instead of this

class StructuredLearner(Learner):
    def __init__(self, data, models, **kwargs):
        super().__init__(data, models, **kwargs)
        self.crit = F.mse_loss

Use something like this

class StructuredLearner2(Learner):
    def __init__(self, data, models, **kwargs):
        super().__init__(data, models, **kwargs)
        self.crit = # write your looss here

(Louis Guthmann) #61

That’s what I did. Because to use StructuredLearner2 , I also needed to modify ColumnarData. It started to look messy in my notebook.
I’m looking for a more efficient way to do it, some kind of best practice approach to those modification.