If you want to try LARS, it’s very easy to implement as an optimizer in pytorch (did it in this gist).
14 Likes
Isn’t that what the NVIDIA demo is doing?
Are we using VGG16 n the model? SrResnet seems to build a model from Scratch?
4 Likes
What is a “learnable convolution” and what is an example of a convolution that isn’t learnable?
1 Like
Curious about your context: isn’t what what the NVIDIA thing is doing?
1 Like
Why are we using these little 3x3 squares of every color, instead of using noise in the new pixels?
I understand why we don’t just leave them blank, and maybe why we don’t copy the nearest-neighbors. But why not noise?
3 Likes
Does this mean I can replace
m = nn.DataParallel(m, [0,2])
with something, to get rid of the error below?
RuntimeError: cuda runtime error (10) : invalid device ordinal at /opt/conda/conda-bld/pytorch_1518244421288/work/torch/lib/THC/THCTensorCopy.cu:204
Because then the sequential layers would functionally just be one layer, I think.
1 Like
Yeah you probably want to change the [0,2] to only contain numbers that actually correspond to GPU’s on your computer. Like, maybe [0,1]?
1 Like
Can he explain progressive resizing again? I don’t understand how to use it
1 Like
thanks … but yeah I had tried [0,0] and it didn’t help; [0,1] didn’t either.
I wonder how to find out what the correct values would be!!
Huh… I wonder if using load state_dict(strict=False) would work as a quick way to load weights from a pretrained model. Say: pretrained keras/tensflow retinanet, if you more/less match the architecture in pytorch.
1 Like
Also, can we use progressive resizing to match the idea of backbone + head?
Is that a checkerboard pattern on the bluejay?
3 Likes
where?
on the neck and head area of the bluejay it seemed to checkerboard.
1 Like
@rachel if there is a good time to fit this in, I’m curious if Jeremy has an explanation.
If not, no biggie!
1 Like
DONT DO A PhD. I was enrolled for two years and dropped out. More opportunities outside without a PhD.
3 Likes
Or at the very least if you do one, take everything your professors tell you with a grain of salt (they have no idea what goes on in the real world). And do as many industry internships / work part time as you can.
I have a PhD, and worked for 2 startups during my degrees. There’s still a definitely opportunity cost to spend that many years of your life. But if you go in with the understanding that your responsible for also ensuring your industry success afterwards, you can leverage it into some interesting skills beyond your specific subject area.
3 Likes