A walk with fastai2 - Vision - Study Group and Online Lectures Megathread

fastai
610

Here is a paper that presents another technique for comparison of dl models and interpretation.
Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability

3 Likes
611

So, heatmaps was not where I wanted it to be by this point, so Iā€™m subbing in a different lecture that week while we discuss GANā€™s

612

Thanks!

613

Thanks !

614

Awesome @foobar8675 !! I will take a look at it later today. Big bounty!! :moneybag:

615

How should I deal with multimodal data? Iā€™m having one block as image and Iā€™ve two text parameters, is this doable using DataBlock or I should check Pipeline?

616

Is anybody else having trouble to watch the last recording (lesson 5) on youtube? It may be because I am in a public network (namely starbucks) I just would like to double-check. :wink:

617

splits = (self.splitter or RandomSplitter())(items)
so all our splitters return a function _inner which is applied to items and outputs the splits. We need to be subsetting these splits.
So we need a function (subsetSplitter) that wraps these pre defined splitter functions (which return a function) and perform some operation on the splits.

The idea of wrapping makes sense to me but i have no clue how to do it :slight_smile:

618

Havenā€™t done it yet, sorry :sweat_smile: week turned busier than anticipated. Iā€™ll steam it today, Iā€™ll have more time.

2 Likes
619

it is not yet uploaded i think @mgloria

620

@muellerzr if possible could you post here what time if you plan on live streaming, will try to join :slight_smile:

621

Sure. Letā€™s rock with 3pm CST. Iā€™ll post the streaming link around 2

2 Likes
622

Link should still be the same:

2 Likes
623

Weā€™re live :slight_smile:

1 Like
624

Thanks everyone who came and watched :slight_smile: I hope that some of you more familiar with the technique can provide more input (@lgvaz) at where I may have mispoke/gotten wrong. Otherwise a list of resources in which I mentioned today:

Lucasā€™s Repository for style transfer: https://github.com/lgvaz/projects/tree/master/vision/style

Residual Blocks: https://arxiv.org/abs/1512.03385
Upsample ConvLayer: http://distill.pub/2016/deconv-checkerboard/

The new fastai2 paper:
fast.ai
arxiv

Jeremyā€™s lecture on style transfer: https://www.youtube.com/watch?v=xXXiC4YRGrQ (starts ~59:14)

2 Likes
625

I noticed that your networks that use residual connections just add the input in the forward function. I just finished reading the fastai paper before I watched your video and learnt that fastai provides an nn.SequentialEx and MergeLayer for these kind of cases. It might be useful to use that when defining ResBlocks or other related blocks in your code :slight_smile:

2 Likes
626

@ilovescience great idea! Could you show an example of a refactored ResBlock like in the lesson using these? :slight_smile: this would also be applicable (I think) to the layers we built for the style transfer too

1 Like
627

Hey so I havenā€™t played too much with fastai2ā€™s Layer API but I think it should be possible to replace this:

class ResBlock(Module):
  def __init__(self, nf):
    self.conv1 = ConvLayer(nf, nf)
    self.conv2 = ConvLayer(nf, nf)
  
  def forward(self, x): return x + self.conv2(self.conv1(x))

with this:

def ResBlock(nf):
    return SequentialEx(ConvLayer(nf, nf),
                                     ConvLayer(nf, nf),
                                     MergeLayer(dense=False))

or:

class ResBlock(nn.Module):
    def __init__(self,nf):
        self.convpath = SequentialEx(ConvLayer(nf, nf),
                                         ConvLayer(nf, nf),
                                         MergeLayer(dense=False))
    def forward(self,x):
       return self.convpath(x)

If dense=False then itā€™s a residual connection but if dense=True then itā€™s a dense connection. IIRC I think SequentialEx will deal with giving the original input to MergeLayer.

Note I havenā€™t tried this out yet, just wrote this up based on the fastai codebase. So it is very much possible that I got this wrong and this doesnā€™t work.

But the two examples of usage in the codebase are here and here. Surprisingly, these are the only two places where they are used, which I find slightly odd, because I would expect it to be used for the actual ResBlocks in the codebase.

Nevertheless, I hope this helps!

5 Likes
628

Iā€™m going to change up the schedule here on the tail end of vision, mostly due to what I can manage. There will be another technique to the pose/image regression/bbox lesson next week (not heatmaps) and then the following week weā€™ll do a quick walkthrough on audio and GAN. This will be technique focused not theory focused for that particular lesson, then weā€™ll begin on tabular :slight_smile:

(Notebooks will still be their full end to end)

3 Likes
629

@muellerzr iā€™m not sure the Lookahead approach would work, since it inherits from the class Optimizer. the splitters, on the other hand, do not have a class hierarchy, ie, RandomSplitter, IndexSplitter, ā€¦ are functions.

am I missing something?