Any Ideas on how to modify Resnet with Images that have 2 channels?

Part 1 (2018)
1

Hi,

After having the first course yesterday I want to replicate the notebook and try it on a different dataset to both challenge myself and have some fun.

So there is a Kaggle competition going on right on link here: https://www.kaggle.com/statoil

Most of the kernels I’ve come across are defining their own CNN classes which seems to be the right way to go with this competition since image data consists of 2 channels (1 - HH band image, 2 - HV band image) .

My question is this: Would there be a way to hack Resnet in order to allow to train it for this problem. These might sound stupid since I am fairly new to DL but some of the ideas I am came up with are:

  • Adding a 3rd channel which will be equal for each image. This way the kernels won’t be affected by the 3rd channel while they are being updated and the net might only focus on our original 2 channels.

img = (np.array(train['band_1'][0]) + np.array(train['band_2'][0])).reshape(75, 75) + np.zeros((75, 75))

  • Redefining the architecture to allow 2 channel images. This seemed really hard to me :smiley: At least what I’ve understand from complex ResNet paper is that many layer’s depend on the residuals of the previous later. So only updating first convolutional layer to let’s say to (75, 75, 2) won’t do it.

Thanks !

6 Likes
2

Hi @kcturgutlu,

It’s great to see that you are already looking at other datasets that you could apply what you learnt in Lesson 1. For this competition from statoil, you may not be able to use Resnet. The dataset dimensions are only 75x75x2 and Resnet requires a minimum size of 197 on width and height along it being a 3D (RGB) Tensor. So, I would suggest you -

  1. Choose a different competition in Kaggle that you might be able to directly apply what you learnt even if it’s a kaggle competition that’s not currently running. Fisheries is a good one - https://www.kaggle.com/c/the-nature-conservancy-fisheries-monitoring
  2. Build a CNN from scratch using fastai library. If you poke around the examples in the repo, I am sure you will find some sample code to get started.

Good luck and best wishes from your fellow learner.

4 Likes
3

I would also recommend trying your had at Dog Breed Classification Kaggle Competition - https://www.kaggle.com/c/dog-breed-identification. It might be closest to the lesson 1 - cats vs dogs.

3 Likes
4

Thanks ramesh

5

How about convert the images to RGB and resize to 197px on width?

1 Like
6

I followed the same steps in the notebook after creating dirs data / train valid test from images and it works with size 75 :smiley:

  • I basically summed up vertical and horizontal images as it is suppose to give composite color images.
  • Then saved them as the fastai library requires in jpg format using 70-30 % startified sampling.
2 Likes
7

Cool! So… how did it go?

8

I need to lower the batch size I guess, since there is not enough iterations to see the learning rate go down and rise again. I tried sz but I guess it’s the image size. So far with no tuning ~0.73 on validation.

9

lr_find
lr_find.png853×810 70.6 KB

10

You need the bs param when creating your data object.

11

Ok thanks!

12

What is the default for bs ?

13

shift-tab will show you :slight_smile:

14

Right :slight_smile: I decreased my batch size but now it’s giving lower accuracy in validation set should I just experiment with different bs - lr combinations ? Thanks

15

Yup pretty much. Although based on that LR chart you show above it look like a tiny dataset?

16

Yes, 1122 train 482 val (70-30) and 8500 test

17

Oh yeah that is small. You’ll have to play around with hyperparams to see what works best… You may find resnet18 works better for such a small dataset, and may need more dropout (which we’ll learn about later).

4 Likes
18

Thanks so much for the tips, I will try my best and submit so far what I can do. Then will wait for the next classes!

19

Let us know how it goes. Perhaps show a Kaggle kernel with your work once it’s going ok?

20

Yes sure, can we share jupyter notebooks here ? I guess Tim did it before right ?