Any Ideas on how to modify Resnet with Images that have 2 channels?

Part 1 (2018)
76

Thanks for your answers. Reducing the batch size indeed made the plot looking more meaningful.

1 Like
77

That’s v cool @sermakarevich! Try resnext101_64 too. And then take the average of that and inception

1 Like
78

BTW as part of fixing resnext et al, I tested resnext50 on dogs v cats, and got 99.75%! We’ll look at how to replicate this result on Monday :slight_smile:

4 Likes
79

@kcturgutlu, so what was causing the issue? (Im having exactly same problem, even after setting test name)

80

I blended inception_v4 with my previous model which was NN on top of extracted features from Keras inception and xception with pseudo labelling. So its like a blend of fastai part 1 v1 and part 1 v2 :slight_smile:

image
image.png2142×312 52.4 KB

UPDT: summary:

model loss accuracy
resnext101_64 0.23 93
inception_v4 0.19 94
(inception_v4 + resnext101_64) / 2 0.183 ~
(inception_v4 + keras inception + keras xception) / 3 0.17 ~
4 Likes
81

In this file all transformations are defined in

here are the definition of basic transforms

transforms_basic    = [RandomRotateXY(10), RandomLightingXY(0.05, 0.05)]
transforms_side_on  = transforms_basic + [RandomFlipXY()]
transforms_top_down = transforms_basic + [RandomDihedralXY()]

You can easily make your own list of transforms with your own parameters.

Here is an example:

tfms = tfms_from_model(f_model, sz, aug_tfms=transforms_basic,
                       max_zoom=1.05)
5 Likes
82

Wow top 10! :smiley:

83

Two top 10 places left :slight_smile: - all need to do is to train inception_v4 with 4 epochs.

84

Hi @yinterian,

Deleting tmp directory magically made the predict(is_test=True) to run without errors, but I cannot quite figure out why. Could you explain it for us?

Thank you!

1 Like
85

@yinterian I am also unable to predict using your code, get error:

TypeError: ‘NoneType’ object is not iterable

After reading what worked for @hiromi have also deleted the folder in data/dogscats/temp and tried afterwards but no luck.

Apart of setting correct test name in ImageClassifierData.from_paths( … test_name = “correct_test_name” )
and deleting temp folder after training, is there any obvious thing to do?

86

In my case the same error was caused by corrupted preprocessed validation/test set. Restarting notebook did not help. I changed image size a bit and create data object once again.

1 Like
87

The tmp file holds the precomputed activations. Most likely the first time you ran your code you didn’t use a test folder. Therefore you had precomputed activations for train and validations. The second time you ran it with a test folder but the library didn’t realized (there is a TODO) that it had to compute the test activations so it didn’t find the test activations gave you an error.

Does it make sense?

4 Likes
88

Hello,

Any idea where I can find good amount of satellite data which I can use to train a whole network then fine tune as in transfer learning for iceberg challenge.

Thanks :smiley:

89

This paper (https://arxiv.org/pdf/1704.02965.pdf) has suggestions for aerial imagery benchmark datasets. They also suggest generating datasets from the Google Maps API.

My approach for this competition might include using a Generative Adversarial Network to generate more training data.

1 Like
90

It does make sense. Thank you so much for clarifying! :slight_smile:

91

Yes, the kaggle Planet Amazon comp is a great source of data. We’ll look at it today. Although it’s rainforest, not ocean!

1 Like
92

@sermakarevich Great work! I’m also working on this comp and my accuracy levels looks good on train/valid sets but when I run the predictions with test_preds = learn.predict(is_test=True) the values I get are all negative. My original assumption was that the value closest to 0 (i.e. np.argmax) was the the one with the highest probability. But, for softmax I’m used to the values being probabilities between 0 and 1 (with probs for all possible classes adding up to 1) and I think that is the format that Kaggle is looking for so just wondering how did you get the probabilities?

93

@jamesrequa those are logg ods, so first you need to do is
df = pd.DataFrame(np.exp(preds))

3 Likes
94

And always check data.classes before submitting, it’s ordering alphabetically I assume using sorted function. So, iceberg will be 0 and ships will be 1. Another thing to keep on mind. Hope this also helps.

95

@sermakarevich I also started working on dog breed competition but I am unable to get lesser than 0.55 loss on validation set(~2000 images). As of now, I have only tried resnet34. But it seems that validation loss has saturated and stopped reducing beyond 0.55. I started with learning rate of 0.1 based on lr_find and gradually moved to cyclical rate with data augmentation and unfrozen earlier layers. Earlier I trained the network with just 2-3 epochs and stopped as I could see training and validation loss diverging. But to test the capability of model, I ran for 10 epochs and could see the training loss reduced to 0.09 – sign of clear overfitting. The network so far has achieved 82.6 % accuracy on validation set. What was the accuracy that you could achieve using any of the resnet models?

1 Like