Nasnet.py changes

amritv · January 2, 2018, 12:43am

In nasnet.py the download url brings up a 404 for downloading the nasnet pretrained model.

pretrained_settings = {
    'nasnetalarge': {
        'imagenet': {
            'url': 'http://webia.lip6.fr/~cadene/Downloads/pretrained-models.pytorch/nasnetalarge-dc8c1432.pth',

There was an update made and the new url is:

 'url': 'http://data.lip6.fr/cadene/pretrainedmodels/nasnetalarge-a1897284.pth',

github.com

Cadene/pretrained-models.pytorch/blob/master/pretrainedmodels/models/nasnet.py

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.utils.model_zoo as model_zoo
from torch.autograd import Variable

pretrained_settings = {
    'nasnetalarge': {
        'imagenet': {
            'url': 'http://data.lip6.fr/cadene/pretrainedmodels/nasnetalarge-a1897284.pth',
            'input_space': 'RGB',
            'input_size': [3, 331, 331], # resize 354
            'input_range': [0, 1],
            'mean': [0.5, 0.5, 0.5],
            'std': [0.5, 0.5, 0.5],
            'num_classes': 1000
        },
        'imagenet+background': {
            'url': 'http://data.lip6.fr/cadene/pretrainedmodels/nasnetalarge-a1897284.pth',
            'input_space': 'RGB',

This file has been truncated. show original

farlion · February 1, 2018, 10:54pm

Nice catch!

However, when I update the URL in nasnet.py, I get the following error:

Ideas?

amritv · February 1, 2018, 11:50pm

Hey @farlion not at a computer but looking at the error I wanted to check, there have been a few changes to the fastai libraries a few days ago Including conv_learner, looks like you are using the older versions, if so can you update the library and then re try.

YJP · February 2, 2018, 5:14am

Hi, @farlion you might want to find “self.linear = nn.Linear(4032, self.num_classes)” on line 548 in nasnet.py and modify it as follow:

Also, if you happen to copy the file from Cadene github, you might want to remove the code highlighted below because you would get another error (output size is too small because of pooling).

farlion · February 3, 2018, 12:23am

Thank you both for your quick replies, you rock!

Unfortunately, haven’t gotten it running yet.
Fastai lib (from github repo) is at latest stage, and I’m using the default Paperspace setup with up-to-date conda env.

When I try the modification suggested by @YJP, I get the following:

Whereas when I replace nasnet.py with the Cadene github version and delete the line you suggest, I get

YJP · February 3, 2018, 1:48am

Hello,

I am not sure

The way I tried was using fastai nasnet.py as a base then just copied and pasted ‘def nasnetlarge’ function only from the Cadene github version (In this version, they do not have the first argument; only ). Then I changed ‘self.linear = nn.Linear(4032, self.num_classes)’ to 'self.last_linear = nn.Linear(4032, self.num_classes).

When you use the Cadene github version, please also check two places with the variable ‘num_classes’ in nasnet.py below:

I am not sure whether this is the appropriate way to resolve this issue but at least I found the model started to work in this way. Hope this works for you.

farlion · February 3, 2018, 12:18pm

Hmm thank you @YJP, I tried your approach, giving me the same error.
I then added the use_classifer=False param back, which got the model to download (with the new URLs from Cadene), but then it fails again with:

Shall we take this to github?

amritv · February 3, 2018, 7:25pm

@farlion
I was able to get the same errors that you and @YJP got. I did the following:
used this version of nasnet.py: https://gist.github.com/asvcode/65cf2bd5e0fa85f8af84457df6e2dda8
I now get a different error:

RuntimeError: Given input size: (4032x7x7). Calculated output size: (4032x-3x-3). Output size is too small at /opt/conda/conda-bld/pytorch_1503965122592/work/torch/lib/THCUNN/generic/SpatialAveragePooling.cu:63

Appreciate any thoughts on this

YJP · February 4, 2018, 12:04am

Good morning,

@amritv and @farlion

I haven’t seen @farlion’s error yet. Just to make sure:

Copy the whole code from fast.ai nasnet.py (69ffda8 Nov 19, 2017) to the current jupyter notebook nasnet.py.
Copy ‘url’: ‘http://data.lip6.fr/cadene/pretrainedmodels/nasnetalarge-a1897284.pth’ from Cadene’s nasnet.py (795f371 on Dec 19, 2017) to the current jupyter notebook.
Copy the code section for ‘def nasnetalarge’ from Cadene’s nasnet.py to the current jupyter notebook version.
Change num_classes from 1001 to 1000.
Change self.linear to self.last_linear

I started to train the model and will see how it goes in terms of accuracy and loss. I think num_classes is the number of classes/classification that we want to predict so this may have to be adjusted once the model started to work. I will experiment to see whether adjusting this number makes difference.

Hope this works.

farlion · February 4, 2018, 12:55am

Good late night

Thanks for the detailed steps @YJP, I reproduced them exactly, and still get

@amritv No idea yet, sorry. Sounds like it’s becoming time for a deeper dive

amritv · February 4, 2018, 6:31am

@YJP
removing that line of code generates this error:

AttributeError: 'NASNetALarge' object has no attribute 'avg_pool'

Not sure how your code works without that line.

YJP · February 4, 2018, 7:44am

Hi @amritv

I think your nasnet.py is the version from Cadene’s github, which has the different code from fastai version.

Line 582 on your version has “def logits(self, features):” which includes “x = self.avg_pool(x)”.
For the same position in fastai version, it has the following code, which does not contain “self.avg_pool”:

Please refer to my previous post (my base is a fastai version) though farlion tried and still has a problem that I cannot replicate.

jeremy · February 4, 2018, 7:46am

@yjp thanks for your help with this. Perhaps you could post a gist with your version? And if you’re finding it’s working correctly, maybe even a PR?

YJP · February 4, 2018, 7:51am

Hi @jeremy,

I made it start to work but I am still training to see whether this model generates an appropriate accuracy and loss first (it seems this takes for a while).

Sorry, but I am not used to all these terms but what is a PR in this context? Pull Request? Thank you.

jeremy · February 4, 2018, 7:52am

That’s right. If you haven’t made one before, try hub. There’s some great posts linked from forum threads here with walkthoughs.

YJP · February 4, 2018, 8:11am

Hi @amritv,

This version is what I am currently trying to train:

https://github.com/YJAJ/fastai_trial/blob/master/nasnet.py

I tried Jeremy’s nasnet.ipynb on dogs vs cats and it seems to work:

amritv · February 4, 2018, 3:41pm

@YJP, you are right I was using Cadene’s nasnet.py, I switched to the version you stated and now the model is training

amritv · February 4, 2018, 3:52pm

I have another question, as Nasnet large is slow and computationally expensive, do you think playing around with the cell sizes will help reduce computational costs?

I also wanted to confirm the in-channels_left and out_channels_left, are these image sizes?

self.cell_17 = NormalCell(in_channels_left=4032, out_channels_left=672, in_channels_right=4032, out_channels_right=672)

YJP · February 4, 2018, 11:30pm

@amritv
I am glad it worked for you.

Hello @farlion,

Are you still having an issue with nasnet.py? If so, could you please try the version I posted in the github and let me know whether it works or not? Thank you in advance.

farlion · February 5, 2018, 9:49pm

@YJP unfortunately, yes

Running on default paperspace, with your exact version of nasnet.py

My notebook: https://gist.github.com/workflow/b943bc9b7364a6b9dcfc56f60212a2b4

Latest fast.ai git commit is

commit 9d8e49a8f0afaedaf8fe53f8e1a94261b7730cc4
Author: Jeremy Howard <j@howard.fm>
Date:   Sat Feb 3 02:49:20 2018 -0800

    bias false

…and conda env is up to date.

Thank you for your kind help!