How to get an empty ConvLearner for single image prediction?

I’d be happy with a simple way to classify new images on any hardware - expensive GPU or otherwise.

My test case for lesson one is facial recognition - specifically telling apart family members. In my experience with cell phones that use facial recognition to unlock a phone the phones’s facial recognition model can’t tell the difference between siblings nor parents and children…

Seth

I got it working!

I had to basically trick the ImageDataBunch constructor by feeding it some fake filenames, but I appear to have managed to deploy a pre-calculated model on a non-GPU machine!

Here’s the rough outline I used. I had to hard-code in my list of labels:

cat_images_path = Path("/tmp")
cat_fnames = [
    "/{}_1.jpg".format(c)
    for c in [
        "Bobcat",
        "Mountain-Lion",
        "Domestic-Cat",
        "Western-Bobcat",
        "Canada-Lynx",
        "North-American-Mountain-Lion",
        "Eastern-Bobcat",
        "Central-American-Ocelot",
        "Ocelot",
        "Jaguar",
    ]
]
cat_data = ImageDataBunch.from_name_re(
    cat_images_path,
    cat_fnames,
    r"/([^/]+)_\d+.jpg$",
    ds_tfms=get_transforms(),
    size=224,
)
cat_learner = ConvLearner(cat_data, models.resnet34)
cat_learner.model.load_state_dict(
    torch.load("usa-inaturalist-cats.pth", map_location="cpu")
)

Now I can evaluate a new image like so:

img = open_image(BytesIO(bytes))
losses = img.predict(cat_learner)
prediction = cat_learner.data.classes[losses.argmax()]

Note that I’m loading models.resnet34 (which means downloading it the first time the code is imported) even though I don’t think it’s actually needed since I’m using the model from disk. I couldn’t figure out how to call the ConvLearner constructor without it.

14 Likes

BAnd here’s the (crazy simple at the moment) API I’ve deployed using this technique: https://cougar-or-not.now.sh/form - upload a photo of a cougar, house cat or bobcat and it will attempt to tell you which one it is.

The underlying code (and pre-calculated model) is available here: https://github.com/simonw/cougar-or-not - including a Dockerfile which builds a container which can then be deployed to a hosting provider such as Zeit Now - that’s how I’m hosting the https://cougar-or-not.now.sh/form deployment.

The nice thing about this way of hosting models is that it’s essentially free: Zeit Now only spins up a server when a request comes in (kind of like AWS Lambda) and their pricing model is extremely generous. It wouldn’t work for giant model files (larger than a few hundred MB) but the 83MB models I got from building on models.resnet34 fit just fine.

13 Likes

That sounds very good (and easy)! :smiley:

However, https://cougar-or-not.now.sh/upload didn’t worked for me, showing this error message: Method Not Allowed
If I access https://cougar-or-not.now.sh it shows: Not Found

wow Django co-creator… nice. please post more often :grin:

You should be able to make it just a touch simpler with something like this

path = Path("/tmp")
classes = ["Bobcat", "Mountain-Lion", "Domestic-Cat", "Western-Bobcat", "Canada-Lynx", "North-American- Mountain-Lion", "Eastern-Bobcat", "Central-American-Ocelot", "Ocelot", "Jaguar"]
empty_ds = ImageClassificationDataset([classes[0]], [classes[0]], classes)
data = ImageDataBunch.create(empty_ds, empty_ds, path=path, ds_tfms=get_transforms(), size=224)
learn = ConvLearner(data, models.resnet34)
learn.load('/whatever/')
4 Likes

Nice workaround! The data object will never complain if files aren’t there unless you try to load them (by asking for data.train_ds[smthg]).
In the future we’ll definitely add something to the library to make this easier, especially with all the new functionalities in pytorch v1 to put models into production.

Yeah this is a VERY hacky initial setup. It only works if you visit https://cougar-or-not.now.sh/form and submit the image there - any othe URL (including / ) currently throws an error.

1 Like

It works - this is great! :slight_smile:

There’s probably some kind of web framework you could use to handle that… :stuck_out_tongue:

1 Like

Update: posting the fastai-compliant way to predict custom image class.

Ok, finally, I think here is a more or less “canonical” approach to generate a prediction using fastai standard classes and methods:

img = open_image(filename)
losses = img.predict(learn)
learn.data.classes[losses.argmax()]

Original Post

I’m training a model on some dataset like this:

data = ImageDataBunch.from_name_func(..., size=224)  # dataset creation goes here
data.normalize(imagenet_stats)
learner = ConvLearner(data, models.resnet34, metrics=[error_rate])
learner.fit_one_cycle(1)

Now I would like to run my model on some custom image. For example, let’s pretend that I have an image in my local file system. I read this image, and convert into a tensor of appropriate shape and type:

img_path = 'path/to/the/image.png'
pil_image = PIL.Image.open(img_path).convert('RGB').resize((224, 224))
x = torch.tensor(np.asarray(pil_image), dtype=torch.float)
w, h, c = x.size()
x = x.view(c, w, h).to(default_device)

Finally, I feed the image into model:

preds = learner.model(img[None])

However, the last step gives me an error:

~/anaconda3/envs/fastai/lib/python3.7/site-packages/torch/nn/functional.py in batch_norm(input, running_mean, running_var, weight, bias, training, momentum, eps)
   1364         size = list(input.size())
   1365         if reduce(mul, size[2:], size[0]) == 1:
-> 1366             raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
   1367     return torch.batch_norm(
   1368         input, weight, bias, running_mean, running_var,

ValueError: Expected more than 1 value per channel when training, got input size [1, 1024]

Could anyone advise, what is the correct way to prepare new data before feeding into the model? I mean, I would like to do something similar to model.predict(X) from scikit-learn, or keras.

I guess I need to apply normalization as well but I think that probably source of the error is something else.

4 Likes

I guess I’ve fixed it =)

learner.model.eval()
learner.model(img[None].to('cuda'))

One should move model into evaluation mode at first as soon as we don’t need BN layers in testing phase.

5 Likes

Hi,
I am working on the MNIST dataset using
path = untar_data(URLs.MNIST_SAMPLE);

I want to run my model agains a custom image. How do i upload the image ?
I am using Salamander - Cloud GPU.

Any help would be appreciated.
Thanks

You can use scp util for this purpose. Or, if you prefer graphical interface, there are such options like FileZilla or Transmit, depending on your OS.

1 Like

Don’t you think the image should be normalized using the imagenet weights before getting a prediction?

1 Like

Yes, you’re right, forget to update my solution. Here is a final snippet I am using:

@dataclass
class ConvPredictor:
    
    learner: ConvLearner
    mean: FloatTensor
    std: FloatTensor
        
    def __post_init__(self):
        device = self.learner.data.device
        self.mean, self.std = [torch.tensor(x).to(device) for x in (self.mean, self.std)]
        
    def predict(self, x):
        out = self.predict_logits(x)
        best_index = F.softmax(out).argmax()
        return self.learner.data.classes[best_index]
    
    def predict_logits(self, x):
        x = x.to(self.learner.data.device)
        x = normalize(x, self.mean, self.std)
        out = self.learner.model(x[None])
        return out
    
    def predict_from_file(self, filename):
        data = open_image(filename).data
        return self.predict(x)

And then:

img = open_image(fnames[6])
predictor = ConvPredictor(learn, *imagenet_stats)
predictor.predict(img.data)

Here is a link to my version of pets notebook, if interested.

8 Likes

Great! One final thought. I think the image should be normalized not out of the statistics of the data, but using the official imagenet statistics like in cell 11: data.normalize(imagenet_stats)

Correct, I am using *imagenet_stats from lecture to normalize predictions. Just decided to keep them as mean and std attributes in a case when a different statistics was used to normalize data.

I got correct predictions, however I think that we need to resize the image (i got a cuda out of memory)…

I suggest this implementation. It switches the model in learn mode and resizes the image parametrically

3 Likes