Yah I posted of a working example in the advanced thread here: https://forums.fast.ai/t/using-our-trained-model-against-a-specific-image/28262?u=wgpubs
However, I’d love to be able to use that Image.predict(img, learn) which requires a working Learner which looks to require a valid DataBunch currently.
Super that you have it working. Yeah, probably would have been better to keep the dataloaders and transform objects separate in the library design, so we could use the transforms easily for post training use cases.
Maybe there’s a better way to do this in the library, and we are just talking a roundabout way for a one-liner.
But, if not, then since the predict function is rather small, we could
pickle, and load it back in productionI’d love to be able to use fastai in production as well, sometime this/next year. So, this would be very useful.
I’d also like to be able to do this.
My goal is to train a model on an expensive GPU machine, save that model to disk (as a *.pth file), then load up the saved model on a much cheaper machine and use it to classify new images using the CPU rather than the GPU.
If I can do this it will be trivial to deploy my models to a production API somewhere that I can then evaluate new images against.
The problem is… I need to be able to instantiate a learner such that I can run the following:
my_learner = instantiate_learner_somehow("my_model.pth")
img = open_image("my_new_image.jpg")
losses = img.predict(my_learner)
prediction = my_learner.data.classes[losses.argmax()]
I can’t figure out how to instantiate the learner. I have 600MB of images and I don’t want to have to deploy them along with that .pth file just so I can instantiate an ImageDataBunch (which I need to pass to the ConvLearner constructor).
I didn’t quite expect people to be asking for people to do this right after lesson 1, so apologies I’m a little unprepared. Let me figure out the best way to handle this…
I’d be happy with a simple way to classify new images on any hardware - expensive GPU or otherwise.
My test case for lesson one is facial recognition - specifically telling apart family members. In my experience with cell phones that use facial recognition to unlock a phone the phones’s facial recognition model can’t tell the difference between siblings nor parents and children…
Seth
I got it working!
I had to basically trick the ImageDataBunch constructor by feeding it some fake filenames, but I appear to have managed to deploy a pre-calculated model on a non-GPU machine!
Here’s the rough outline I used. I had to hard-code in my list of labels:
cat_images_path = Path("/tmp")
cat_fnames = [
"/{}_1.jpg".format(c)
for c in [
"Bobcat",
"Mountain-Lion",
"Domestic-Cat",
"Western-Bobcat",
"Canada-Lynx",
"North-American-Mountain-Lion",
"Eastern-Bobcat",
"Central-American-Ocelot",
"Ocelot",
"Jaguar",
]
]
cat_data = ImageDataBunch.from_name_re(
cat_images_path,
cat_fnames,
r"/([^/]+)_\d+.jpg$",
ds_tfms=get_transforms(),
size=224,
)
cat_learner = ConvLearner(cat_data, models.resnet34)
cat_learner.model.load_state_dict(
torch.load("usa-inaturalist-cats.pth", map_location="cpu")
)
Now I can evaluate a new image like so:
img = open_image(BytesIO(bytes))
losses = img.predict(cat_learner)
prediction = cat_learner.data.classes[losses.argmax()]
Note that I’m loading models.resnet34 (which means downloading it the first time the code is imported) even though I don’t think it’s actually needed since I’m using the model from disk. I couldn’t figure out how to call the ConvLearner constructor without it.
BAnd here’s the (crazy simple at the moment) API I’ve deployed using this technique: https://cougar-or-not.now.sh/form - upload a photo of a cougar, house cat or bobcat and it will attempt to tell you which one it is.
The underlying code (and pre-calculated model) is available here: https://github.com/simonw/cougar-or-not - including a Dockerfile which builds a container which can then be deployed to a hosting provider such as Zeit Now - that’s how I’m hosting the https://cougar-or-not.now.sh/form deployment.
The nice thing about this way of hosting models is that it’s essentially free: Zeit Now only spins up a server when a request comes in (kind of like AWS Lambda) and their pricing model is extremely generous. It wouldn’t work for giant model files (larger than a few hundred MB) but the 83MB models I got from building on models.resnet34 fit just fine.
That sounds very good (and easy)! 
However, https://cougar-or-not.now.sh/upload didn’t worked for me, showing this error message: Method Not Allowed
If I access https://cougar-or-not.now.sh it shows: Not Found
wow Django co-creator… nice. please post more often 
You should be able to make it just a touch simpler with something like this
path = Path("/tmp")
classes = ["Bobcat", "Mountain-Lion", "Domestic-Cat", "Western-Bobcat", "Canada-Lynx", "North-American- Mountain-Lion", "Eastern-Bobcat", "Central-American-Ocelot", "Ocelot", "Jaguar"]
empty_ds = ImageClassificationDataset([classes[0]], [classes[0]], classes)
data = ImageDataBunch.create(empty_ds, empty_ds, path=path, ds_tfms=get_transforms(), size=224)
learn = ConvLearner(data, models.resnet34)
learn.load('/whatever/')Nice workaround! The data object will never complain if files aren’t there unless you try to load them (by asking for data.train_ds[smthg]).
In the future we’ll definitely add something to the library to make this easier, especially with all the new functionalities in pytorch v1 to put models into production.
Yeah this is a VERY hacky initial setup. It only works if you visit https://cougar-or-not.now.sh/form and submit the image there - any othe URL (including / ) currently throws an error.
It works - this is great! 
There’s probably some kind of web framework you could use to handle that… 
Update: posting the fastai-compliant way to predict custom image class.
Ok, finally, I think here is a more or less “canonical” approach to generate a prediction using fastai standard classes and methods:
img = open_image(filename)
losses = img.predict(learn)
learn.data.classes[losses.argmax()]
I’m training a model on some dataset like this:
data = ImageDataBunch.from_name_func(..., size=224) # dataset creation goes here
data.normalize(imagenet_stats)
learner = ConvLearner(data, models.resnet34, metrics=[error_rate])
learner.fit_one_cycle(1)
Now I would like to run my model on some custom image. For example, let’s pretend that I have an image in my local file system. I read this image, and convert into a tensor of appropriate shape and type:
img_path = 'path/to/the/image.png'
pil_image = PIL.Image.open(img_path).convert('RGB').resize((224, 224))
x = torch.tensor(np.asarray(pil_image), dtype=torch.float)
w, h, c = x.size()
x = x.view(c, w, h).to(default_device)
Finally, I feed the image into model:
preds = learner.model(img[None])
However, the last step gives me an error:
~/anaconda3/envs/fastai/lib/python3.7/site-packages/torch/nn/functional.py in batch_norm(input, running_mean, running_var, weight, bias, training, momentum, eps)
1364 size = list(input.size())
1365 if reduce(mul, size[2:], size[0]) == 1:
-> 1366 raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
1367 return torch.batch_norm(
1368 input, weight, bias, running_mean, running_var,
ValueError: Expected more than 1 value per channel when training, got input size [1, 1024]
Could anyone advise, what is the correct way to prepare new data before feeding into the model? I mean, I would like to do something similar to model.predict(X) from scikit-learn, or keras.
I guess I need to apply normalization as well but I think that probably source of the error is something else.
I guess I’ve fixed it =)
learner.model.eval()
learner.model(img[None].to('cuda'))
One should move model into evaluation mode at first as soon as we don’t need BN layers in testing phase.
Hi,
I am working on the MNIST dataset using
path = untar_data(URLs.MNIST_SAMPLE);
I want to run my model agains a custom image. How do i upload the image ?
I am using Salamander - Cloud GPU.
Any help would be appreciated.
Thanks
You can use scp util for this purpose. Or, if you prefer graphical interface, there are such options like FileZilla or Transmit, depending on your OS.
Don’t you think the image should be normalized using the imagenet weights before getting a prediction?
Yes, you’re right, forget to update my solution. Here is a final snippet I am using:
@dataclass
class ConvPredictor:
learner: ConvLearner
mean: FloatTensor
std: FloatTensor
def __post_init__(self):
device = self.learner.data.device
self.mean, self.std = [torch.tensor(x).to(device) for x in (self.mean, self.std)]
def predict(self, x):
out = self.predict_logits(x)
best_index = F.softmax(out).argmax()
return self.learner.data.classes[best_index]
def predict_logits(self, x):
x = x.to(self.learner.data.device)
x = normalize(x, self.mean, self.std)
out = self.learner.model(x[None])
return out
def predict_from_file(self, filename):
data = open_image(filename).data
return self.predict(x)
And then:
img = open_image(fnames[6])
predictor = ConvPredictor(learn, *imagenet_stats)
predictor.predict(img.data)
Here is a link to my version of pets notebook, if interested.