How do I predict a batch of images without labels

Part 1 (2019)
1

I have a trained saved model, and I can’t see a way to predict the label for a large batch of images without somehow faking a databunch that has a test set on. Can anyone help, surely there’s a proper way…

2 Likes
2

You can read that data in – same transforms etc as you used to train, using the data block API and then grab your labels from the original location, and then get_preds

Elsewise, don’t same read and set the new images as your test set and get_preds with is_test = true.

1 Like
3

So when I trained I had a path with directory1 and directory2 which were my labels. Now I have unlabeled data, where am I to put the images?

data = ImageDataBunch.from_folder(path, train=".", valid_pct=0.2,
ds_tfms=get_transforms(), size=224, num_workers=4, device=DEVICE).normalize(imagenet_stats)

1 Like
4

not sure if this tutorial helps: i saw some inference stuff written there.

2 Likes
5

Thanks, this is just for a single image. I have 1000s of images to classify so I don’t really want to have to do it in a for loop.

6

maybe the pred_batch would work in this case?

7

Thanks, I’ve looked at that. But it’s unclear what type a batch is, or indeed how I would create that type even if I knew what it was. Looking at the source code it’s tuple of xb and yb, but I have no idea what type they are. Nor can I see any usage of it in the git repo

pred_batch ( ds_type : DatasetType = <DatasetType.Valid: 2> , batch : Tuple = None ) → List [ Tensor ]

1 Like
8

Radek and wdhorton both have good examples if you are using the data block API.

3 Likes
9

learn.predict seems hackeable. I wonder if it’s possible to change this line batch = self.data.one_item(item) to change it to predict a batch instead

   def predict(self, item:ItemBase, **kwargs):
        "Return prect class, label and probabilities for `item`."
        self.callbacks.append(RecordOnCPU())
        batch = self.data.one_item(item)       <<< hack this line
        res = self.pred_batch(batch=batch)
        pred = res[0]
        x = self.callbacks[-1].input
        norm = getattr(self.data,'norm',False)
        if norm:
            x = self.data.denorm(x)
            if norm.keywords.get('do_y',False): pred = self.data.denorm(pred)
        self.callbacks = self.callbacks[:-1]
        ds = self.data.single_ds
        pred = ds.y.analyze_pred(pred, **kwargs)
        out = ds.y.reconstruct(pred, ds.x.reconstruct(x[0])) if has_arg(ds.y.reconstruct, 'x') else ds.y.reconstruct(pred)
        return out, pred, res[0]
1 Like
10

Thanks again, both are using learn.get_preds(ds_type=DatasetType.Test) which only works when you have a full dataset of train, validation and test. Whilst I could do this, I want to give the model to one of my colleagues to run in isolation and this just feels like a hack. Surely there’s a non hacky way to predict a batch in isolation without also needing a train and val set

11

So what would your desired method be for this? Feed in a pandas dataframe and have an array with the predictions returned? This may be a method somebody could code if it doesn’t currently exist.

So, new batch of predictions comes in, you feed it the model and dataframe and a numpy array is returned is how I would imagine this working.

4 Likes
12

For all other deep learning libraries it’s just basic functionality to generate some predictions from some unlabelled data. E.g. if you just want to deploy a model that predicts batches of images rather than single images.

So far my experience of using fastai over just using PyTorch and torchvision is that it get’s you up and running quickly, but now I want to just do the most simple thing and I’ve been trying for 3 hours and it doesn’t seem to be possible without doing it in a horrible hacky way.

13

This is the equivalent I would do in native PyTorch

def get_predictions(model, image_dir, batch_size=16):

trn = transforms.Compose([
    transforms.Resize(input_size),
    transforms.CenterCrop(input_size),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])

images = datasets.ImageFolder(image_dir, trn)
image_names = [x[0] for x in images.imgs]
dataloader = torch.utils.data.DataLoader(images, batch_size=batch_size, shuffle=False, num_workers=3)

model.eval()   # Set model to evaluate mode
                         
output = {}
num_images = 0

ce_loss = nn.CrossEntropyLoss(reduction="none")

# Iterate over data.
for inputs, labels in dataloader:
    inputs = inputs.to(device)
    labels = labels.to(device)

    with torch.no_grad():
        logits = model(inputs)
        _, preds = torch.max(logits, 1)
        loss = ce_loss(logits, labels)
        batch_len =  len(preds)
        for i, fname in enumerate(image_names[num_images:(num_images+batch_len)]):
            output[fname]={"p":logits[i,:].to("cpu").detach().numpy(),
                           "l":loss[i].item(),
                           "y_hat":preds[i].item(),
                           "y":labels[i].item()}
                   
        num_images += len(preds)

return output
1 Like
14

try this: learn.get_preds

Screenshot%20from%202018-12-08%2001-59-21
Screenshot from 2018-12-08 01-59-21.png1076×794 81 KB

15

Thanks, but it hasn’t solved the problem in that I can’t get ‘data’ without first having (or faking) a train and test set which was my original question. I can get it in a hacky way. But if were to deploy this model to production, there must be a way to get batch predictions without a training or validation set

16

Yeah, your’re right, I realised that,There should be a way to pass test data to get_pred

1 Like
17

Sorry for being arsey, I appreciate your help - just getting a bit frustrated with this API. It seems they haven’t yet built a DataBunch that can be made of purely unlabelled data, or at least we haven’t found it.

3 Likes
18

I am a complete nub, but couldn’t you just write a very basic fuction that masks the fact that you are passing in the training and test that are empty, but have the same data bunch characteristics as what you are trying to classify. Please keep in mind I am just learning, I am thinking that they are needed to create the computation graph. Just a thought.

19

:grinning: I was just watching lesson 2 and saw this (and you being mentioned) too

classes = ['black', 'grizzly', 'teddys']
data2 = ImageDataBunch.single_from_classes(path, classes, tfms=get_transforms(), size=224).normalize(imagenet_stats)
learn = create_cnn(data2, models.resnet34).load('stage-2')
pred_class,pred_idx,outputs = learn.predict(img)
pred_class

But after watching half of lesson 2, I come to the conclusion that the scenario encountered by Nick seldom happens. it’s a good-to-have. Sorry Nick

Most of the time you only need to serve one image for prediction

1 Like
20

Hey, yeah it’s not massively common I guess. And you could hack it. Something in me really hates doing hacks though!! :smile: I just think if we could get the attention of somebody from fastai they’ll just tell us which bit of the API to use. I’ve reverted back to Pytorch + torchvision and used the similar settings and learning rate annealing and its got me to pretty much the same place. I think fastai v1 is still very new so there’s pieces missing. Honestly though, some of the utilities they have for cleaning data etc. are awesome and I haven’t seen them anywhere else. Everything is very long winded in native Pytorch so you end up writing your own helper functions. I think this is where fastai was born out of, them making a friendly API for Pytorch. At this point it’s still probably easier to do some stuff from scratch such as combining image and tabular data.

That said, there’s some really neat tricks built in to fastai such as the differential learning rates which I’ve figured out how to do in Pytorch now but again it is long winded.

4 Likes