How do I predict a batch of images without labels

Yeah, your’re right, I realised that,There should be a way to pass test data to get_pred

1 Like

Sorry for being arsey, I appreciate your help - just getting a bit frustrated with this API. It seems they haven’t yet built a DataBunch that can be made of purely unlabelled data, or at least we haven’t found it.

3 Likes

I am a complete nub, but couldn’t you just write a very basic fuction that masks the fact that you are passing in the training and test that are empty, but have the same data bunch characteristics as what you are trying to classify. Please keep in mind I am just learning, I am thinking that they are needed to create the computation graph. Just a thought.

:grinning: I was just watching lesson 2 and saw this (and you being mentioned) too

classes = ['black', 'grizzly', 'teddys']
data2 = ImageDataBunch.single_from_classes(path, classes, tfms=get_transforms(), size=224).normalize(imagenet_stats)
learn = create_cnn(data2, models.resnet34).load('stage-2')
pred_class,pred_idx,outputs = learn.predict(img)
pred_class

But after watching half of lesson 2, I come to the conclusion that the scenario encountered by Nick seldom happens. it’s a good-to-have. Sorry Nick

Most of the time you only need to serve one image for prediction

1 Like

Hey, yeah it’s not massively common I guess. And you could hack it. Something in me really hates doing hacks though!! :smile: I just think if we could get the attention of somebody from fastai they’ll just tell us which bit of the API to use. I’ve reverted back to Pytorch + torchvision and used the similar settings and learning rate annealing and its got me to pretty much the same place. I think fastai v1 is still very new so there’s pieces missing. Honestly though, some of the utilities they have for cleaning data etc. are awesome and I haven’t seen them anywhere else. Everything is very long winded in native Pytorch so you end up writing your own helper functions. I think this is where fastai was born out of, them making a friendly API for Pytorch. At this point it’s still probably easier to do some stuff from scratch such as combining image and tabular data.

That said, there’s some really neat tricks built in to fastai such as the differential learning rates which I’ve figured out how to do in Pytorch now but again it is long winded.

4 Likes

I haven’t tested yet but I guess this post can help you. https://forums.fast.ai/t/how-to-get-an-empty-convlearner-for-single-image-prediction/28245/50 . This is for single image but I think I will work with a batch of image too. Load the model you have trained and predict your images in pure Pytorch .

I am facing a similar problem and so do many people. I guess fastai will make a friendly API for this soon :smiley:

The correct way to do this is to use a test set (it’s literally what test sets are for in fastai). You’ll just have an empty train and validation set in your databunch - which is fine, since you won’t be using them.

DataBunch handles all the serialization for you now of your preprocessors (such as your list of classes), and makes life far easier than doing things manually with Pytorch.

3 Likes

Thanks for the clarification here – That’s what I suspected after reading as much as possible on the forums, but I wasn’t sure :slight_smile:

After the course we’ll be simplifying this further, by serializing data processing/metadata with the learner. But I just added some tiny tweaks to make it a bit simpler, as well as adding some docs to the inference tutorial to show an example. https://docs.fast.ai/tutorial.inference.html#A-classification-problem

11 Likes

That’s literally perfect :slight_smile:

Great, thanks this is what I was after. I seem to remember trying with an empty train, val dataset but it threw an index out of range error. This looks much simpler. Will give it a shot!

@jeremy that’s working great thanks. One small request though…

LabelLists seems to be taking the path from the location of the original training data, despite me trying to override it. This forces me to replicate the folder structure used to train the model in order to load my test set. This happens even if I explicity set the path of the LabelList object i.e.

sd = LabelLists.load_empty(test_path/'export.pkl', tfms=get_transforms(), size=224).add_test_folder('test')
sd.path = test_path
empty_data = sd.databunch().normalize(imagenet_stats)

I think the problem lies in this method:

@classmethod
def load_empty(cls, fn:PathOrStr, tfms:TfmList=None, tfm_y:bool=False, **kwargs):
    train_ds = LabelList.load_empty(fn, tfms=tfms[0], tfm_y=tfm_y, **kwargs)
    valid_ds = LabelList.load_empty(fn, tfms=tfms[1], tfm_y=tfm_y, **kwargs)
    return LabelLists(valid_ds.path, train=train_ds, valid=valid_ds)

Suggest changing it to something like:

def load_empty(cls, fn:PathOrStr, tfms:TfmList=None, tfm_y:bool=False, **kwargs):
	path = os.path.dirname(fn)
    train_ds = LabelList.load_empty(fn, tfms=tfms[0], tfm_y=tfm_y, **kwargs)
    valid_ds = LabelList.load_empty(fn, tfms=tfms[1], tfm_y=tfm_y, **kwargs)
    return LabelLists(path, train=train_ds, valid=valid_ds)

EDIT: My bad, if I override the path before before I add_test_folder then it works. But my suggestion stands…

2 Likes

Just pushed a fix, you should now pass path and fn (defaults to export.pkl) separately and the path you give will override the one of the training.

1 Like

Thanks both!


Not the most elegant way, but it may be useful to others, so here’s how I did it.

P.S. I don’t classify vans for fun, this is actually for work!

5 Likes

Sorry to hijack this thread, but version 1.034 of the library doesn’t have load_empty under LabelLists

Hey, it’s definitely there. See https://github.com/fastai/fastai/blob/master/fastai/data_block.py#L460

Although I have a feeling (not sure) they may have moved ItemLists to a different file so your error could be a red herring. You might just need to restart your kernel.

import fastai.utils
show_install()

Make sure you are in fact using 1.0.34. I haven’t testing the load_empty stuff yet, but it seems like a good way to go about it.

load_empty isn’t available until 1.0.36 according to this: https://github.com/fastai/fastai/blob/master/CHANGES.md#1036-2018-12-08

Major kudos for using CHANGES.md! :smiley:

2 Likes

As a minor related question, is there a way to ensure that your image transforms are actually being applied? Because I followed @safekidda 's screenshot but my classifier labeled everything as a single class - but if I shuffled the images into my test set things worked fine. The only explanation I can think of is that the images are different dimensions than the transformed training images and that the LabelList method is enforcing a different set of transformations even though my parameters to get_transforms() are the same.