Beginner Question: How to predict on Test Set


(Navneet Kumar Chaudhary) #1

Hi guys,
Context: DogBreedClassifier
I am new to DL. I have trained my model and saved my weights.
Now I want to use this model to predict on Test Folder, and update my submission.csv file.
Thanks in Advance


(Navneet Kumar Chaudhary) #2

(Navneet Kumar Chaudhary) #3

def split_data_add_test_folder(self, test_folder:str=‘test’, label:Any=None):
“Add test set containing items from folder test_folder and an arbitrary label”
items = ImageFileList.from_folder(self.path/test_folder)
return self.add_test(items, label=label)

SplitData.add_test_folder = split_data_add_test_folder.

We can add the test folder using this.


(Navneet Kumar Chaudhary) #4

File Structure for my scenario

path\

train\
    clas1\
    clas2\
    ...
 valid\
    clas1\
    clas2\
    ...
 test\

Workaround:
path = ‘data/train’
Also while defining the data I had to
data = ImageDataBunch.from_folder(path, train=‘train’, valid_pct=0.2, test=’…/test’, ds_tfms=tfms, size=224)
See here I had to define test = ‘…/input’,
somehow with path set at ‘data/’ I could not get the test_ds loaded.


(Prithvi MK) #5

You can use the data block API mentioned in data_block section of the fastai docs

data = (ImageItemList.from_folder("add_your_path_here") #Where to find the data? -> in path and its subfolders
        .split_by_folder()              #How to split in train/valid? -> use the folders
        .label_from_folder()            #How to label? -> depending on the folder of the filenames
        .add_test_folder()              #Optionally add a test set (here default name is test)
        .transform()       #Data augmentation? -> use tfms with a size of 64
        .databunch())                   #Finally? -> use the defaults for conversion to ImageDataBunch

(Matthew Arthur) #6

Should I be able to mix from types in the same databunch() API call? I have a dataframe input for my train/valid images and I want to use a folder for my test images. How should I approach this?


#7

When you call add_test you can pass any ItemList so you can create your train/valid with ImageItemList.from_df then when you add the test set:

.add_test(ImageItemList.from_folder(...)

or even more direct: add_test_from_folder should work.


(Md Mofijul Islam) #8

Thanks for the explanation, it works. But I could not able to retrieve the test image files name.


#9

The test image filenames will be in data.test_ds.x.items.