How to add a test set

fastai
1

Hello everyone.

I am working on some kaggle competitions and I’d like to make some predictions to test my models.
Sadly, I cannot add a test set to my model.

Here is what I did:

data = ImageDataBunch.from_csv(‘data/train/’, ds_tfms=tfms, size=128).add_test_folder()

data = (ImageItemList.from_csv(‘data/train/’, csv_name=‘labels.csv’).label_from_df(cols=‘Id’).add_test_folder())

I tried with add_test_folder and add_test but no matter what I use, I end up with an error.

Any idea on how to fix that ?

2

What kind of error do you get?

3

I get an AttributeError.

4

I have loaded test data in this kaggle kernel:
https://www.kaggle.com/nitron/poisonous-plant-classifier
Hope this helps :slight_smile:

1 Like
5

Thanks Nitron that’s worked for me as well while facing kind of similar issue.

Regards,
H. Smith https://dltutuapp.com/tutuapp-download/ . . https://kodi.software/

6

Wasn’t super obvious, but with a structure like:

path\
  train\
  test\
  labels.csv

I got it to work with
data = ImageDataBunch.from_csv(path, ds_tfms=tfms, test=‘test’, size=128)

7

I know I’m late, but for future users:
I had a dataset which had a train.csv containing all the filenames and labels, and a folder “train” which contains both train and test images that are not included in the train.csv file, but I had “sample_submissions.csv” which cointained the test filenames, I loaded the dataset by doing the following:

test_data = ImageList.from_csv(path="./data/train",csv_name="../sample_submission.csv",header="infer")
data = ImageDataBunch.from_csv(path="./data/",folder="train",csv_labels="./train.csv",ds_tfms=get_transforms()
data.add_test(test_data)

If you print “data”
you could see the test set is loaded to the ImageDataBunch
ImageDataBunch;

Train: LabelList (13628 items)
x: ImageList
Image (3, 150, 150),Image (3, 150, 150),Image (3, 150, 150),Image (3, 150, 150),Image (3, 150, 150)
y: CategoryList
0,4,5,0,4
Path: data;

Valid: LabelList (3406 items)
x: ImageList
Image (3, 150, 150),Image (3, 150, 150),Image (3, 150, 150),Image (3, 150, 150),Image (3, 150, 150)
y: CategoryList
3,4,1,2,0
Path: data;

Test: LabelList (7301 items)
x: ImageList
Image (3, 150, 150),Image (3, 150, 150),Image (3, 150, 150),Image (3, 150, 150),Image (3, 150, 150)
y: EmptyLabelList
,,,,
Path: data
9 Likes
8

just as a note to users coming from the future, this is not working now after the update to fastai library.