How to predict on the test set


(Jayashree Sridhar) #1

Hello,
I have watched first 30 minutes of Lesson 1 video and executed lesson1.ipynb file till the prediction part. I have a doubt.
It is mentioned in the jupyter notebook that
# this gives prediction for validation set. Predictions are in log scale
log_preds = learn.predict()
log_preds.shape

This prediction is for the validation set. I want to know how to do the prediction for the test set.


Wiki: Lesson 1
(Alan O'Donnell) #2

learn.predict(is_test=True) :slight_smile:


(Jayashree Sridhar) #3

Thank you Alan :slight_smile:


(Jayashree Sridhar) #4

Hi Alan,
When I changed
log_preds = learn.predict(is_test=True)
log_preds.shape
I am getting this error


TypeError Traceback (most recent call last)
in ()
1 # this gives prediction for validation set. Predictions are in log scale
----> 2 log_preds = learn.predict(is_test=True)
3 log_preds.shape

~/fastai/courses/dl1/fastai/learner.py in predict(self, is_test)
253 self.load(‘tmp’)
254
–> 255 def predict(self, is_test=False): return self.predict_with_targs(is_test)[0]
256
257 def predict_with_targs(self, is_test=False):

~/fastai/courses/dl1/fastai/learner.py in predict_with_targs(self, is_test)
257 def predict_with_targs(self, is_test=False):
258 dl = self.data.test_dl if is_test else self.data.val_dl
–> 259 return predict_with_targs(self.model, dl)
260
261 def predict_dl(self, dl): return predict_with_targs(self.model, dl)[0]

~/fastai/courses/dl1/fastai/model.py in predict_with_targs(m, dl)
130 if hasattr(m, ‘reset’): m.reset()
131 res = []
–> 132 for *x,y in iter(dl): res.append([get_prediction(m(*VV(x))),y])
133 preda,targa = zip(*res)
134 return to_np(torch.cat(preda)), to_np(torch.cat(targa))

TypeError: ‘NoneType’ object is not iterable

I have downloaded plant species dataset from kaggle and trying to train a model and predict the species for the test datatset.

I have used the same code as used for cats dogs classification.

Since this a multi label classification,do I need to change my code anywhere.

arch=resnet34
data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(arch, sz))
learn = ConvLearner.pretrained(arch, data, precompute=True)
learn.fit(0.01, 50)

This is the label for a val data
data.val_y

How to get labels for test data?

preds = np.argmax(log_preds, axis=1) # from log probabilities to 0 or 1
probs = np.exp(log_preds[:,1]) # pr(dog)

Predictions are in log scale. For binary classification we are converting it to be in between 0 and 1. For multi label classfication problems,how should we convert the predictions in to corresponding labels for test data?


(ecdrid) #5

For this competition,

I hope you have set the directory structure properly like( separating all the individual classes together in a particular folder as it lacks a CSV )

Have a search here, it’s answered somewhere…

Hope it helps…


(Jayashree Sridhar) #6

Yes, I have maintained directory structure for the validation set.I want to know how to proceed with the predictions on the test set.


(ecdrid) #7

@jayashree
Haven’t updated my kernel so just change the concerned lines…


(Jayashree Sridhar) #8

Thanks, @ecdrid for the prompt reply.I will try the necessary code changes.


(ecdrid) #9

Pleasure


(Jayashree Sridhar) #10

Hello @ecdrid
When I run this code

log_preds,y = learn.TTA()
accuracy(log_preds,y)
I am getting this error
AttributeError Traceback (most recent call last)
in ()
1 # this gives prediction for validation set. Predictions are in log scale
2 log_preds,y = learn.TTA()
----> 3 accuracy(log_preds,y)

~/fastai/courses/dl1/fastai/metrics.py in accuracy(preds, targs)
4 def accuracy(preds, targs):
5 preds = np.argmax(preds, axis=1)
----> 6 return (preds==targs).mean()
7
8 def accuracy_thresh(thresh):

AttributeError: ‘bool’ object has no attribute ‘mean’

Can you help me in sorting this out?


(ecdrid) #11

Can you share your notebook and name the dataset (also)…


(Jayashree Sridhar) #12

Hi @ecdrid , I am using this code on lesson1.ipynb dogs cats dataset. Do I need to change any parameter while training the data?


(ecdrid) #13

We need to play with the parameters …

And I won’t be able to help you now because I haven’t ran the DL kernels in a while…(so can’t update myself to the recent fast.ai codes ) Sorry
So my replies might not make sense…

Thanks…

@alessa(sorry) @cqfd(sorry) @ramesh(sorry) @radek(sorry)
(Can’t tag more as I don’t remember the names exactly…)


(Aless Bandrabur) #14

Actually there were some updates for TTA function,

you need to call the following:

log_preds, y = learn.tta()
probs = np.mean(np.exp(log_preds), axis=0)
accuracy(probs,y)

Lesson 2 Dog Breeds Error on call of accuracy(log_preds,y)
(Jayashree Sridhar) #15

Thanks, alessa. The code above gives the accuracy for the validation set. When I did this for predicting the test set

log_preds_test=learn.predict(is_test=True)
log_preds_test.shape

I am getting this error

TypeError Traceback (most recent call last)
in ()
----> 1 log_preds_test=learn.predict(is_test=True)
2 log_preds_test.shape

~/fastai/courses/dl1/fastai/learner.py in predict(self, is_test)
253 self.load(‘tmp’)
254
–> 255 def predict(self, is_test=False): return self.predict_with_targs(is_test)[0]
256
257 def predict_with_targs(self, is_test=False):

~/fastai/courses/dl1/fastai/learner.py in predict_with_targs(self, is_test)
257 def predict_with_targs(self, is_test=False):
258 dl = self.data.test_dl if is_test else self.data.val_dl
–> 259 return predict_with_targs(self.model, dl)
260
261 def predict_dl(self, dl): return predict_with_targs(self.model, dl)[0]

~/fastai/courses/dl1/fastai/model.py in predict_with_targs(m, dl)
130 if hasattr(m, ‘reset’): m.reset()
131 res = []
–> 132 for *x,y in iter(dl): res.append([get_prediction(m(*VV(x))),y])
133 preda,targa = zip(*res)
134 return to_np(torch.cat(preda)), to_np(torch.cat(targa))

TypeError: ‘NoneType’ object is not iterable

What should be changed in the code above?

I tried modifying TTA function too

log_preds_test,y_test = learn.TTA(is_test=True)
probs_test = np.mean(np.exp(log_preds_test),0)

Still getting same error
Thanks


(Jeremy Howard (Admin)) #16

One more change: you need to use accuracy_np() (which works with numpy arrays), not accuracy() (which works with pytorch tensors). This is a very recent change.


(Jayashree Sridhar) #17

@jeremy How to use TTA to make predictions on test set?


Dog Breed Identification challenge
(Jeremy Howard (Admin)) #18

@jayashree please don’t at-mention me unless it’s specifically an issue for me alone. In this case, your question has been answered in previous forum threads.


(Jayashree Sridhar) #19

Sure.


(ananth) #20

Hi jayashree, I have the same issue and not able to locate the solution in the forums. Can you share your findings?