I have the following to get my prediction for a single image. How can I get the reference to the classification for the predictions?
Here are a few of the possible classifications
Are the classifications still available through learn.data.classes in your code?
Yes I do. When I use argmax, I can get the most likely class out of there, but not sure how to get all the non-zero predicted values.
Also, I am now seeing this error. I am using the same values as my successful runs, but about 10x the amount of data. There are about 200k in this dataset so finding an error in the CSV is proving tough. ny thoughts on what the issue might be here?
~/fastai/courses/dl1/fastai/dataset.py in <listcomp>(.0)
64 skip = 1 if skip_header else 0
65 csv_lines = [o.strip().split(',') for o in open(fn)][skip:]
---> 66 fnames = [fname for fname, _ in csv_lines]
67 csv_labels = {a:b.split(' ') for a,b in csv_lines}
68 all_labels = sorted(list(set(p for o in csv_labels.values() for p in o)))
ValueError: too many values to unpack (expected 2)
(fastai) ubuntu@ip-172-31-34-254:~/data/shopstyle$ wc -l prod_train.csv
198869 prod_train.csv
(fastai) ubuntu@ip-172-31-34-254:~/data/shopstyle$ ls train/ | wc -l
198869This happened to be a single line with a comma in the tags…something to add to my generator. 
Still looking for info on how to grab all the classes that have a non-zero prediction for an image.
Also, does anyone have a code example of service prediction results via an endpoint? That is my use case here.
Birch
One more question related to this.
I have all my images in the ‘train’ folder. my CSV has the image id and the tags separated by space, all that is good. What should be in the test directory? In the planet data directory, there are almost as many in test as there are in train, and I am not seeing a test csv. Is the fastai library automatically spitting the training/test set for me? If so, why did it complain that the test dir was empty before I added stuff to it?
Ah. In that case, wouldn’t you just use zip(* to transpose the columns out, or just iterate through the rows and columns? (I guess there’s a more elegant solution …)
I haven’t tried to connect all this to a service, would be interested to hear how you get on with it
I wouldn’t have thought that the test set would be mandatory, what’s the message you’re getting?
Ty @pete.condon. We will be doing the service part as part of an upcoming hackathon next week, I will report back.
I’ll give the zip a try. When I am done with this training run, I will do it again with no test data and share the error.
Birch
1 Like
I am running the following code, and get an error while trying to pass the data to the predict_array function. Is this the proper code to do this?
from fastai.conv_learner import *
from planet import f2
PATH = 'data/shopstyle/'
metrics=[f2]
f_model = resnet34
def get_data(sz):
tfms = tfms_from_model(f_model, sz, aug_tfms=transforms_side_on, max_zoom=1.05)
return ImageClassifierData.from_csv(PATH, 'train', label_csv, tfms=tfms, suffix='.jpg', val_idxs=val_idxs, test_name='test')
def print_list(list_or_iterator):
return "[" + ", ".join( str(x) for x in list_or_iterator) + "]"
label_csv = f'{PATH}prod_train.csv'
n = len(list(open(label_csv)))-1
val_idxs = get_cv_idxs(n)
sz = 64
data = get_data(sz)
print("Loading model...")
learn = ConvLearner.pretrained(f_model, data, metrics=metrics)
learn.load(f'{sz}')
#learn.load("tmp")
print("Predicting...")
learn.precompute=False
trn_tfms, val_tfrms = tfms_from_model(f_model, sz)
#im = val_tfrms(open_image(f'{PATH}valid/4500132.jpg'))
im = val_tfrms(np.array(PIL.Image.open(f'{PATH}valid/4500132.jpg')))
preds = learn.predict_array(im[None])
p=list(zip(data.classes, preds))
print("predictions = " + print_list(p))
Getting this error
Traceback (most recent call last):
File "predict.py", line 34, in <module>
preds = learn.predict_array(im[None])
File "/home/ubuntu/fastai/courses/dl1/fastai/learner.py", line 266, in predict_array
def predict_array(self, arr): return to_np(self.model(V(T(arr).cuda())))
File "/home/ubuntu/src/anaconda3/envs/fastai/lib/python3.6/site-packages/torch/nn/modules/module.py", line 325, in __call__
result = self.forward(*input, **kwargs)
File "/home/ubuntu/src/anaconda3/envs/fastai/lib/python3.6/site-packages/torch/nn/modules/container.py", line 67, in forward
input = module(input)
File "/home/ubuntu/src/anaconda3/envs/fastai/lib/python3.6/site-packages/torch/nn/modules/module.py", line 325, in __call__
result = self.forward(*input, **kwargs)
File "/home/ubuntu/src/anaconda3/envs/fastai/lib/python3.6/site-packages/torch/nn/modules/batchnorm.py", line 37, in forward
self.training, self.momentum, self.eps)
File "/home/ubuntu/src/anaconda3/envs/fastai/lib/python3.6/site-packages/torch/nn/functional.py", line 1011, in batch_norm
raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size))
ValueError: Expected more than 1 value per channel when training, got input size [1, 1024]Why are you passing None?
I don’t know this , looks great…
Can you explain?
I have been cobbling instructions from various posts. Not convinced it is right. Although I thought I had it working like this yesterday.
Without the None, I get this:
ValueError: Expected 4D tensor as input, got 3D tensor instead.@binarypoet
You want to predict on a image?
Jeremy shows how it’s done somewhere in lesson1.ioynb(not quite sure) or in his Lecs…
The last error is justified…
Because we generally passin to the model
(nunber of images,img_ht,img_wdt,channels)?
It’s like because Keras also expects 4D tensors as inputs…(not sure any PyTorch)
So it could be removed if we add another axis…
np.expand_dims(IMG,axis=0) or image = image[..., np.newaxis] should work…
Yes, just want to be able to predict an image at a time for now. All variations of what I have found here and in git have failed in some form or fashion. Looking for a definitive example on how to predict a single image at atime that is passed in.
Might need some simple changes…
TY, have tried various variations of that as well with no luck. Can’t seem to find anything n the current git notebooks for doing this. Seems like I am just missing something simple
This code seemed to work yesterday. The only thing I know I am doing different is loading the model from disk instead of training it right before. AM I loading the model correctly?
from fastai.conv_learner import *
from planet import f2
PATH = 'data/shopstyle/'
metrics=[f2]
f_model = resnet34
label_csv = f'{PATH}prod_train.csv'
n = len(list(open(label_csv)))-1
val_idxs = get_cv_idxs(n)
sz = 128
def get_data(sz):
tfms = tfms_from_model(f_model, sz, aug_tfms=transforms_side_on, max_zoom=1.05)
return ImageClassifierData.from_csv(PATH, 'train', label_csv, tfms=tfms, suffix='.jpg', val_idxs=val_idxs, test_name='test')
def print_list(list_or_iterator):
return "[" + ", ".join( str(x) for x in list_or_iterator) + "]"
data = get_data(sz)
print("Loading model...")
learn = ConvLearner.pretrained(f_model, data, metrics=metrics)
learn.load(f'{sz}')
learn.precompute=False
print("Predicting...")
trn_tfms, val_tfms = tfms_from_model(f_model, sz)
im = val_tfms(np.array(PIL.Image.open(f'{PATH}valid/4500132.jpg')))
preds = learn.predict_array(im[None])
print("predictions = " + preds)What is ‘x’ in this context?
Had asked different people, they say probably the error is in image dimensions…
Try adding the extra dimension using
np.reshape()
I am not quite sure…
Don’t know but may be shape of your data point is not correct . It should be like (1,299,299,3). Something like this a 4d tensor…
img = cv2.imread(img_path)
img = cv2.resize(img, dsize = (200,200))
img = np.einsum('ijk->kij', img)
img = np.expand_dims(img, axis =0)
img = torch.from_numpy(img)
learn.model(Variable(img.float()).cuda())
This code is from Rajat_(Slack)
2 Likes