Your DataLoaders is learn.dls. So you can access each DataLoader as learn.dls.valid and learn.dls.train. So you can do next(iter(learn.dls.valid)) or just first(learn.dls.valid) or even learn.dls.valid.one_batch() to grab a batch, exactly as the model will see it.
2 Likes
happened again - seeing random preds that don’t add up to 1:
preds:
tensor([[ 1.4974, -0.1202, 0.4346, 1.1905, -1.8412],
[ 2.0858, -1.0677, -1.5246, -1.3662, 0.0106]])
also had similar garbage coming back from single image preds.
I’ll restart the kernel but it seems if the server is up and running for a while then later results are off…or maybe I somehow mucked with the dls settings after trying to get the test images… 
The fact they’re all not above 0 concerns me, as they all should be. You’re sure those aren’t losses? Is it multi-category or just classification?
same classification (single result from 5 classes) as I was just using and showed in above code :). just left the server running and needed to run a new images.
I’m restarting server now but just wanted to post since I saw simliar oddity last week which was then making me unclear if it was returning losses,etc.
Note - restarting the kernel fixed it - same 2 images:
tensor([[0.4320, 0.0857, 0.1492, 0.3178, 0.0153],
[0.8150, 0.0348, 0.0220, 0.0258, 0.1023]])
I’ll have to put some code in that if we get a negative number in these preds to alert and not make a prediction.
Hey @LessW2020,
This is really helpful as I am trying to use a similar end2end scenario for Tabular Data.
Once you are done can you please add a working extended example on any example notebook of V2 Class notebooks for Image one which is similar to your dataset.
1 Like
Hi @navneetkrch - glad it is helpful and yes I can make a more extended example and post it as a notebook. It’s a good idea and I’ll try to do it for ImageWoof or similar common dataset so it’s ties in with the course notebooks.
Is it possible to make a seperate data loader for the test data?
I made data loaders for the training data by doing the following:
train_data = DataBlock(blocks = (ImageBlock(cls=RGBYPILImage), MultiCategoryBlock),
get_x = lambda x:os.path.join(TRAIN, f'{x[0]}'),
splitter = RandomSplitter(),
get_y = lambda x:x[1].split(' '))
dls = train_data.dataloaders(train_df, bs=batch_size)
(here RGBYPILImage is my own class that takes 4 images and combines them into a single RGB image)
Training with this works fine.
I then thought that I’d be able to do the same thing for the test data, so I made a data loader like this:
test_data = DataBlock(blocks = (ImageBlock(cls=RGBYPILImage)),
get_x = lambda x:os.path.join(TEST, f'{x[0]}'))
test_dls = test_data.dataloaders(test_df, bs=batch_size)
So its exactly the same as the training data loader, but has no split and no y values. This works fine with “show_batch”, however when I try to set this data loader into the “get_preds” function it returns me a “‘DataLoaders’ has no len()” error
preds,_ = learn.get_preds(dl=test_dls)
Would anyone know what I’m doing wrong & how I can create a data loader for inference?
You should do dis.test_dl() and pass in the items you want.
Can “test_dl()” take a data loader?
The trouble is that I don’t have individual images - I have a red, green, blue and yellow image that I then need to combine into a single RGB image, which is what the “ImageBlock(cls=RGBYPILImage)” is doing.
If its not possible for test_dl() to take a data loader can my test data loader be appended onto the “learn.dls” set, so I can then reference it using the “ds_idx” value? - I tried appending it but got a “‘L’ object is not callable” error.
The test_dl is associated with your original type transforms in your DataLoaders. So if you trained with your special image type it should also then build it from your passed in file name as well and make a new DataLoader for you. Try it, report back what happened
Just to add to this - test_dl is really a validation set internally in terms of transforms and image prep. So that is what will be applied to your test images.
I found this part confusing until I traced it through but test/validation set are the same transform wise.
Hope that helps.
That’s worked thanks! - so I supply it my data frame, containing the list of test images, and it uses this as it did with my training data.
I’d tried this before, but mistakenly was giving it the data loader I’d created for the test data, rather than simply supplying the data frame.
Thanks again.
2 Likes
Going mad here with predictions, what’s the proper way to inference for train\valid\test?
I have this, but it gives different results each time I run it:
def show_results(learn, start=0, n=5, detect_thresh=0.35, figsize=(10,25), ds="train"):
ds = learn.dls.train_ds
dl = learn.dls[0].new(shuffled=False, drop_last=False)
_,axs = plt.subplots(n, 2, figsize=figsize)
for b in range(start, n, BS):
x,_,_ = dl.one_batch()
with torch.no_grad():
output = learn.model.eval()(x)
...You should do ‘test_dl(learn.dls, items)’
And then do dl.one_batch() should work
And for train and valid?
Train is shuffled at the start, valid never is, so just index into them. (You don’t need to make another dl)
What I am trying to understand is why scores coming from the model are so different (from 0.2 to 0.6 after sigmoid) each time for the same x?
I checked them with
x,_,_ = learn.dls.train.one_batch()
with torch.no_grad():
output2 = learn.model.eval()(x)
display(torch.sigmoid(output2[0][0]).max(0)[0])
tensor([0.2594], device='cuda:0')
# second run
tensor([0.4380], device='cuda:0')
I noticed that x is different too, why?
x,_,_ = learn.dls[0].new(shuffled=False, drop_last=False).one_batch()
with torch.no_grad():
output2 = learn.model.eval()(x)
display(x[0])How to decode single test image?
I’ve created a test_dl using:
test_dl = dls.test_dl(test_imgs, with_labels=True)
Now, I want to get back exact original image from a tensor received by test_dl. Before that, I tried visualizing the same normalized tensor using:
t = TensorImage(test_dl.one_batch()[0][0]); t.show()
I can see there are two methods decode_batch and decode which I tried to apply on the above tensor t like so: test_dl.decode(t), but this didn’t help and caused following error-
TypeError Traceback (most recent call last)
<ipython-input-115-838fe5b43b94> in <module>()
----> 1 test_dl.decode(t)
5 frames
/usr/local/lib/python3.6/dist-packages/fastai2/data/core.py in decode(self, b)
71 if isinstance(f,Pipeline): f.split_idx=split_idx
72
---> 73 def decode(self, b): return self.before_batch.decode(to_cpu(self.after_batch.decode(self._retain_dl(b))))
74 def decode_batch(self, b, max_n=9, full=True): return self._decode_batch(self.decode(b), max_n, full)
75
/usr/local/lib/python3.6/dist-packages/fastai2/data/core.py in _retain_dl(self, b)
51 def _retain_dl(self,b):
52 if not getattr(self, '_types', None): self._one_pass()
---> 53 return retain_types(b, typs=self._types)
54
55 @delegates(DataLoader.new)
/usr/local/lib/python3.6/dist-packages/fastcore/dispatch.py in retain_types(new, old, typs)
166 def retain_types(new, old=None, typs=None):
167 "Cast each item of `new` to type of matching item in `old` if it's a superclass"
--> 168 if not is_listy(new): return retain_type(new, old, typs)
169 return type(new)(L(new, old, typs).map_zip(retain_type, cycled=True))
/usr/local/lib/python3.6/dist-packages/fastcore/dispatch.py in retain_type(new, old, typ)
160 typ = old if isinstance(old,type) else type(old)
161 # Do nothing the new type is already an instance of requested type (i.e. same type)
--> 162 if typ==NoneType or isinstance(new, typ): return new
163 return retain_meta(old, cast(new, typ))
164
/usr/local/lib/python3.6/dist-packages/fastcore/foundation.py in __eq__(self, b)
333 def __contains__(self,b): return b in self.items
334 def __invert__(self): return self._new(not i for i in self)
--> 335 def __eq__(self,b): return False if isinstance(b, (str,dict,set)) else all_equal(b,self)
336 def __repr__(self): return repr(self.items) if _is_array(self.items) else coll_repr(self)
337 def __mul__ (a,b): return a._new(a.items*b)
/usr/local/lib/python3.6/dist-packages/fastcore/imports.py in all_equal(a, b)
48 "Compares whether `a` and `b` are the same length and have the same contents"
49 if not is_iter(b): return False
---> 50 return all(equals(a_,b_) for a_,b_ in itertools.zip_longest(a,b))
51
52 def noop (x=None, *args, **kwargs):
TypeError: zip_longest argument #1 must support iteration
Decode only decodes a batch, not part of it. It will return a list of samples, so you can access individual images that way.
1 Like