Yeah, I noticed that too, experimentally. Worse than that, it will often improve the accuracy over the (same) valid set, but your model will silently lose generalization power, and you won’t notice until deployment, unless you have a large (and statistically representative) test set.
2 Likes
I see, I didn’t know that. Thank you. That is a hard one to solve. I have to experiment with this 
1 Like
Adult tabular dataset: if you took the course before, you may have outdated dataset.
I came across an error that ‘salary’ is not an index in the data. It turned out my dataset was from Nov. 2018 and did not contain ‘salary’. So I deleted it from ~/.fastai/data and ran it again and it worked!
1 Like
this question is not really related to lesson one but I don’t want to start a new topic.
what’s the best way get the labels for the predictions got from learn.get_preds()? Is there something build in fastai?!
right now I’m doing this:
dl = learn.dls.test_dl(imgs)
preds,_,dec_preds = learn.get_preds(dl=dl, with_decoded=True)
lbl_preds = [learn.dls.vocab[p] for p in dec_preds]
I tried to understand the code used in learn.predict() - which returns the real labels - but I don’t really get what’s going on there… too many maps decodes tuplify and detuplifys for me 
What’s the best practice for saving predictions per batch? I know there’s the save_preds method for learners, but to my knowledge it just gets the predictions of all items in your dataset/dataloader then you have to go through each prediction and save it to file one-by-one.
Look at the tail end of learn.predict()’s code, specifically at the decode_batch() function. It’ll wind up returning your original X (the image) and the now fully decoded y
For a code example, see the “fastai straight” section here:
1 Like
Dear all,
I update my own dataset in Paperspace
from fastai2.vision.all import *
path = "/notebooks/storage/course-v4/nbs/images/dataset"
def is_cube(x): return x[0].startswith("c")
dls = ImageDataLoaders.from_name_func(
path, get_image_files(path), valid_pct=0.2, seed=42,
label_func=is_cube, item_tfms=Resize(224))
learn = cnn_learner(dls, resnet34, metrics=error_rate)
learn.fine_tune(4)
In the dataset folder I have only 20 pictures in .jpg format
10 cat A and 10 in cat B
Any idea why it is all nan and the values are all wrong?
@Albertotono first thing to do when debugging this situation is look at your dataloaders. What are their lengths? (how many batches)? You can explore this via len(dls.train) and len(dls.valid). Now what I can tell you is your training dataloader has zero batches. Why? The batch size (by default) is 64 and you only have 20 images. As such we can only make one batch, and here it made a batch of validation data. If you truly only want to use that low amount of images you should bring your batch size to 2 to see some data in each dataloader. Also what Sylvain says about the dropped batch below 
1 Like
You have a default batch size of 64 and in the training set, the last batch is dropped if it doesn’t have batch_size elements. So you’re not training on anything.
To adjust the batch size, pass bs=... in ImageDataLoaders
2 Likes
@muellerzr @sgugger you are awesome, thank you so much.
with DataAugmentation
it seems that I am doing something wrong, probably 20 images it not enough.
probably we do not need a lot of data, but we need a good amount lol
I looked at SinGAN paper
I am trying to build the DataBlocks API for MNIST and trying to figure out why
path = untar_data(URLs.MNIST_TINY)
dblock = DataBlock(blocks=(ImageBlock, CategoryBlock),
get_items=get_image_files,
get_x=PILImage.create,
get_y=parent_label,
splitter=GrandparentSplitter())
This API doesn’t work when I do dls=dblock.dataloaders(path) .
Here is the stack trace:
AttributeError Traceback (most recent call last)
<ipython-input-59-0353eb4f73af> in <module>
----> 1 dls = dblock.dataloaders(path)
~/repos/fastai2/fastai2/data/block.py in dataloaders(self, source, path, verbose, **kwargs)
96
97 def dataloaders(self, source, path='.', verbose=False, **kwargs):
---> 98 dsets = self.datasets(source)
99 kwargs = {**self.dls_kwargs, **kwargs, 'verbose': verbose}
100 return dsets.dataloaders(path=path, after_item=self.item_tfms, after_batch=self.batch_tfms, **kwargs)
~/repos/fastai2/fastai2/data/block.py in datasets(self, source, verbose)
93 splits = (self.splitter or RandomSplitter())(items)
94 pv(f"{len(splits)} datasets of sizes {','.join([str(len(s)) for s in splits])}", verbose)
---> 95 return Datasets(items, tfms=self._combine_type_tfms(), splits=splits, dl_type=self.dl_type, n_inp=self.n_inp, verbose=verbose)
96
97 def dataloaders(self, source, path='.', verbose=False, **kwargs):
~/repos/fastai2/fastai2/data/core.py in __init__(self, items, tfms, tls, n_inp, dl_type, **kwargs)
272 def __init__(self, items=None, tfms=None, tls=None, n_inp=None, dl_type=None, **kwargs):
273 super().__init__(dl_type=dl_type)
--> 274 self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
275 self.n_inp = (1 if len(self.tls)==1 else len(self.tls)-1) if n_inp is None else n_inp
276
~/repos/fastai2/fastai2/data/core.py in <listcomp>(.0)
272 def __init__(self, items=None, tfms=None, tls=None, n_inp=None, dl_type=None, **kwargs):
273 super().__init__(dl_type=dl_type)
--> 274 self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
275 self.n_inp = (1 if len(self.tls)==1 else len(self.tls)-1) if n_inp is None else n_inp
276
~/repos/fastai2/fastcore/fastcore/foundation.py in __call__(cls, x, *args, **kwargs)
39 return x
40
---> 41 res = super().__call__(*((x,) + args), **kwargs)
42 res._newchk = 0
43 return res
~/repos/fastai2/fastai2/data/core.py in __init__(self, items, tfms, use_list, do_setup, split_idx, train_setup, splits, types, verbose)
212 if do_setup:
213 pv(f"Setting up {self.tfms}", verbose)
--> 214 self.setup(train_setup=train_setup)
215
216 def _new(self, items, **kwargs): return super()._new(items, tfms=self.tfms, do_setup=False, types=self.types, **kwargs)
~/repos/fastai2/fastai2/data/core.py in setup(self, train_setup)
232 for f in self.tfms.fs:
233 self.types.append(getattr(f, 'input_types', type(x)))
--> 234 x = f(x)
235 self.types.append(type(x))
236 types = L(t if is_listy(t) else [t] for t in self.types).concat().unique()
~/repos/fastai2/fastcore/fastcore/transform.py in __call__(self, x, **kwargs)
70 @property
71 def name(self): return getattr(self, '_name', _get_name(self))
---> 72 def __call__(self, x, **kwargs): return self._call('encodes', x, **kwargs)
73 def decode (self, x, **kwargs): return self._call('decodes', x, **kwargs)
74 def __repr__(self): return f'{self.name}: {self.encodes} {self.decodes}'
~/repos/fastai2/fastcore/fastcore/transform.py in _call(self, fn, x, split_idx, **kwargs)
80 def _call(self, fn, x, split_idx=None, **kwargs):
81 if split_idx!=self.split_idx and self.split_idx is not None: return x
---> 82 return self._do_call(getattr(self, fn), x, **kwargs)
83
84 def _do_call(self, f, x, **kwargs):
~/repos/fastai2/fastcore/fastcore/transform.py in _do_call(self, f, x, **kwargs)
84 def _do_call(self, f, x, **kwargs):
85 if not _is_tuple(x):
---> 86 return x if f is None else retain_type(f(x, **kwargs), x, f.returns_none(x))
87 res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
88 return retain_type(res, x)
~/repos/fastai2/fastcore/fastcore/dispatch.py in __call__(self, *args, **kwargs)
96 if not f: return args[0]
97 if self.inst is not None: f = MethodType(f, self.inst)
---> 98 return f(*args, **kwargs)
99
100 def __get__(self, inst, owner):
~/repos/fastai2/fastai2/vision/core.py in create(cls, fn, **kwargs)
96 if isinstance(fn,ndarray): return cls(Image.fromarray(fn))
97 if isinstance(fn,bytes): fn = io.BytesIO(fn)
---> 98 return cls(load_image(fn, **merge(cls._open_args, kwargs)))
99
100 def show(self, ctx=None, **kwargs):
AttributeError: 'DataBlock' object has no attribute '_open_args'
Rather:
dblock = DataBlock(blocks=(ImageBlock, CategoryBlock),
get_items=get_image_files,
get_y=parent_label,
splitter=GrandparentSplitter())
Works.
ImageBlock points out to this code fastai2/vision/core.py
_bypass_type=Image.Image
_show_args = {'cmap':'viridis'}
_open_args = {'mode': 'RGB'}
@classmethod
def create(cls, fn:(Path,str,Tensor,ndarray,bytes), **kwargs)->None:
"Open an `Image` from path `fn`"
if isinstance(fn,TensorImage): fn = fn.permute(1,2,0).type(torch.uint8)
if isinstance(fn,Tensor): fn = fn.numpy()
if isinstance(fn,ndarray): return cls(Image.fromarray(fn))
if isinstance(fn,bytes): fn = io.BytesIO(fn)
return cls(load_image(fn, **merge(cls._open_args, kwargs)))
In the create function, it handles certain types like TensorImage, ndarray. In the example, you pointed the output of get_x would be passed to ImageBlock which is already an image. If you do not use PILImage.create then just file path to the image is passed to the Image block and the lask line of create function gets executed.
1 Like
Thanks, I understand it now.
@arora_aman also FYI, you should pass into ImageBlock saying you’re doing one channel images like so: ImageBlock(cls=PILImageBW) otherwise if you want to do get_x, make sure you use BW
Mnist is a 1channel image. But has two dimension height and width. 32x32x1. A rgb image is 3channels 32x32x3.
2 Likes
Thanks @muellerzr and @barnacl . Not passing PILImageBW works too because it converts my images to 3 channel (by copying one channel across the others), I find this beneficial because then I don’t need to update model for single channel.
(PILImage mode=RGB size=28x28, TensorCategory(1))
FYI, this is an example of dl.train_ds[0]
2 Likes
It’s just a different approach I think. I don’t know much about this too tbh but when I participated in BengaliAI, Kagglers got top scores both by using single channel or multi channel images for Black and White handwritten graphemes images.
I don’t know when to use one before the other.
1 Like
I find using 3 channel intutive, as imagenet models are trained on images with 3 channels. In some of my experiments I found 3 channel to work better even for BW images.
3 Likes
I can confirm this from my experience too.
3 Likes