data = ImageDataBunch.from_folder(path, valid_pct=0.2,size=224, bs=bs, ds_tfms = get_transforms())
Lets , ran I used the above mentioned code. How would i check the length of the training data and the valid data?
Also, while running the cnn_learner does the acurracy that it return is the accuracy on the training data or the valid data?
Accuracy is on the validation (what doesn’t get the weights updated on our model)
To get the length you can do len(data.train_dl) and valid_dl. Then multiply this by the batch size and that’s your number of instances. You may notice that the training may be a few images short, this is because we drop the last partial batch and also shuffle our data (only the training Dataloader only)
Thanks for explaining… Ummm… Also as i started training i got this error:
UnidentifiedImageError: Caught UnidentifiedImageError in DataLoader worker process 4.
Original Traceback (most recent call last):
File “E:\Conda\envs\fastai\lib\site-packages\torch\utils\data_utils\worker.py”, line 178, in _worker_loop
data = fetcher.fetch(index)
File “E:\Conda\envs\fastai\lib\site-packages\torch\utils\data_utils\fetch.py”, line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File “E:\Conda\envs\fastai\lib\site-packages\torch\utils\data_utils\fetch.py”, line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File “E:\Conda\envs\fastai\lib\site-packages\fastai\data_block.py”, line 655, in getitem
if self.item is None: x,y = self.x[idxs],self.y[idxs]
File “E:\Conda\envs\fastai\lib\site-packages\fastai\data_block.py”, line 120, in getitem
if isinstance(idxs, Integral): return self.get(idxs)
File “E:\Conda\envs\fastai\lib\site-packages\fastai\vision\data.py”, line 271, in get
res = self.open(fn)
File “E:\Conda\envs\fastai\lib\site-packages\fastai\vision\data.py”, line 267, in open
return open_image(fn, convert_mode=self.convert_mode, after_open=self.after_open)
File “E:\Conda\envs\fastai\lib\site-packages\fastai\vision\image.py”, line 398, in open_image
x = PIL.Image.open(fn).convert(convert_mode)
File “E:\Conda\envs\fastai\lib\site-packages\PIL\Image.py”, line 2896, in open
“cannot identify image file %r” % (filename if filename else fp)
PIL.UnidentifiedImageError: cannot identify image file ‘E:\fastai\data\PetImages\train\Dog\11702.jpg’
What does the above error mean?I am using the dogs and cats dataset from Microsoft…
Try opening that image in pillow. It could mean the image maybe corrupt. Can you open it yourself?
It’s not opening… Do i need tomanually delete the file from the dir