[SOLVED] Lesson 2: errors while trying to train model with Google Images

Part 1 (2019) UTC-8 Pacific
1

I got as far as learn.fit_one_cycle(4) but this is the output I got. Any help or suggestions greatly appreciated.

 0.00% [0/4 00:00<00:00]
epoch	train_loss	valid_loss	error_rate	time

 Interrupted
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-34-495233eaf2b4> in <module>
----> 1 learn.fit_one_cycle(4)

/opt/anaconda3/lib/python3.7/site-packages/fastai/train.py in fit_one_cycle(learn, cyc_len, max_lr, moms, div_factor, pct_start, final_div, wd, callbacks, tot_epochs, start_epoch)
     20     callbacks.append(OneCycleScheduler(learn, max_lr, moms=moms, div_factor=div_factor, pct_start=pct_start,
     21                                        final_div=final_div, tot_epochs=tot_epochs, start_epoch=start_epoch))
---> 22     learn.fit(cyc_len, max_lr, wd=wd, callbacks=callbacks)
     23 
     24 def lr_find(learn:Learner, start_lr:Floats=1e-7, end_lr:Floats=10, num_it:int=100, stop_div:bool=True, wd:float=None):

/opt/anaconda3/lib/python3.7/site-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks)
    189         if defaults.extra_callbacks is not None: callbacks += defaults.extra_callbacks
    190         fit(epochs, self.model, self.loss_func, opt=self.opt, data=self.data, metrics=self.metrics,
--> 191             callbacks=self.callbacks+callbacks)
    192 
    193     def create_opt(self, lr:Floats, wd:Floats=0.)->None:

/opt/anaconda3/lib/python3.7/site-packages/fastai/basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
     90             cb_handler.set_dl(data.train_dl)
     91             cb_handler.on_epoch_begin()
---> 92             for xb,yb in progress_bar(data.train_dl, parent=pbar):
     93                 xb, yb = cb_handler.on_batch_begin(xb, yb)
     94                 loss = loss_batch(model, xb, yb, loss_func, opt, cb_handler)

/opt/anaconda3/lib/python3.7/site-packages/fastprogress/fastprogress.py in __iter__(self)
     64         self.update(0)
     65         try:
---> 66             for i,o in enumerate(self._gen):
     67                 yield o
     68                 if self.auto_update: self.update(i+1)

/opt/anaconda3/lib/python3.7/site-packages/fastai/basic_data.py in __iter__(self)
     73     def __iter__(self):
     74         "Process and returns items from `DataLoader`."
---> 75         for b in self.dl: yield self.proc_batch(b)
     76 
     77     @classmethod

/opt/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py in __next__(self)
    621         if self.rcvd_idx in self.reorder_dict:
    622             batch = self.reorder_dict.pop(self.rcvd_idx)
--> 623             return self._process_next_batch(batch)
    624 
    625         if self.batches_outstanding == 0:

/opt/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py in _process_next_batch(self, batch)
    656         self._put_indices()
    657         if isinstance(batch, ExceptionWrapper):
--> 658             raise batch.exc_type(batch.exc_msg)
    659         return batch
    660 

OSError: Traceback (most recent call last):
  File "/opt/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in _worker_loop
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/opt/anaconda3/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in <listcomp>
    samples = collate_fn([dataset[i] for i in batch_indices])
  File "/opt/anaconda3/lib/python3.7/site-packages/fastai/data_block.py", line 625, in __getitem__
    if self.item is None: x,y = self.x[idxs],self.y[idxs]
  File "/opt/anaconda3/lib/python3.7/site-packages/fastai/data_block.py", line 106, in __getitem__
    if isinstance(idxs, Integral): return self.get(idxs)
  File "/opt/anaconda3/lib/python3.7/site-packages/fastai/vision/data.py", line 271, in get
    res = self.open(fn)
  File "/opt/anaconda3/lib/python3.7/site-packages/fastai/vision/data.py", line 267, in open
    return open_image(fn, convert_mode=self.convert_mode, after_open=self.after_open)
  File "/opt/anaconda3/lib/python3.7/site-packages/fastai/vision/image.py", line 393, in open_image
    x = PIL.Image.open(fn).convert(convert_mode)
  File "/opt/anaconda3/lib/python3.7/site-packages/PIL/Image.py", line 2657, in open
    % (filename if filename else fp))
OSError: cannot identify image file '/home/jupyter/tutorials/fastai/course-v3/nbs/dl1/data/cars/corvette/00000187.jpg'
2

Hello,

Could you share more information on which platform you’re using and your code?

From the information we have, since you have a valid image file you might want to check if

  • You have read access to the image file: if not the model wouldn’t be able to train on it.
  • The image file is corrupted: can you open and view the image?
  • You accidentally moved the image to another destination: by following the file path in the last line are you able to find your image?
3

Using Google Cloud Platform. It took a while for me to dig through the error messages and realize that it was just that one photo causing the issue. I deleted it, and everything worked fine. I guess it’s the case that verify_images can sometimes miss photos that are corrupted or have some kind of issue?

Thanks for your reply @fawl. Next time I will be sure to attach a github link to my code.

4

No problem man! Glad I could help. And yeah, I’ve found that verify_images can be a bit buggy sometimes, although it works pretty well majority of the time.

1 Like