Hi,
I created 5 classes of fruits and uploaded to my GCP based fastai setup as below.
I just have train folder and do not have valid and test folders (which I believe are optional).
But whenever I try to create the ImageDataBunch as below,
sPath = '/home/jupyter/tutorials/fastai/course-v3/data/Fruits' data = ImageDataBunch.from_folder(path=sPath, valid_pct='20', size=224)
I get the error
IndexError: index 0 is out of bounds for axis 1 with size 0
Can anyone help, please?
This worked.
data = (ImageItemList.from_folder(sPath).random_split_by_pct().label_from_folder().transform(tfms, size=224).databunch())
Hi krisho007, I have exacty the same problem as yours [IndexError: index 0 is out of bounds for axis 1 with size 0 ] . My data folder structure is similar to yours as well, but I can’t seem to solve it with the method you mentioned using the ImageItemList… can you elaborate more details how you replace from data=ImageDataBunch.from_folder to data=ImageItemList.from_folder? Thanks.
Can you post the output of print(sPath) and print(ImageItemList.from_folder(sPath))?
Your issue is most probably with having the right sPath. If it cannot find any data in the sPath, then you will end up having same error.
Check out the docs.
https://docs.fast.ai/vision.data.html#ImageDataBunch.from_folder
I am sure that you can spot the difference between your setup and the correct ImageNet style setup. 
Just realize my error is slightly different than yours, mine was like " IndexError: index 0 is out of bounds for axis 0 with size 0" My error was at axis 0 not axis 1 as yours.
After multiple attempts checking posts and docs, I still couldn’t rectify the error. So I did a silly trial by open up a new notebook, copy all the scripts over line by line, and it run FINE with the ImageDataBunch <@@> ! So basically I still don’t know what causes the ImageDataBunch not running with the original notebook… but since I can continue with the rest of the steps afterward with the new notebook, I proceed as is. Just in case anyone having problem like this, probably the best way is to ‘reset’ it with new notebook 
and your day is still 
afterward than crunching the codes again and again 
but still doesn’t work…
I found that setting valid_pct to a value fixed the error when I got it. The method seems to be looking for train, validation, and testing folders, but if you have the data split by class in folders then you’ll probably need to tell the method how to split those images.
Came here because of a very similar error message that doesn’t seem to be very related though:
index 3 is out of bounds for dimension 0 with size 3
For me it shows up when trying to perform a
learn.fit_one_cylce(1)
after successfully loading a (new, fastai v2) ImageDataLoaders. So maybe this hints at the error message being somewhat generic?
@tokleine
This is usually because of a mismatch in target classes, and the labels in the data. There are only 3 categories in your data, if I’m not wrong. However, one of the data points(or more) has a 4th label as well. You’ll need to rectify that, and it should be good to go!
Cheers!
@PalaashAgrawal
Thanks for responding. I have 10 target classes (digits 0-9) and my folder structure is
some/path/generated_images/train/0/0_digit_001.png
some/path/generated_images/valid/0/0_digit_001.png
I am unsure how to use your feedback to fix this issue. Could you maybe elaborate?
Can you please post the entire error message?
This is the entire error message.
`---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
in
1 learn = cnn_learner(data, models.resnet34)#,normalize=True)#, metrics=[error_rate])#, loss_func=LabelSmoothingCrossEntropy)
----> 2 learn.fit_one_cycle(1)
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastcore/utils.py in _f(*args, **kwargs)
452 init_args.update(log)
453 setattr(inst, ‘init_args’, init_args)
–> 454 return inst if to_return else f(*args, **kwargs)
455 return _f
456
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/callback/schedule.py in fit_one_cycle(self, n_epoch, lr_max, div, div_final, pct_start, wd, moms, cbs, reset_opt)
111 scheds = {‘lr’: combined_cos(pct_start, lr_max/div, lr_max, lr_max/div_final),
112 ‘mom’: combined_cos(pct_start, *(self.moms if moms is None else moms))}
–> 113 self.fit(n_epoch, cbs=ParamScheduler(scheds)+L(cbs), reset_opt=reset_opt, wd=wd)
114
115 # Cell
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastcore/utils.py in _f(*args, **kwargs)
452 init_args.update(log)
453 setattr(inst, ‘init_args’, init_args)
–> 454 return inst if to_return else f(*args, **kwargs)
455 return _f
456
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in fit(self, n_epoch, lr, wd, cbs, reset_opt)
205 self.opt.set_hypers(lr=self.lr if lr is None else lr)
206 self.n_epoch,self.loss = n_epoch,tensor(0.)
–> 207 self._with_events(self._do_fit, ‘fit’, CancelFitException, self._end_cleanup)
208
209 def _end_cleanup(self): self.dl,self.xb,self.yb,self.pred,self.loss = None,(None,),(None,),None,None
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in with_events(self, f, event_type, ex, final)
153
154 def with_events(self, f, event_type, ex, final=noop):
–> 155 try: self(f’before{event_type}’) ;f()
156 except ex: self(f’after_cancel{event_type}’)
157 finally: self(f’after_{event_type}’) ;final()
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in _do_fit(self)
195 for epoch in range(self.n_epoch):
196 self.epoch=epoch
–> 197 self._with_events(self._do_epoch, ‘epoch’, CancelEpochException)
198
199 @log_args(but=‘cbs’)
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in with_events(self, f, event_type, ex, final)
153
154 def with_events(self, f, event_type, ex, final=noop):
–> 155 try: self(f’before{event_type}’) ;f()
156 except ex: self(f’after_cancel{event_type}’)
157 finally: self(f’after_{event_type}’) ;final()
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in _do_epoch(self)
189
190 def _do_epoch(self):
–> 191 self._do_epoch_train()
192 self._do_epoch_validate()
193
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in _do_epoch_train(self)
181 def _do_epoch_train(self):
182 self.dl = self.dls.train
–> 183 self._with_events(self.all_batches, ‘train’, CancelTrainException)
184
185 def _do_epoch_validate(self, ds_idx=1, dl=None):
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in with_events(self, f, event_type, ex, final)
153
154 def with_events(self, f, event_type, ex, final=noop):
–> 155 try: self(f’before{event_type}’) ;f()
156 except ex: self(f’after_cancel{event_type}’)
157 finally: self(f’after_{event_type}’) ;final()
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in all_batches(self)
159 def all_batches(self):
160 self.n_iter = len(self.dl)
–> 161 for o in enumerate(self.dl): self.one_batch(*o)
162
163 def _do_one_batch(self):
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in one_batch(self, i, b)
177 self.iter = i
178 self._split(b)
–> 179 self._with_events(self._do_one_batch, ‘batch’, CancelBatchException)
180
181 def _do_epoch_train(self):
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in with_events(self, f, event_type, ex, final)
153
154 def with_events(self, f, event_type, ex, final=noop):
–> 155 try: self(f’before{event_type}’) ;f()
156 except ex: self(f’after_cancel{event_type}’)
157 finally: self(f’after_{event_type}’) ;final()
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in call(self, event_name)
131 def ordered_cbs(self, event): return [cb for cb in sort_by_run(self.cbs) if hasattr(cb, event)]
132
–> 133 def call(self, event_name): L(event_name).map(self._call_one)
134
135 def _call_one(self, event_name):
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastcore/foundation.py in map(self, f, *args, **kwargs)
381 else f.format if isinstance(f,str)
382 else f.getitem)
–> 383 return self._new(map(g, self))
384
385 def filter(self, f, negate=False, **kwargs):
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastcore/foundation.py in _new(self, items, *args, **kwargs)
331 @property
332 def _xtra(self): return None
–> 333 def _new(self, items, *args, **kwargs): return type(self)(items, *args, use_list=None, **kwargs)
334 def getitem(self, idx): return self._get(idx) if is_indexer(idx) else L(self._get(idx), use_list=None)
335 def copy(self): return self._new(self.items.copy())
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastcore/foundation.py in call(cls, x, args, **kwargs)
45 return x
46
—> 47 res = super().call(((x,) + args), **kwargs)
48 res._newchk = 0
49 return res
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastcore/foundation.py in init(self, items, use_list, match, *rest)
322 if items is None: items = []
323 if (use_list is not None) or not _is_array(items):
–> 324 items = list(items) if use_list else _listify(items)
325 if match is not None:
326 if is_coll(match): match = len(match)
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastcore/foundation.py in _listify(o)
258 if isinstance(o, list): return o
259 if isinstance(o, str) or _is_array(o): return [o]
–> 260 if is_iter(o): return list(o)
261 return [o]
262
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastcore/foundation.py in call(self, *args, **kwargs)
224 if isinstance(v,_Arg): kwargs[k] = args.pop(v.i)
225 fargs = [args[x.i] if isinstance(x, _Arg) else x for x in self.pargs] + args[self.maxi+1:]
–> 226 return self.fn(*fargs, **kwargs)
227
228 # Cell
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in _call_one(self, event_name)
135 def _call_one(self, event_name):
136 assert hasattr(event, event_name), event_name
–> 137 [cb(event_name) for cb in sort_by_run(self.cbs)]
138
139 def _bn_bias_state(self, with_bias): return norm_bias_params(self.model, with_bias).map(self.opt.state)
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/learner.py in (.0)
135 def _call_one(self, event_name):
136 assert hasattr(event, event_name), event_name
–> 137 [cb(event_name) for cb in sort_by_run(self.cbs)]
138
139 def _bn_bias_state(self, with_bias): return norm_bias_params(self.model, with_bias).map(self.opt.state)
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/callback/core.py in call(self, event_name)
42 (self.run_valid and not getattr(self, ‘training’, False)))
43 res = None
—> 44 if self.run and _run: res = getattr(self, event_name, noop)()
45 if event_name==‘after_fit’: self.run=True #Reset self.run to True at each end of fit
46 return res
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/callback/schedule.py in before_batch(self)
84 def init(self, scheds): self.scheds = scheds
85 def before_fit(self): self.hps = {p:[] for p in self.scheds.keys()}
—> 86 def before_batch(self): self._update_val(self.pct_train)
87
88 def _update_val(self, pct):
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/callback/schedule.py in _update_val(self, pct)
87
88 def _update_val(self, pct):
—> 89 for n,f in self.scheds.items(): self.opt.set_hyper(n, f(pct))
90
91 def after_batch(self):
/local_disk0/.ephemeral_nfs/envs/pythonEnv-52aca8ad-c2d8-47c4-bbdd-fd0e2658d08d/lib/python3.7/site-packages/fastai/callback/schedule.py in _inner(pos)
67 if pos == 1.: return scheds-1
68 idx = (pos >= pcts).nonzero().max()
—> 69 actual_pos = (pos-pcts[idx]) / (pcts[idx+1]-pcts[idx])
70 return schedsidx
71 return _inner
IndexError: index 3 is out of bounds for dimension 0 with size 3
`
That should be fixed now - please ensure you have the latest version of fastai.
Apologies for the bug!