ImageClassifierData.from_arrays

daveluo · March 5, 2018, 12:53am

Hi,

It looks like you’re missing the transforms (“tfms=”) field in your ImageClassifierData.from_arrays function call. Try:

data = ImageClassifierData.from_arrays(PATH, trn=(X_train, y_train), val=(X_val, y_val), tfms=tfms_from_model(arch, sz), bs=bs, classes=labels, num_workers=4, test=test)

My understanding is that even if you’re not doing any data augmentation, you still need to include tfms= for fast.ai library to properly reshape, resize, and normalize your inputs to what pytorch expects.

If this doesn’t resolve it, you may also need to triple that last dimension to “x3”. You can do this with np.stack of your array onto itself 3 times in the appropriate axis, e.g.:

array_3 = np.stack([array_1, array_1, array_1], axis=3)

You’re probably also getting the “dims of x and y do not match” AssertionError because you have triple the number of labels in your y_train vs X_train samples (and the same for y_val and X_val).

The trn and val dimensions to put into ImageClassiferData.from_arrays should probably look like:

X_train: (11200, 28, 28, 3)
X_val: (2800, 28, 28, 3)
y_train: (11200,)
y_val: (2800,)

EDIT: Worth mentioning that the RuntimeError you’re seeing is specifically saying that pytorch expected an input in the shape of [n, 3, x, y]. In other words, your 4-dimensional tensor is expected to have 3 channels (in the second axis) before your x, y data (in the 3rd and 4th axes). Fastai, via the transform, should roll your input axes from [n, x, y, num_channels] to the correct [n, num_channels, x, y] shape. If you’re still getting a similar error after the above changes, I guess you could try manually rolling your channel axis into the correct position and using that as the input:

X_train: (11200, 3, 28, 28)

chloews · March 9, 2018, 11:17pm

Thanks so much for your help Dave! I really appreciate it.

Your final comment about the input expecting a shape of [batch size, channels, height, width] is what the issue is. Also, I was re-watching one of the lessons, and Jeremy confirms that PyTorch’s tensor dims are a different order from other libraries that typically put channels at the back (i.e., [batch size, height, width, channels]), so I need to reshape my array accordingly.

Thanks again!

dortonway · March 17, 2018, 2:29pm

Hi

@daveluo, can you, please, tell me, how trn in from_array() looks like? As far as I understand, it’s a tuple of two matrix: the second is an array of labels, the first is a matrix of: each row represents according label in the second matrix, each column represents 1 image. But what is image? Is that a buffer, or path to the image (as I can see in your post it’s some numbers, so - no), or what?

Thanks!

daveluo · March 17, 2018, 5:22pm

Hi @dortonway,

Here’s what my ImageClassifierData.from_arrays() function call looks like:

arch=resnet50
bs=16
sz=75
data = ImageClassifierData.from_arrays(path=RESULTS, trn=(x_train, y_train), val=(x_holdout, y_holdout), tfms=tfms_from_model(arch, sz), bs=bs, classes=2)

You’re right that trn= should be a tuple of 2 ndarrays (x_train, y_train):

x_train is a 4-dimensional array in the shape (# samples, # rows, # columns, # channels) where (# rows, # columns) is the 2D pixel representation of each image in size sz x sz in a single channel.
y_train is a 1-dimensional array storing the respective labels for x_train in shape (# samples,)

For example, in my dataset, x_train.shape, y_train.shape shows

((1176, 75, 75, 3), (1176,))

meaning 1176 training samples of 75x75pixel images in 3 channels, with 1176 corresponding y labels.

The 75x75 part of the 1st sample in the 1st channel (x_train[0,:,:,0]) is the standard deviation-normalized 2D array of the original (or resized to) 75x75 pixel image and looks like:

array([[ 0.42137, -0.43078, -0.4308 , ...,  0.93337,  0.51248, -0.25854],
       [ 0.16545, -0.1627 ,  0.26667, ...,  0.73933,  0.56651, -0.25854],
       [-0.5351 , -0.16272,  0.30614, ..., -0.67258,  0.01582, -0.28268],
       ...,
       [-0.64376, -1.06065, -1.26056, ...,  0.56625,  0.85387,  0.40087],
       [-0.38035, -0.6163 , -0.56212, ...,  1.20078,  1.75155,  1.17225],
       [ 0.12359, -0.6163 , -0.25752, ...,  0.78885,  1.69286,  1.55917]], dtype=float32)

When displayed with plt.imshow(x_train[0,:,:,0]), looks like:

<matplotlib.image.AxesImage at 0x7fadee351c18>

(this is a synthetic-aperture radar (SAR) satellite image of a possible iceberg or ship from the Kaggle statoil/c-core iceberg classifier challenge)

Hope that helps clear things up!

sahilk1610 · May 19, 2018, 4:20pm

@mmr how did you solve this problem?

This is the code I use:

arch=resnet50
bs = 16
sz = 75
tfms = tfms_from_model(arch, sz, aug_tfms=transforms_top_down, max_zoom=1.05)
data = ImageClassifierData.from_arrays(PATH, trn = (x_train, y_train), val = (x_val, y_val), tfms = tfms , bs = bs, classes = 2, test = X_test)
learn = ConvLearner.pretrained(arch, data, precompute=True)

And I get a similar error KeyError:

Summary

0%| | 0/81 [00:00<?, ?it/s]

KeyError Traceback (most recent call last)
in ()
4 tfms = tfms_from_model(arch, sz, aug_tfms=transforms_top_down, max_zoom=1.05)
5 data = ImageClassifierData.from_arrays(PATH, trn = (x_train, y_train), val = (x_val, y_val), tfms = tfms , bs = bs, classes = 2, test = X_test)
----> 6 learn = ConvLearner.pretrained(arch, data, precompute=True)

~/fastai/courses/dl1/fastai/conv_learner.py in pretrained(cls, f, data, ps, xtra_fc, xtra_cut, custom_head, precompute, pretrained, **kwargs)
112 models = ConvnetBuilder(f, data.c, data.is_multi, data.is_reg,
113 ps=ps, xtra_fc=xtra_fc, xtra_cut=xtra_cut, custom_head=custom_head, pretrained=pretrained)
–> 114 return cls(data, models, precompute, **kwargs)
115
116 @classmethod

~/fastai/courses/dl1/fastai/conv_learner.py in init(self, data, models, precompute, **kwargs)
98 if hasattr(data, ‘is_multi’) and not data.is_reg and self.metrics is None:
99 self.metrics = [accuracy_thresh(0.5)] if self.data.is_multi else [accuracy]
–> 100 if precompute: self.save_fc1()
101 self.freeze()
102 self.precompute = precompute

~/fastai/courses/dl1/fastai/conv_learner.py in save_fc1(self)
166 m=self.models.top_model
167 if len(self.activations[0])!=len(self.data.trn_ds):
–> 168 predict_to_bcolz(m, self.data.fix_dl, act)
169 if len(self.activations[1])!=len(self.data.val_ds):
170 predict_to_bcolz(m, self.data.val_dl, val_act)

~/fastai/courses/dl1/fastai/model.py in predict_to_bcolz(m, gen, arr, workers)
15 lock=threading.Lock()
16 m.eval()
—> 17 for x,*_ in tqdm(gen):
18 y = to_np(m(VV(x)).data)
19 with lock:

~/anaconda3/envs/fastai/lib/python3.6/site-packages/tqdm/_tqdm.py in iter(self)
895 “”", fp_write=getattr(self.fp, ‘write’, sys.stderr.write))
896
–> 897 for obj in iterable:
898 yield obj
899 # Update and possibly print the progressbar.

~/fastai/courses/dl1/fastai/dataloader.py in iter(self)
86 # avoid py3.6 issue where queue is infinite and can result in memory exhaustion
87 for c in chunk_iter(iter(self.batch_sampler), self.num_workers*10):
—> 88 for batch in e.map(self.get_batch, c):
89 yield get_tensor(batch, self.pin_memory, self.half)
90

~/anaconda3/envs/fastai/lib/python3.6/concurrent/futures/_base.py in result_iterator()
584 # Careful not to keep a reference to the popped future
585 if timeout is None:
–> 586 yield fs.pop().result()
587 else:
588 yield fs.pop().result(end_time - time.time())

~/anaconda3/envs/fastai/lib/python3.6/concurrent/futures/_base.py in result(self, timeout)
423 raise CancelledError()
424 elif self._state == FINISHED:
–> 425 return self.__get_result()
426
427 self._condition.wait(timeout)

~/anaconda3/envs/fastai/lib/python3.6/concurrent/futures/_base.py in __get_result(self)
382 def __get_result(self):
383 if self._exception:
–> 384 raise self._exception
385 else:
386 return self._result

~/anaconda3/envs/fastai/lib/python3.6/concurrent/futures/thread.py in run(self)
54
55 try:
—> 56 result = self.fn(*self.args, **self.kwargs)
57 except BaseException as exc:
58 self.future.set_exception(exc)

~/fastai/courses/dl1/fastai/dataloader.py in get_batch(self, indices)
73
74 def get_batch(self, indices):
—> 75 res = self.np_collate([self.dataset[i] for i in indices])
76 if self.transpose: res[0] = res[0].T
77 if self.transpose_y: res[1] = res[1].T

~/fastai/courses/dl1/fastai/dataloader.py in (.0)
73
74 def get_batch(self, indices):
—> 75 res = self.np_collate([self.dataset[i] for i in indices])
76 if self.transpose: res[0] = res[0].T
77 if self.transpose_y: res[1] = res[1].T

~/fastai/courses/dl1/fastai/dataset.py in getitem(self, idx)
165 xs,ys = zip(*[self.get1item(i) for i in range(*idx.indices(self.n))])
166 return np.stack(xs),ys
–> 167 return self.get1item(idx)
168
169 def len(self): return self.n

~/fastai/courses/dl1/fastai/dataset.py in get1item(self, idx)
158
159 def get1item(self, idx):
–> 160 x,y = self.get_x(idx),self.get_y(idx)
161 return self.get(self.transform, x, y)
162

~/fastai/courses/dl1/fastai/dataset.py in get_y(self, i)
289 class ArraysIndexDataset(ArraysDataset):
290 def get_c(self): return int(self.y.max())+1
–> 291 def get_y(self, i): return self.y[i]
292
293

~/anaconda3/envs/fastai/lib/python3.6/site-packages/pandas/core/series.py in getitem(self, key)
621 key = com._apply_if_callable(key, self)
622 try:
–> 623 result = self.index.get_value(self, key)
624
625 if not is_scalar(result):

~/anaconda3/envs/fastai/lib/python3.6/site-packages/pandas/core/indexes/base.py in get_value(self, series, key)
2558 try:
2559 return self._engine.get_value(s, k,
-> 2560 tz=getattr(series.dtype, ‘tz’, None))
2561 except KeyError as e1:
2562 if len(self) > 0 and self.inferred_type in [‘integer’, ‘boolean’]:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_value()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_value()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item()

KeyError: 16

As mentioned by Jeremy that we have a pandas dataframe and not a numpy array, but in the data object we create, we put in the array which we get from the train_test_split. so where does the dataframe come into picture ?

mmr · May 20, 2018, 2:16am

@sahilk1610 - There was an issue regarding the way I was taking in the input data. I am sorry it was sometime ago, and this is purely from my recollection.

fredguth · May 30, 2018, 6:16pm

If I have a dataset with images of different sizes, instead of X_train.shape == (11200, 28, 28, 3) I will have something like (11200,) and the function doesn’t work (even with tfms set).

I guess I will have to resize the images before calling the function, but shouldn’t do it for me automatically?

fredguth · May 30, 2018, 7:12pm

Yeah, I just had to resize it myself.

Woodstock · June 1, 2018, 4:51am

I wonder how we can use

ImageClassifierData.from_arrays(continuous=True)

just like ImageClassifierData.from_csv?

I have X-rays datasets that try to predict some motor scores (continuous data) , but the images are DICOM format, so I have to use pydicom convert to numpy array.