Import tensors as X and Y into DataBlock?

Part 1 (2019)
1

I am working on an audio classification project where I’m converting audio into spectrograms (1-channel images) and want the model to classify whether the spectrogram belongs to one of two classes. I perform this conversion on the fly, i.e. not saving any images locally, and have my input data now in form of a tensor with the shape: torch.Size([398709, 1, 128]) (398,709 training examples, 1 channel, 128 image size.)

I have my targets in form of a tensor, too, with shape: torch.Size([398709, 1]) (398,709 outputs of 0 or 1) I can also have this data in form of a list with a length of 398709 elements.

My question is: How to pass my data in form of tensors into Datasets or DataLoaders?

I have tried creating get_x and get_y functions like this (where x is the tensor with spectrograms and y the tensor with targets):

def get_x(x) : 
  return x

def get_y(x) :
  return y

But I’m getting

image
image743×31 2 KB

Creating a dataset using list(zip(x,y)), then creating a training DataLoader and a valid DataLoader but then I’m also getting errors, and nothing seems to work. Any help would be highly appreciated!

2

You are not showing how you are building your dataloader so …
This is example for doing txt processing with some Transforms for X and Y. Hope this can give you some hints…

from fastai.basics import *
from fastai.text.all import *

xs = ["Bad Dog", "Good Dog", "Mean Dog"]
ys = ["neg","pos", "neg"]
inps = L(zip(xs, ys))
print(inps)


class GetX(ItemTransform): 
    def encodes(self, x): return (x[0])
class GetY(ItemTransform): 
    def encodes(self, x): return (x[1])
    
    
wt = WordTokenizer()
tls = Datasets(inps, [(GetX, Tokenizer(wt), Numericalize(min_freq=1)), (GetY, Categorize)])

dls = tls.dataloaders(inps, batch_size=1, num_workers=0)
print("--------------------------------")
print(L(dls.train))

(#3) [('Bad Dog', 'neg'),('Good Dog', 'pos'),('Mean Dog', 'neg')]
--------------------------------
(#3) [(TensorText([[ 2,  8, 11,  8,  9]], device='cuda:0'), TensorCategory([1], device='cuda:0', dtype=torch.int32)),(TensorText([[ 2,  8, 12,  8,  9]], device='cuda:0'), TensorCategory([0], device='cuda:0', dtype=torch.int32)),(TensorText([[ 2,  8, 10,  8,  9]], device='cuda:0'), TensorCategory([0], device='cuda:0', dtype=torch.int32))]
3

Thanks for your fast reply Edward, that was very helpful! I have solved the issue by almost copying your solution from text processing to image processing and it worked! Not sure though, whether this approach is not too lame. Here’s my code in case someone else would have a similar problem. Make sure you have your X as a tensor with images of shape (image number, channels, height) and your Y as a list (tensor did not work, it created a vocab containing every output as a different category):

from fastai.vision.all import *

inps = L(zip(x,y_list))

class GetX(ItemTransform): 
    def encodes(self, x): return (x[0])
class GetY(ItemTransform): 
    def encodes(self, x): return (x[1])

dblock = DataBlock(blocks=(ImageBlock, CategoryBlock),
                   get_x=GetX,
                   get_y=GetY,
                   batch_tfms=Normalize.from_stats(*imagenet_stats)) # convert images from 1-channel to 3-channel

dls = dblock.dataloaders(inps, bs=512)
4

Right since CategoryBlock expects a list and will build a Dictionary and provide integers/tensors. You can also build your own CategoryMap and pass it into the CategoryBlock my_cat = CategoryMap(ys, add_na=True)
then create the DataBlock with CategoryBlock(vocab=my_cat)

If you really wanted to feed it int’s, you should be able to use

def TensorBlock():
    return TransformBlock(batch_tfms=IntToFloatTensor)
1 Like