Fastai: Multi-value regression with images as input

Hello

I hope this is the right subforum to ask fastai questions not related to the course.

I’d like to read in a dataset for a problem with images as input and multiple float values as output, i.e. a multi-value regression. Those values are contained in a csv file and the matching between the images and those values is done using a common time stamp, i.e. the image “[timestamp].jpg” has corresponding output values float_value_1, float_value_2 etc. listed in the line starting with [timestamp] in the csv file.

How can I use the data block API to create that dataset in FastAI? Can I directly use an ImageList.from_folder(path) or ImageList.from_csv followed by a label_from_df? If yes, how does that work exactly?

Thanks.

4 Likes

I have the same question. The input is an image. The output is a steering value(float -1 to 1) and throttle value(float 0 to 1) to drive a car. How to use datablock api to do this?

A single float works fine.

data = (ImageList
    .from_df(path=tub_path, df=df)
    .split_by_rand_pct()
    .label_from_df(cols=1,label_cls=FloatList)
    .transform(get_transforms(do_flip=False), size=(120,160))
    .databunch()
    .normalize(imagenet_stats))

Solution:
I found that passing a list of values, rather than a single val, into the cols argument achieved what I was looking for in this case. Each integer is the index into the column of the dataframe.

so:

df = pd.DataFrame({'imgs': image_filenames, 'steering': user_angles, 'throttle': user_throttles})

data = (ImageList
    .from_df(path=tub_path, df=df)
    .split_by_rand_pct()
    .label_from_df(cols=[1,2],label_cls=FloatList)
    .transform(get_transforms(do_flip=False), size=(120,160))
    .databunch()
    .normalize(imagenet_stats))

where the cols=[1,2] argument is a list of indexes into the data frame df. (referring to steering and throttle)

4 Likes