Creating DataBunch for Show and Tell Caption Generator Paper

Part 2 (2019)
1

Hi folks,

I am trying to replicate this paper. I have understood the paper. However, I’m stuck at the first step of creating a DataBunch for this.
I have downloaded the train and val images and have them in a folder.
I have a dictionary mapping the image ids to the captions and I am trying to use the following code to create a DataBunch:

q = (ImageList.from_folder(coco_imgs)
.split_by_folder(‘train2014’,‘val2014’)
.label_from_func(get_caption_for_image)
.databunch(bs=30)
)

However, it seems like as a target it is expecting classes and not text.
I have been given this warning:

UserWarning: You are labeling your items with CategoryList.
Your valid set contained the following unknown labels, the corresponding items have been discarded.

I have tried going through the codebase for what other kinds of targets are permissible in fastai but I can’t seem to find any.

I did manage to find this example in fastai_dev. However ObjectDetectDataset is no longer a part of fastai as it now resides in fasai/old/do_not_use

1 Like
2

Hi @chatuur, did you ever figure this out? I’m trying to do a similar thing.

3

It was implemented in fastaiv1 here https://github.com/fg91/Neural-Image-Caption-Generation-Tutorial

2 Likes