UserWarning: Your generator is empty. warn("Your generator is empty.")

fastai
1

I’m doing a pet project where I train a language model learner on an export of my personal social media posts.

I tried to train the learner like so:

learn = language_model_learner(dls_lm, AWD_LSTM, drop_mult=0.3, metrics=[accuracy, Perplexity()]).to_fp16()
learn.fit_one_cycle(1)

and I get:

UserWarning: Your generator is empty.

When I “show_batch” on the dataloader I can see that there’s data in there:

image
image2318×571 91.7 KB

Any help would be appreciated…

2

Hi Jonathan,

To debug, you can test your code with a dataset which fastai provides, such as IMDb. If it runs fine, then the issue likely is with the dataset you’re using. It maybe too small.

You can try adding some more data to your dataset; alternatively, you may decrease the batch size (default is 64). I hope this helps.

1 Like