Some questions in Lesson 2 (UserWarning: Your generator is empty.)

I know it’s a little late, but for anyone who might be dealing with image format issues, I had come up with a general approach to resolving this. I’ve created a function which you provide an image path, all images are scanned and converted to the desired format (if not already) and the string path is returned. I made it Windows friendly as well just in case. I do want to mention, if you have any files other than images in that directory, it will attempt to convert it (obviously throwing an exception). You could always add image extension verification if desired. :slight_smile:

import os
import platform
from PIL import Image


def image_path(imgFilePath, desiredFormat='png'):
    '''
    :param imgFilePath: Path containing images
    :param desiredFormat: Format which all images will be converted to (If they are not already)
    :return: imgFilePath will be returned back after conversion is complete
    '''
    dirChar = ('\\' if platform.system()=='Windows' else '/') #Handle win vs linux pathing
    images = os.listdir(imgFilePath)
    desiredFormat = desiredFormat.replace(".", "")
    for imgFileName in images:
        extIndex = -(len(imgFileName) - imgFileName.index("."))
        if imgFileName[extIndex:] != f".{desiredFormat}":
            fullImagePath = rf'{imgFilePath}{dirChar}{imgFileName}'
            print(f'Image not {desiredFormat}, will be converted: {imgFileName}')
            imgOutPath = rf'{imgFilePath}{dirChar}{imgFileName[:extIndex]}.{desiredFormat}'
            imgOut = Image.open(fullImagePath)
            imgOut.save(imgOutPath)
            print(rf'cleaning old image...')
            os.remove(fullImagePath)
            print(f'new image output: {imgOutPath}')
    return imgFilePath

Hello everyone. I’m running into similar issue and getting the same (Your generator is empty) warning message. First, I really don’t understand what is meant my “generator”. Below is my code with the problem. I’m using my own dataset

data[‘is_valid’] = False
dls = TextDataLoaders.from_df(data, path, text_col=‘tweet’, label_col=‘label’, valid_col=‘is_valid’, bs=6)
dls.show_batch(max_n=3)

learn = text_classifier_learner(dls, AWD_LSTM, metrics=accuracy)
learn.fine_tune(4, 1e-2)

/opt/conda/lib/python3.10/site-packages/fastprogress/fastprogress.py:73: UserWarning: Your generator is empty.
warn(“Your generator is empty.”)

I figured out the solution to my question. For anyone in the future who runs into this problem do this. If you’re using two separate files for you training and validation set, you need to specify you validation set by setting ‘splitter’ in your DataBlock to the path of the validation file. Use one of the functions from here fastai - Data transformations to process the validation file.

1 Like

Hi! I am getting the same error after assigning the splitter value. I assigned the value as such:
splitter=‘/kaggle/input/val_data.txt’
But I got error this time. Can you share the solution?
Moreover, the metrics caches are ‘None’, like that:
Screenshot 2024-02-01 162430
I don’t know what to do?

Heya, I hope you find this useful.

Since you’re working with text files you want to call the fastai function “get_text_files” so the solution should be splitter=get_text_files(‘/kaggle/input/val_data.txt’).

“get_text_files()” is one of the data transformations functions provided by fastai. Read more here