Itemgot conflicts

fastai
1

I’m trying to run the first example scripts from the Deep Learning for Coders course. The given function

def search_images(term, max_images=100):   
    
    urls = L(search_images_ddg(term, max_images)).itemgot('image')
    return urls

raises a TypeError: string indices must be integers, not 'str'. From the traceback I’m guessing that the cause is in line 175 of foundation.py but I do not really understand what is going on. Below is all the output including the traceback. It looks like I might be missing some packages and that might be related to my problem or cause problems later.

runfile('/home/jvkloc/Desktop/fast_ai/image_scraper.py', wdir='/home/jvkloc/Desktop/fast_ai')
/home/jvkloc/mambaforge/envs/spyder-env/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
/home/jvkloc/mambaforge/envs/spyder-env/lib/python3.11/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/home/jvkloc/mambaforge/envs/spyder-env/lib/python3.11/site-packages/torchvision/image.so: undefined symbol: _ZNK3c107SymBool10guard_boolEPKcl'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
/home/jvkloc/mambaforge/envs/spyder-env/lib/python3.11/site-packages/fastbook/__init__.py:11: UserWarning: Missing `ipywidgets` - please install it
  except ModuleNotFoundError: warn("Missing `ipywidgets` - please install it")
/home/jvkloc/mambaforge/envs/spyder-env/lib/python3.11/site-packages/fastbook/__init__.py:13: UserWarning: Missing `sentencepiece` - please run `pip install 'sentencepiece<0.1.90'`
  except ModuleNotFoundError: warn("Missing `sentencepiece` - please run `pip install 'sentencepiece<0.1.90'`")
Traceback (most recent call last):

  File ~/mambaforge/envs/spyder-env/lib/python3.11/site-packages/spyder_kernels/py3compat.py:356 in compat_exec
    exec(code, globals, locals)

  File ~/Desktop/fast_ai/image_scraper.py:90
    image_scraper()

  File ~/Desktop/fast_ai/image_scraper.py:79 in image_scraper
    download_images(folder, urls=search_images(image_class))

  File ~/Desktop/fast_ai/image_scraper.py:32 in search_images
    urls = L(search_images_ddg(term, max_images)).itemgot('image')

  File ~/mambaforge/envs/spyder-env/lib/python3.11/site-packages/fastcore/foundation.py:175 in itemgot
    for idx in idxs: x = x.map(itemgetter(idx))

  File ~/mambaforge/envs/spyder-env/lib/python3.11/site-packages/fastcore/foundation.py:156 in map
    def map(self, f, *args, **kwargs): return self._new(map_ex(self, f, *args, gen=False, **kwargs))

  File ~/mambaforge/envs/spyder-env/lib/python3.11/site-packages/fastcore/basics.py:840 in map_ex
    return list(res)

  File ~/mambaforge/envs/spyder-env/lib/python3.11/site-packages/fastcore/basics.py:825 in __call__
    return self.func(*fargs, **kwargs)

TypeError: string indices must be integers, not 'str'

My whole script is here:

def image_scraper():#class_1, class_2, folder, nbr_of_images
    
    parser = argparse.ArgumentParser()
    parser.add_argument(
        '--class_1', '-c1', type=str, default='dancer from above',
         help='DucDuckGo search term for class one images.'
     )
    parser.add_argument(
        '--class_2', '-c2', type=str, default='chair',
         help='DuckDuckGo search term for class two images.'
     )
    parser.add_argument(
        '--folder', '-f', type=str, default='training_images',
         help='Folder for downloaded images. Subfolders for both image '
             'classes will be created into this folder.'
     )
    parser.add_argument(
        '--number_of_images', '-n', type=int, default=100,
         help='The number of attempted downloads of both classes of images.'
     )
    args = parser.parse_args()
    folder = Path(args.folder)
    searches = (args.class_1, args.class_2)
    for image_class in searches:
        folder = (folder/image_class)
        folder.mkdir(exist_ok=True, parents=True)
        download_images(folder, urls=search_images(image_class))
        sleep(10)
        resize_images(
            folder/image_class, max_size=400, dest=folder/image_class
        )
    failed = verify_images(get_image_files(folder))
    failed.map(Path.unlink)
    failed_downloads = len(failed)
    print(f'{failed_downloads} downloads failed')
    
if __name__ == '__main__':
    image_scraper()

Thank you for all the help already in advance. I’m feeling very optimistic about fast.ai and learning how to build machine learning models. I’ve been going through all kinds of tutorials from blogs and YouTube but this seems to be the way to actually produce something that works.

2

Hey!

The issue is that itemgot takes a number as input, not a string. The purpose of itemgot is to - for each element in the list - get a subitem from that element.

For example, if we have my_list = L([[1,2], [3,4]]), then my_list.itemgot(1) will for each item in my_list (i.e., for [1,2] and for [3,4]) return it’s first item. Remember, this “first” item is actually the second, because counting starts at 0.

So my_list.itemgot(1) will return [2,4].

In your case, you don’t need itemgot at all. L(search_images_ddg(term, max_images)) already returns a list of URLs. This is already what you want!

So

def search_images(term, max_images=10):   
    urls = L(search_images_ddg(term, max_images))
    return urls

will do the job.

Here’s a working example.

Edit: Made notebook public, so link works now.

2 Likes
3

Yes, I only needed this, not itemgot() at all. Thank you.