Is download_images function broken?

It’s not Google. The maximum the function downloads is 400 images even if you set it to 700 images and use 700 urls.

Does it? Are you sure you have >400 urls? Did you set max_pics to 400?

1 Like

You’re right. It must have been my number of urls. I just downloaded 900 images.

1 Like

Adding here since it’s related to download_images: (fastai 1.0.32)

Looks like if you can’t write to a dictionary if you use more than one thread/process (max_workers > 1).

I was trying to save filename:url mappings to a dictionary so I could easily remove urls after deleting images in my file explorer. I made up an ImageDownloader class and threw download_images, _download_image_inner and download_image in there, then editted it a bit to keep track of what was downloaded with a dictionary.

Nothing gets written to the dictionary if max_workers is greater than 1, although within the scope of download_image, everything does get written. I looked around and found this stackoverflow thread, so I guess this is normal. If max_workers is -1, 0, or 1 then the dictionary does get populated with filenames and urls.


edit: as a note, if you want to keep track of the filename – url mapping when using download_images and run at full speed with multiple processes, you can just modify download_image to print out the filename and url upon a successful download. :slight_smile:

The download_image function is just replacing the previous images in the destination …What can i do??

In my case I’ve just added:
from fastai.vision.data import *
This has solved the problem for me.
I hope it helps.

1 Like

Yes each time you open the jupyter notebook - need to re-import the library and subsequent commands

Hi. I am facing the same issue. While downloading the images from different url lists the previous image in the destination get overwritten.
Have you found out the solution for this problem?

Edit: I found a work around this problem by concatenating the urls in a single list and then using the function download_images

I got this error:
“NameError: name ‘download_images’ is not defined”

running the lesson2-download notebook on my machine.

fastai version 2.1.10

I have tried from fastai.vision.data import * and

%reload_ext autoreload
%autoreload 2
%matplotlib inline

any help would be appreciated.

PS i apologize if i have asked in the wrong thread or made some other noob mistake.

edit:

from fastai.vision.utils import * fixed it but now i get:
IsADirectoryError: [Errno 21] Is a directory: 'bears/grizzly' after running:

download_images(path/file, dest, max_pics=200)

The new version of fastai uses a .all module when importing like that. So you should try:

from fastai.vision.data.all import *

Although in this case you may be better off doing:

from fastai.vision.all import *
1 Like