It’s not Google. The maximum the function downloads is 400 images even if you set it to 700 images and use 700 urls.
Does it? Are you sure you have >400 urls? Did you set max_pics
to 400?
You’re right. It must have been my number of urls. I just downloaded 900 images.
Adding here since it’s related to download_images
: (fastai 1.0.32)
Looks like if you can’t write to a dictionary if you use more than one thread/process (max_workers
> 1).
I was trying to save filename:url mappings to a dictionary so I could easily remove urls after deleting images in my file explorer. I made up an ImageDownloader class and threw download_images
, _download_image_inner
and download_image
in there, then editted it a bit to keep track of what was downloaded with a dictionary.
Nothing gets written to the dictionary if max_workers
is greater than 1, although within the scope of download_image
, everything does get written. I looked around and found this stackoverflow thread, so I guess this is normal. If max_workers
is -1, 0, or 1 then the dictionary does get populated with filenames and urls.
edit: as a note, if you want to keep track of the filename – url mapping when using download_images
and run at full speed with multiple processes, you can just modify download_image
to print out the filename and url upon a successful download.
The download_image function is just replacing the previous images in the destination …What can i do??
In my case I’ve just added:
from fastai.vision.data import *
This has solved the problem for me.
I hope it helps.
Yes each time you open the jupyter notebook - need to re-import the library and subsequent commands
Hi. I am facing the same issue. While downloading the images from different url lists the previous image in the destination get overwritten.
Have you found out the solution for this problem?
Edit: I found a work around this problem by concatenating the urls in a single list and then using the function download_images
I got this error:
“NameError: name ‘download_images’ is not defined”
running the lesson2-download notebook on my machine.
fastai version 2.1.10
I have tried from fastai.vision.data import *
and
%reload_ext autoreload
%autoreload 2
%matplotlib inline
any help would be appreciated.
PS i apologize if i have asked in the wrong thread or made some other noob mistake.
edit:
from fastai.vision.utils import *
fixed it but now i get:
IsADirectoryError: [Errno 21] Is a directory: 'bears/grizzly'
after running:
download_images(path/file, dest, max_pics=200)
The new version of fastai uses a .all
module when importing like that. So you should try:
from fastai.vision.data.all import *
Although in this case you may be better off doing:
from fastai.vision.all import *