Download images using DuckDuckGo [Alternative to bing API]

Part 1 (2019)
1

The bing image search need you to signup for azure, and its a hassle.
The problem with DuckDuckGo is that they do not have an official API, so the functions of fastai relies on the particular structure of their web interface, which may change. Here is what I did to solve that.

#code starts here

!pip install -Uqq fastbook
import fastbook
fastbook.setup_book()
import os
from fastbook import *

bear_types = 'teddy','grizzly','black'
path = Path('bears')

if not path.exists():
    path.mkdir(exist_ok = True)
    ind = 0
    for o in bear_types:
      dest = (path/o)
      dest.mkdir(exist_ok=True)
      results = search_images_ddg(f'{o} bear', max_images=300)
      for i in results:
        ind +=1
        dest = f"bears/{o}/{o}{ind}.jpg"
        try:
          path1, dirs, files = next(os.walk(f"/content/bears/{o}"))
          if len(files) < 100:
            download_url(i, dest)
          else:
            break
        except:
          continue

#code ends here
Note:
If you are executing in colab, clear the outputs while execution, so that the downloads process complete faster. click the three dots at the beginning of the output box everytime it fills up with the green bars…and wait for a bit…

I’ve put max_images as 300 because some of the urls were not working, so I counted all the folder contents and put a condition to limit the images to 100 with len(files). please do change the sizes
for your needs.

1 Like