Problem downloading images in Chapter 2 ("Downloading images with DuckDuckGo")

Part 1 2022
1

Hi everyone :waving_hand:

I’m working through Chapter 2 of the fastai book, where we use DuckDuckGo to download images for the bear classifier example.

I’m using the latest version of the duckduckgo-search library (now imported as from ddgs import DDGS), but I keep getting this error when I run the search_images_ddg function:

TypeError: images() missing 1 required positional argument: 'query'

Even after updating the function according to some GitHub issues, I still get 0 results or multiple HTTP 403 errors when trying to download the URLs returned by DDGS.
Here’s a simplified version of my code:

from ddgs import DDGS
from fastai.vision.all import *
import time

def search_images_ddg(term, max_results=150):
    all_urls = []
    with DDGS() as ddgs:
        results = ddgs.images(
            term,
            region='wt-wt',
            safesearch='moderate'
        )
        all_urls.extend([r['image'] for r in results if 'image' in r])
    return list(set(all_urls))[:max_results]

bear_types = ['grizzly', 'black', 'teddy']
path = Path('bears')
path.mkdir(exist_ok=True)

for o in bear_types:
    dest = path / o
    dest.mkdir(exist_ok=True)
    urls = search_images_ddg(f'{o} bear', max_results=150)
    print(f"{o}: {len(urls)} images found")

I tried several variations (e.g. passing the term as query=term, keywords=term, etc.) and verified that the DDGS version is recent (duckduckgo-search==8.1.1).
However, the API seems to have changed and I’m not sure what’s the correct usage for the current version — or whether there’s a better alternative (Unsplash, Pexels, etc.) that others are using now for this lesson.

Has anyone managed to get this example working recently?
Any tips on how to download image datasets safely without triggering 403 or Cloudflare blocks would be greatly appreciated :folded_hands:

Thanks in advance!

1 Like
2

What environment are you running your code in? When I opened a blank Colab notebook and tried to run the code you provided it seemed to run without hitting the type error you are getting(see attached screenshot).

image
image699×572 22.8 KB

Maybe try reinstalling ddgs to make sure you have the latest version.

pip uninstall -y duckduckgo-search ddgs
pip install ddgs

1 Like
3

I was having problems with this, but have got it working now. I am using ddgs v9.8.0.

from ddgs import DDGS

def search_images(term, max_results=150):
    results = DDGS().images(
        query=term,
        max_results=max_results,
    )
    return L([result['image'] for result in results])

if not path.exists():    path.mkdir()
for o in bear_types:
    dest = (path/o)
    dest.mkdir(exist_ok=True)
    results = search_images(f'{o} bear')
    download_images(dest, urls=results)

This worked for me. Hope that helps!