Lesson 1 official topic

When I try to run this code

from fastdownload import download_url
dest = 'bird.jpg'
download_url(urls[0], dest, show_progress=False)

from fastai.vision.all import *
im = Image.open(dest)
im.to_thumb(256,256)

I’m getting this error
**URLError** : <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)>

mkdir has two arguments and u forgot the “parent” argument, by default its equalled to false u gotta make it true. Ig it helps

1 Like

But Jeremy used only one argument in the starting part of the lecture. Although in his Kaggle NB he uses the parent argument.

Mmh, i really dk abt that. But yeah the error raised is pointing that “parent” arg is missing.

You need to create the bird_or_not directory

1 Like

I am trying to do a project on Lesson 1.

I tried and experimented with two ideas that I got, but the results weren’t great. I figured out the reasons why it didn’t work and was wondering if I am right.

  • Project 1: Bike transportation is the common means of transportation here in India . I tried to create a “Is s/he driving with or without helmet?”
    model. It didn’t work because the search results on ddg weren’t accurate. Some pictures were the opposite of the search. I realised I should pick something which doesn’t mess up the search mechanism.

  • Project 2: The 3 most spoken languages in the world as of today are English, Mandarin, and Hindi. I tried to create “What text is fhat?” based on this, where I searched for “english text”, “mandarin text”, and “hindi text”. Sadly, this also didn’t work. The search was fine this time. Was is it because the text images are all “same” to the model, on the level of pixels?

Is my understanding right in both of these?

I am just trying to understand the limitations that are there with what we did in Lesson 1 so that I can try resolve them in the upcoming lessons.

Samar I love your ideas! Perhaps you’re jumping a little too far too fast?
In project 2 are you using full page pdfs of text? If so it might help if you used tried single sentances or single words? Even using full pages of text it would make sense to me that a model could be trained to classify is_english vs is_mandarin. Perhaps you might learn some things by starting really small like “english word” vs “mandarin character” and then expand from there? In my experience I’ve found starting simple and small quickly transfers to bigger and more complex.

1 Like

Thanks for the insights.

I was indeed thinking that it would work better on documents of texts instead of images, which I haven’t learnt yet. This must be covered in later lessons.

I agree that I might be pushing a bit hard :sweat_smile:. But mentally I am “free”. Just exploring my ideas and archiving for later if something doesn’t work.

I tried like 5-7 classification models based on lesson 1:

  • planets
  • cats vs dogs
  • seasons (winter vs summer vs rainy vs autumn)
  • caterpillar vs butterfly

The show_batch was showing all the images fine but after running the probability, all of the time in all the above examples the probability was very low (< 10%).

Did this happen with anyone?

Any suggestions or insight as to what might be happening? The code is the exact same from lesson 1.

I want to run just one, simple classification example and move to lesson 2.

I can’t recall the detailed code from Lesson 1. It would make it easier to help if you could link to the notebook, rather than readers having the burden to go find notebook.

10% is not “low probability”. Its high probability of the opposite. Likely you got your labels mixed up.

Low probability is a 50/50 prediction .

My bad. I was assuming folks would be familiar with the Lesson 1 code :sweat_smile:. Will share NBs from next time.

If you are saying its not low probability then I need not worry then :blush:.

It’s a relief thanks!

1 Like

Just had to debug a bit to get iPywidget working – quick, minor tweaks.

I am leaving here a small end-to-end snippet that will work. Since the code is changed, make sure to comment out the old uploader and PILImage lines and replace it with following:

uploader = widgets.FileUpload()
uploader # shift-enter to get uploader widget
# Above two lines are not changed from the old version

## IN NEXT CELL, following behaviour is new
# to get the first image and use it against the learn object
item = uploader.value[0]    # uploader.value is a tuple of dictionaries
img = PILImage.create(item['content'].tobytes())

# [optional] use the image in predict
print(img.show(), learn.predict(img))

# [optional] to iterate you may run following (not tested)
for item in uploader.value:
    img = PILImage.create(item['content'].tobytes())
    print(img.show(), learn.predict(img))

Feel free to add to this if I missed anything.

2 Likes

Very minor correction. That xkcd was actually from the end of 2014 (September 24, 2014)

Can some of the admins create a category for the textbook, if that sounds like a good idea?

We have all these spaces for courses and other stuff but none to discuss things from the textbook.

2 Likes

Hi there,

I don’t know if this is really the right place to ask this or not.

I’m having issues getting the search_images or search_images_ddg working locally.

I’ve even created a separate notebook to test just the search_images_ddg function and this doesn’t work either. The issues appears to be with the urlread line.

I’m running a Mac M1, 32GB, Ventura 13.0 with the Conda setup described here

Any help would be really appreciated as I can’t follow along with the exercises right now…it’s been a few hours of trying to get this working and no luck :frowning:

Thanks a lot !

The environmetn can have a significant impact, and I don’t have a Mac, so I’m little help.

I could help if you were using one of the cloud services and set your notebook to public viewing. Using another platform would also provide you an extra datum for your troubleshooting. It often is very useful to compare a working and non-working implementation.

It is considered easier to commence your DL journey using a cloud service, so you don’t have to deal so much with platform issues.

1 Like

I have the same configuration as you (M1 MBP, 32GB, Ventura etc.) and I was able to run the search_images_ddg method with no issues locally:

The error screenshot you posted does not show all the error info. There’s a bit at the very end (or the very beginning? I forget which …) which should show the actual error. So if you can post the full error stack, it would be helpful in figuring out what might be going on.

1 Like

Hey Fahim, really appreciate your help!!

The full error I get back is below…

Would it be possible to share your setup / conda env with me to see what I might have done differently?

---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)
File ~/miniconda3/envs/fastai/lib/python3.10/urllib/request.py:1348, in AbstractHTTPHandler.do_open(self, http_class, req, **http_conn_args)
   1347 try:
-> 1348     h.request(req.get_method(), req.selector, req.data, headers,
   1349               encode_chunked=req.has_header('Transfer-encoding'))
   1350 except OSError as err: # timeout error

File ~/miniconda3/envs/fastai/lib/python3.10/http/client.py:1282, in HTTPConnection.request(self, method, url, body, headers, encode_chunked)
   1281 """Send a complete request to the server."""
-> 1282 self._send_request(method, url, body, headers, encode_chunked)

File ~/miniconda3/envs/fastai/lib/python3.10/http/client.py:1328, in HTTPConnection._send_request(self, method, url, body, headers, encode_chunked)
   1327     body = _encode(body, 'body')
-> 1328 self.endheaders(body, encode_chunked=encode_chunked)

File ~/miniconda3/envs/fastai/lib/python3.10/http/client.py:1277, in HTTPConnection.endheaders(self, message_body, encode_chunked)
   1276     raise CannotSendHeader()
-> 1277 self._send_output(message_body, encode_chunked=encode_chunked)

File ~/miniconda3/envs/fastai/lib/python3.10/http/client.py:1037, in HTTPConnection._send_output(self, message_body, encode_chunked)
   1036 del self._buffer[:]
-> 1037 self.send(msg)
   1039 if message_body is not None:
   1040 
   1041     # create a consistent interface to message_body

File ~/miniconda3/envs/fastai/lib/python3.10/http/client.py:975, in HTTPConnection.send(self, data)
    974 if self.auto_open:
--> 975     self.connect()
    976 else:

File ~/miniconda3/envs/fastai/lib/python3.10/http/client.py:1447, in HTTPSConnection.connect(self)
   1445 "Connect to a host on a given (SSL) port."
-> 1447 super().connect()
   1449 if self._tunnel_host:

File ~/miniconda3/envs/fastai/lib/python3.10/http/client.py:941, in HTTPConnection.connect(self)
    940 sys.audit("http.client.connect", self, self.host, self.port)
--> 941 self.sock = self._create_connection(
    942     (self.host,self.port), self.timeout, self.source_address)
    943 # Might fail in OSs that don't implement TCP_NODELAY

File ~/miniconda3/envs/fastai/lib/python3.10/socket.py:845, in create_connection(address, timeout, source_address)
    844 try:
--> 845     raise err
    846 finally:
    847     # Break explicitly a reference cycle

File ~/miniconda3/envs/fastai/lib/python3.10/socket.py:833, in create_connection(address, timeout, source_address)
    832     sock.bind(source_address)
--> 833 sock.connect(sa)
    834 # Break explicitly a reference cycle

TimeoutError: [Errno 60] Operation timed out

During handling of the above exception, another exception occurred:

URLError                                  Traceback (most recent call last)
Input In [9], in <cell line: 1>()
----> 1 results = search_images_ddg('grizzly bear', max_images=10)
      2 # ims = results.attrgot('contentUrl')
      3 print(len(results), results)

File ~/miniconda3/envs/fastai/lib/python3.10/site-packages/fastbook/__init__.py:57, in search_images_ddg(term, max_images)
     55 assert max_images<1000
     56 url = 'https://duckduckgo.com/'
---> 57 res = urlread(url,data={'q':term})
     58 searchObj = re.search(r'vqd=([\d-]+)\&', res)
     59 assert searchObj

File ~/miniconda3/envs/fastai/lib/python3.10/site-packages/fastcore/net.py:117, in urlread(url, data, headers, decode, return_json, return_headers, timeout, **kwargs)
    115 "Retrieve `url`, using `data` dict or `kwargs` to `POST` if present"
    116 try:
--> 117     with urlopen(url, data=data, headers=headers, timeout=timeout, **kwargs) as u: res,hdrs = u.read(),u.headers
    118 except HTTPError as e:
    119     if 400 <= e.code < 500: raise ExceptionsHTTP[e.code](e.url, e.hdrs, e.fp, msg=e.msg) from None

File ~/miniconda3/envs/fastai/lib/python3.10/site-packages/fastcore/net.py:108, in urlopen(url, data, headers, timeout, **kwargs)
    106     if not isinstance(data, (str,bytes)): data = urlencode(data)
    107     if not isinstance(data, bytes): data = data.encode('ascii')
--> 108 try: return urlopener().open(urlwrap(url, data=data, headers=headers), timeout=timeout)
    109 except HTTPError as e: 
    110     e.msg += f"\n====Error Body====\n{e.read().decode(errors='ignore')}"

File ~/miniconda3/envs/fastai/lib/python3.10/urllib/request.py:519, in OpenerDirector.open(self, fullurl, data, timeout)
    516     req = meth(req)
    518 sys.audit('urllib.Request', req.full_url, req.data, req.headers, req.get_method())
--> 519 response = self._open(req, data)
    521 # post-process response
    522 meth_name = protocol+"_response"

File ~/miniconda3/envs/fastai/lib/python3.10/urllib/request.py:536, in OpenerDirector._open(self, req, data)
    533     return result
    535 protocol = req.type
--> 536 result = self._call_chain(self.handle_open, protocol, protocol +
    537                           '_open', req)
    538 if result:
    539     return result

File ~/miniconda3/envs/fastai/lib/python3.10/urllib/request.py:496, in OpenerDirector._call_chain(self, chain, kind, meth_name, *args)
    494 for handler in handlers:
    495     func = getattr(handler, meth_name)
--> 496     result = func(*args)
    497     if result is not None:
    498         return result

File ~/miniconda3/envs/fastai/lib/python3.10/urllib/request.py:1391, in HTTPSHandler.https_open(self, req)
   1390 def https_open(self, req):
-> 1391     return self.do_open(http.client.HTTPSConnection, req,
   1392         context=self._context, check_hostname=self._check_hostname)

File ~/miniconda3/envs/fastai/lib/python3.10/urllib/request.py:1351, in AbstractHTTPHandler.do_open(self, http_class, req, **http_conn_args)
   1348         h.request(req.get_method(), req.selector, req.data, headers,
   1349                   encode_chunked=req.has_header('Transfer-encoding'))
   1350     except OSError as err: # timeout error
-> 1351         raise URLError(err)
   1352     r = h.getresponse()
   1353 except:

URLError: <urlopen error [Errno 60] Operation timed out>```

Hey Sam, happy to help :slight_smile:

My setup won’t help you since your URL query is timing out for some reason — it’s basically just a network error and not anything else, as far as I can tell. This is the relevant bit:

URLError: <urlopen error [Errno 60] Operation timed out>```

I believe the URL that the code is trying to connect to is:

https://duckduckgo.com/

The first debugging step you might want to try is to see if you can connect to the URL directly from your browser. If that works, then you might need to do a bit more checking. But if it works, then you at least know it should generally work and that there’s some issue with the Python code perhaps?

Thanks Fahim!

Narrowing it down haha…

Looks like I can’t access duckduckgo.com.

If I restart my router, then duckduckgo opens, however when I run search_images_ddg, duckduckgo then become unresponsive again. Tried this twice now. For some reason, running the search_images_ddg stops duckduckgo being accessible on my wifi.

No idea what’s going on here :man_facepalming:

EDIT: I think I’ve found a similar issue - going to try a different tak: Ddg not replying?