Lesson 1 official topic

I think you made the same mistake I always do - forgot to make it public! Click ‘edit’, then the share arrow, and set it to public.

2 Likes

Thanks Jeremy. I have made it public and updated url now.

Thanks @brismith, your suggestion worked now I’m getting correct probability.
@mike.moloch , I didn’t explicitly set the order of classes anywhere in the notebook .Seems the order is picked up alphabetically.

4 Likes

This still doesn’t work for me, btw, but glad you seem to have found the source of the error, in any case!

1 Like

If you want to get all the labels aligned to the probabilities you can use dls.vocab to get the vocabulary from the dataloader. This has a method o2i for converting a vocab name to its corresponding index.

For example

print(f"Probability it's a zucchini : {probs[dls.vocab.o2i['zucchini']]:0.2%}")

Or if you want to print all the labels and probabilities in order of descending likelihood

for prob, label in sorted(zip(probs, dls.vocab), reverse=True):
    print(f'{label}: {prob:0.2%}')
7 Likes

4 posts were merged into an existing topic: Help: SGD and Neural Net foundations :white_check_mark:

Great - I see it now :slight_smile:

1 Like

That’s a bit too advanced a topic for lesson 1 IMO. Feel free to ask it again in lesson 7 though!

3 Likes

Many thanks for your thoughtful comment. Rather than deleting, I’ll put my thoughts here in a reply since it’s useful to set some boundaries around our forum discussions.

In general, I’d like to avoid politically-oriented discussions on these forums, since they can take over everything else. Whilst it would be nice if mask-wearing was not a political discussion, that’s not the case… masks have become very politicised unfortunately. Therefore, let’s avoid discussing it here.

7 Likes

I’m having difficulty with learn.export() I’ve tried it on the Is it a bird? model on both my own machine and Kaggle server. I get an error: TypeError: can't pickle _thread.lock objects I’ve used this command successfully in prior versions of the course. I’ve gone back to some of those nbs and now the command fails with the same error. I’ve looked through the documentation to see if the signature of the function has changed and that doesn’t seem to be the case.

Can anyone suggest what has changed or more likely what I am doing wrong?

EDIT: Several other observations subsequent to the error after using learn.export():

  1. Even though there is an error, the file export.pkl is created.
  2. learn. show_results() now fails with: ValueError: This DataLoader does not contain any batches
  3. nvidia-smi indicates that the error has not released the memory from the process, which must be shutdown with kill -9 PID
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_33/1466232964.py in <module>
----> 1 learn.export()

/opt/conda/lib/python3.7/site-packages/fastai/learner.py in export(self, fname, pickle_module, pickle_protocol)
    376         #To avoid the warning that come from PyTorch about model not being checked
    377         warnings.simplefilter("ignore")
--> 378         torch.save(self, self.path/fname, pickle_module=pickle_module, pickle_protocol=pickle_protocol)
    379     self.create_opt()
    380     if state is not None: self.opt.load_state_dict(state)

/opt/conda/lib/python3.7/site-packages/torch/serialization.py in save(obj, f, pickle_module, pickle_protocol, _use_new_zipfile_serialization)
    377         if _use_new_zipfile_serialization:
    378             with _open_zipfile_writer(opened_file) as opened_zipfile:
--> 379                 _save(obj, opened_zipfile, pickle_module, pickle_protocol)
    380                 return
    381         _legacy_save(obj, opened_file, pickle_module, pickle_protocol)

/opt/conda/lib/python3.7/site-packages/torch/serialization.py in _save(obj, zip_file, pickle_module, pickle_protocol)
    482     pickler = pickle_module.Pickler(data_buf, protocol=pickle_protocol)
    483     pickler.persistent_id = persistent_id
--> 484     pickler.dump(obj)
    485     data_value = data_buf.getvalue()
    486     zip_file.write_record('data.pkl', data_value, len(data_value))

TypeError: can't pickle _thread.lock objects

I can report that I tried this on the Bird kaggle kernel and get the same error, fastai version is 2.6.2

On my local setup fastai version is 2.6.0 (the 4/25/22 paperspace container) and I don’t get this error.

EDIT: I upgraded my local fastai to 2.6.2 and now I’m getting the same error when doing learn.export()

Thanks @mike.moloch. So it’s not just me.
I would also add that even though export.pkl is created, it fails to load with load_image(‘export.pkl’). The error is: UnidentifiedImageError: cannot identify image file 'export.pkl'

1 Like

Yup, getting this on the latest as well.

In the meantime, if you instead pip install fastai==2.6.0 you should be able to continue working for now (I’m sure a fix will be forthcoming).

3 Likes

Thanks for letting me know - will work on a fix now.

OK should be fixed now in v2.6.3

6 Likes

Thanks! works like a charm! I was able to save and load a pickled model.

1 Like

Now that’s a fast fix! Many thanks.

2 Likes

2 posts were merged into an existing topic: Help: Basics of fastai, PyTorch, numpy, etc :white_check_mark:

3 posts were merged into an existing topic: Help: Basics of fastai, PyTorch, numpy, etc :white_check_mark:

If I want to use a held-out test set to do the validation for my model, what would be the way to do that with fastai? I’m looking at the docs for ‘Inference functions’, but the wording is a bit opaque to me at the moment.

I basically want to first make all the predictions for the test set, then evaluate those against the labels for that test set, then output a single accuracy or error rate for those predictions. I’m assuming I’d want to do this in a batched way vs one-by-one, but just not sure what the right function for that is…

1 Like