if you defined the classes/categories in the order [‘cucumber’,‘zucchini’] then zucchini would be preds[1] … I had the same issue, I just flipped the indexing but yes it threw me off for a bit. @brismith answer makes sense.
2 Likes
To add to great responses above. There is an alternative approach: instead of generating negative examples one can view this as outlier detection problem. For example you can train an autoencoder to embed your “positive” data into reduced dimension vector. You will have some kind of cluster in the embedding space corresponding to a positive class. The embeddings for negative data should fall further away from the cluster center. Then you can classify all the data-points far from cluster center as negatives.
Depending on your particular problem one or both of this approaches can be applicable. If you have particular task in mind you can share it and people who worked on similar problems before will likely share some experience
2 Likes
I think you made the same mistake I always do - forgot to make it public! Click ‘edit’, then the share arrow, and set it to public.
2 Likes
Thanks Jeremy. I have made it public and updated url now.
Thanks @brismith, your suggestion worked now I’m getting correct probability.
@mike.moloch , I didn’t explicitly set the order of classes anywhere in the notebook .Seems the order is picked up alphabetically.
4 Likes
This still doesn’t work for me, btw, but glad you seem to have found the source of the error, in any case!
1 Like
If you want to get all the labels aligned to the probabilities you can use dls.vocab to get the vocabulary from the dataloader. This has a method o2i for converting a vocab name to its corresponding index.
For example
print(f"Probability it's a zucchini : {probs[dls.vocab.o2i['zucchini']]:0.2%}")
Or if you want to print all the labels and probabilities in order of descending likelihood
for prob, label in sorted(zip(probs, dls.vocab), reverse=True):
print(f'{label}: {prob:0.2%}')
8 Likes
4 posts were merged into an existing topic: Help: SGD and Neural Net foundations 
Great - I see it now 
1 Like
That’s a bit too advanced a topic for lesson 1 IMO. Feel free to ask it again in lesson 7 though!
3 Likes
Many thanks for your thoughtful comment. Rather than deleting, I’ll put my thoughts here in a reply since it’s useful to set some boundaries around our forum discussions.
In general, I’d like to avoid politically-oriented discussions on these forums, since they can take over everything else. Whilst it would be nice if mask-wearing was not a political discussion, that’s not the case… masks have become very politicised unfortunately. Therefore, let’s avoid discussing it here.
7 Likes
I’m having difficulty with learn.export() I’ve tried it on the Is it a bird? model on both my own machine and Kaggle server. I get an error: TypeError: can't pickle _thread.lock objects I’ve used this command successfully in prior versions of the course. I’ve gone back to some of those nbs and now the command fails with the same error. I’ve looked through the documentation to see if the signature of the function has changed and that doesn’t seem to be the case.
Can anyone suggest what has changed or more likely what I am doing wrong?
EDIT: Several other observations subsequent to the error after using learn.export():
- Even though there is an error, the file
export.pklis created. -
learn. show_results()now fails with:ValueError: This DataLoader does not contain any batches -
nvidia-smiindicates that the error has not released the memory from the process, which must be shutdown withkill -9 PID
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/tmp/ipykernel_33/1466232964.py in <module>
----> 1 learn.export()
/opt/conda/lib/python3.7/site-packages/fastai/learner.py in export(self, fname, pickle_module, pickle_protocol)
376 #To avoid the warning that come from PyTorch about model not being checked
377 warnings.simplefilter("ignore")
--> 378 torch.save(self, self.path/fname, pickle_module=pickle_module, pickle_protocol=pickle_protocol)
379 self.create_opt()
380 if state is not None: self.opt.load_state_dict(state)
/opt/conda/lib/python3.7/site-packages/torch/serialization.py in save(obj, f, pickle_module, pickle_protocol, _use_new_zipfile_serialization)
377 if _use_new_zipfile_serialization:
378 with _open_zipfile_writer(opened_file) as opened_zipfile:
--> 379 _save(obj, opened_zipfile, pickle_module, pickle_protocol)
380 return
381 _legacy_save(obj, opened_file, pickle_module, pickle_protocol)
/opt/conda/lib/python3.7/site-packages/torch/serialization.py in _save(obj, zip_file, pickle_module, pickle_protocol)
482 pickler = pickle_module.Pickler(data_buf, protocol=pickle_protocol)
483 pickler.persistent_id = persistent_id
--> 484 pickler.dump(obj)
485 data_value = data_buf.getvalue()
486 zip_file.write_record('data.pkl', data_value, len(data_value))
TypeError: can't pickle _thread.lock objects
I can report that I tried this on the Bird kaggle kernel and get the same error, fastai version is 2.6.2
On my local setup fastai version is 2.6.0 (the 4/25/22 paperspace container) and I don’t get this error.
EDIT: I upgraded my local fastai to 2.6.2 and now I’m getting the same error when doing learn.export()
Thanks @mike.moloch. So it’s not just me.
I would also add that even though export.pkl is created, it fails to load with load_image(‘export.pkl’). The error is: UnidentifiedImageError: cannot identify image file 'export.pkl'
1 Like
Yup, getting this on the latest as well.
In the meantime, if you instead pip install fastai==2.6.0 you should be able to continue working for now (I’m sure a fix will be forthcoming).
3 Likes
Thanks for letting me know - will work on a fix now.
OK should be fixed now in v2.6.3
6 Likes
Thanks! works like a charm! I was able to save and load a pickled model.
1 Like
Now that’s a fast fix! Many thanks.
2 Likes
2 posts were merged into an existing topic: Help: Basics of fastai, PyTorch, numpy, etc