Advice for image classification when classes are very, very similar


(WG) #1

For example, imagine your task is to differentiate between American Labrador Retrievers and English labs.

What can we do to improve the abilities of our classifier given that they are very very similar. I’ve tried running through a dataset like this and can’t get the classifier to do better than a coin flip.


(Jeremy Howard) #2

I don’t think there’s any special approaches in this case. If you can provide more details about the dataset you’re using, and what you’ve tried, we can try to give specific recommendations.


(WG) #3

This may sound kind of lame, but I’m trying to build a classifier that can distinguish between In-N-Out burgers and other burgers.

I have about 400 pictures of In-N-Out burgers and almost 2,000 images of other burgers. Most of the images are pretty big.

So far, I’ve tried basically using the lesson 1 approach on the dataset. I’m only getting about .18 accuracy and my validation loss is NaN (which can’t be good).


(Jeremy Howard) #4

Sounds kinda awesome to me. Not that I like In-N-Out burgers myself - can’t understand the attraction…

Anyhoo if you’re getting NaN loss something is very wrong. I’d guess a too large learning rate. If you show a screenshot of the training process we’d be able to tell more.


(WG) #5


(Jeremy Howard) #6

What arch and sz are you using? How big are the original images? Can you show a couple of examples? That lr_find plot looks really odd…


(WG) #7

arch = tried resnet34
sz = 224

Images are pretty big … I think the average was something around 1000x800

Positive examples:
2031

Negative Examples:
bk_4bk_137

mc_430ss_337


(WG) #8

Could it be because I am not running on GPU?

I’m just testing on my macbook pro … just using the CPU.


(Kevin Bird) #9

I don’t have any advice here, but this is an amazing project!


(Jeremy Howard) #10

The CPU code is new and I haven’t looked at it. Worth trying on a GPU. Also worth trying a very small learning rate. Need to figure out where that NaN is coming from…


(WG) #11

For us in Southern California, eating any other kind of burger is borderline blasphemous so it could save lives, or even, souls.


(Kevin Bird) #12

I don’t think they have any of those in Nebraska. They do look good though.


(Jeremy Howard) #13

Nah they’re really not. And the fries are always drier than any fry deserves to be. Sorry @wgpubs I can see you feel strongly about this, but I just can’t avoid blasphemy.


(WG) #14

I’m going to agree with you about the fries … but only the fries.

You got me curious though, what are you folks eating in the way of burgers up in northern California?


(Ravi Sekar Vijayakumar) #15

Awesome project. My vote is for In-n-out as well and am sure once your model works, it will pick it too!


(Jeremy Howard) #16

(WG) #17

Never heard of it.

I got a kid applying to some schools up north, when we do our road-trip in a couple months I’ll check it out.


(WG) #18

Running on the GPU and its working beautifully.

Getting 93.1% accuracy before TTA … 92.9% after TTA.

Now I just have to figure how to build an Android app that can run this model … any advice on how to “productionize” our work would be appreciated.

… and thanks again for your help!


(sergii makarevych) #19

I still see .cuda assignments without checking if cuda is available


(Jeremy Howard) #20

A PR to fix the CPU training would be welcome :slight_smile: