Set game app

(Brian Holland) #1

Hello all, I’m interested in creating a machine-learning based app for a card game I love.

The card game is called Set and is quite stimulating.
You can read about it on wikipedia, and you can also play it on the publisher’s site. It’s a fairly interesting game, mathematically and computationally speaking. Peter Norvig was curious about 2 statistical claims given in the rules, so he wrote a python script to play the game automatically in order to empirically prove those claims (spoiler alert: they’re right). Others have written academic papers and math-based books on the subject.

So the game has two times where an app would be IMMENSELY helpful. The first is a moment you’re probably about to have: how do you learn what makes a set? The other comes usually one to three times per game: is there a set on the board? Both of these questions would be helped immensely by having an app that could take a picture of some number of cards and respond with a number of sets present (or optionally a reveal of the exact sets present).

So on the surface, this seems like a fairly trivial problem. Imagenet (vggXX) has loads of inner kernel/filters with the sorts of shapes already built in to it. Just train imagenet. But I’ve been trying with not much success, so I think it would be better to just scrape the slate clean and start fresh.

I have been taking pictures of the cards and sets of cards and have a little over a thousand images compiled. Some labeled, some not. If you’re interested in this problem, let me know or comment as such here and I can work out how to share the dataset with you. Knowing how good object detection is getting, I want this app more and more. I’m super keen on getting this done any way possible. :slight_smile: My schedule is pretty open right now, especially in the mornings CST, so if you’d rather do something synchronous I’m fine with that too. I need help! I have two engineering degrees, so I’m pretty used to coding and technical problems. And I have loved every one of Jeremy’s lectures. I’m very open to correction and suggestions, vague or specific. I’ve probably forgotten some key information or something in this post, so just let me know what would be most helpful and I’ll do my best.

(marc) #2

What type of model are you using?

Maybe you could generate a fake dataset (draw the card on top of any background).

(Brian Holland) #3

The dataset isn’t really a problem. I think that’s a valuable journey in how to create a dataset. Not only how, but what. What should be in the dataset? Multiple types of photos? Many sizes? Hundreds? Thousands? Only dozens?

As far as model is concerned, I was playing around with different ones and getting nonsense all around. That’s part of why I’m seeing if there’s any interest in helping me out. What model(s) would you recommend (against)?

(marc) #4

I would use any object detection architecture. What have you tried so far? How many labeled images (with bounding boxes) do you have?