I have a question about what is possible after completing the course.
I am interested in competitive pinball play and have thought for a long time that it would be cool to be able to analyze video of a game and turn it into statistics about the play. As an example, I would like to be able to track how many times a certain ramp shot was hit during the game. I think the way to do that would be to have a model that could track where the ball is in the game at all times. Then have some logic that says something like if the ball is at x1, y1 in frame 1 and then x2, y2 in frame 2 then that would correspond to a made ramp shot.
One more example that is maybe easier to understand: I would like to keep track of how many times the player flipped the flipper during the course of the game.
So, ultimately, I am looking to turn video of play into statistics about play. I would love to hear some feedback about if people think this is possible and what topics I should look into in order to make this project a reality. Is this kind of project something that already has a name? I am happy to clarify what I’m looking to do, if needed.
you’d just break it into frames, probably chuck a lot of them away (ottomh i think i’d start with keeping 10fps) and then train a cnn to recognise pictures of the events you care about. teach it to recognise a picture of the ball in/on the ramp, the ball on the left flipper, the ball on the right flipper etc…all the things you care about.
then feed the frames to the model and see what it says they are, bearing in mind the nature of the data being frames, so:
nothing, nothing, nothing, ramp, ramp, ramp, ramp, ramp, ramp, bumper is really just one ramp “event” spanning several frames.
a simple single label classifier which you learn to build in the first couple of lessons of the course (if you’re doing that) should do this pretty well. as long as your coding is up to breaking the video up, feeding it to the model and keeping count of everything which happens, the model should tell you what’s happening on screen.
if you want to track several things at the same time (ball on ramp + left flipper up) it’s more work but you can start with the simple one and work up from there. the basic logic would be the same.
also make sure you post it in “share your work” when it’s done because that sounds cool.
Thanks! I really appreciate the reply. I can already see myself getting lost in details that I imagine are required to make this “perfect”. But I think it’s best to hold off on getting into those. What you’ve outlined is enough to get started and make a simple version of this. If I get that done and still want to improve it, then I will let myself get bogged down in those details. Thanks for the encouragement to post in “share your work,” too.