New blog post - user classification by mouse movements

Part 2 & Alumni (2018)
1

Cats and Dogs goes long way :slight_smile:

Thanks again Jeremy!
(Corporate blog post so it had to follow certain restrictions - but in essence this is DL, part 1!)

16 Likes
2

wow, what a nice read! congratulations @gesman !

Iā€™m really impressed at the algorithm to ā€œtranslateā€ the time-series mouse movement data into the mouse movement images. Is this specialized high contrast color encoding algorithm publicly available? If not, are there plans to publish this research? Thanks again for sharing this.

3

Thatā€™s a great post - so happy to see your success @gesman and really interested to learn more about the approach. Translating mouse velocities into color is a nice idea :slight_smile:

4

This is absolutely brilliant and shows how thinking out of the box can lead to unique solutions and yet leveraging trained networks like vgg. Thanks for sharing and thereby motivating us.

5

Incredible work!

Iā€™m curious - when you said: ā€˜Weā€™ve devised a specialized high contrast color encoding algorithm to represent direction, speed, and acceleration of mouse movements in a consistent way.ā€™ Was this done in order to get the specific problem to fit into the tried-and-true VGG architecture which handles digital images or did you already try calibrating a more general architecture on the data directly and the VGG approach was better? (not trying to poke holes, just generally curious)

Fascinating stuff.

6

Fascinating work. Curious to know more about how you represented direction and speed in the imageā€¦

7

Thank you.
I am pushing our corporate to make content available - and that includes complete source code + color encoding algorithms.
It might take a little while but I hopeful to get permission for complete disclosure by September - for our upcoming Splunk big user conference where I likely will be a speaker.

Gleb

1 Like
8

Thanks Jeremy.

Color encoding was important part of the process because we wanted to make CNNā€™s job as easy as possible - and essentially convey tiny behavior differences in big color contrasts so that algorithms would see more definitive patterns.

Gleb

9

Thanks Vincent,

With mouse movements converted to images - there are not much of ā€œimageā€-ish data to work with.
Unlike cat and dogs where furry creature occupies the whole frame - the whole mouse movement picture is about 5% of lines and 95% of empty space around.
And then we also needed somehow to convey speed, acceleration, direction etcā€¦ within that 5% of actual image data.
So it took some careful thinking and discoveries to devise the way to encode everything in a CNN-friendly way.

My past experience in professional photography helped too :slight_smile:

Gleb

1 Like
10

Hi, thanks.

As I mentioned in reply above - I will be looking to make the rest of approach available as soon as possible.

Gleb

11

[quote=ā€œgesman, post:7, topic:2595, full:trueā€]
I am pushing our corporate to make content available - and that includes complete source code + color encoding algorithms. [/quote]

Thank you! One quick followup on the actual vgg16 model, did you train a new VGG or is this a transfer learning using the imagenet weight and imagenet mean, std for normalization? Have you tried the vgg16 with batch normalization?

12

Batch norm was used, correct.
However i didnā€™t do transfer learning because mouse movement imagery are so much different from VGG16 natural sets.
And then - the training was so fast - so there were no need to go through extra troubles.

1 Like
13

Very cool. Have you tried using a GAN?

14

David, have you seen research/implementation of using GANs to improve classification? Iā€™m very curious and would appreciate it if you can point me to the research/examples. Thank you!

15

Hmm, for some reason I thought that you did use transfer learningā€¦ donā€™t know how I got that idea from the write-up. Intuitively, this makes sense as he images are different, but I thought that the first two layers would have still helped from VGG.

One of the papers I read (https://arxiv.org/abs/1411.1792) mentioned that first few layers are very generic and can also be seen from the visualizations. If It is not too much trouble, can you also try it with freezing first two layers or so and see if it improves the results (or not freezing any layers, but using these initial weights from the trained network)?

Thanks.

1 Like
16

Is the content available now?could you please share the details if yes