Visualise tabular embeddings?

safekidda · July 15, 2020, 4:40am

Has anyone tried to reduce the dimensions of one of the embedding layers from a tabular model to visualise it?

I’m thinking along the lines of https://projector.tensorflow.org/ (in particular I like T-SNE) where instead of words as labels for each point you instead use features e.g. gender, product name etc.

muellerzr · July 15, 2020, 12:01pm

@Pak did some work with this. See the end of his notebook where he visualized the distances of the embeddings

github.com

Pak911/fastai2-tabular-interpretation/blob/master/11-football-interpr.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-10T11:47:24.805820Z",
     "start_time": "2020-06-10T11:47:24.784787Z"
    }
   },
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,

This file has been truncated. show original

safekidda · July 15, 2020, 6:44pm

Thanks @muellerzr you’re a super helpful guy! I’ve had a very quick look and couldn’t figure out how to interpret them, but I’ll have a gander at the source code and see if I can figure it out.