Hi. I’ve put together a library to display tensors in a more human-readable way:
I’m not completely set on the UX/workflow, but I feel it’s at a stage where I could use some a bit of feedback. Would anyone have any thoughts/suggestions?
Install
pip install lovely-tensors
How to use
How often do you find yourself debuggin a neural network? You dump a
tensor to the cell output, and see this:
numbers
tensor([[[-0.3541, -0.3369, -0.4054, ..., -0.5596, -0.4739, 2.2489],
[-0.4054, -0.4226, -0.4911, ..., -0.9192, -0.8507, 2.1633],
[-0.4739, -0.4739, -0.5424, ..., -1.0390, -1.0390, 2.1975],
...,
[-0.9020, -0.8335, -0.9363, ..., -1.4672, -1.2959, 2.2318],
[-0.8507, -0.7822, -0.9363, ..., -1.6042, -1.5014, 2.1804],
[-0.8335, -0.8164, -0.9705, ..., -1.6555, -1.5528, 2.1119]],
[[-0.1975, -0.1975, -0.3025, ..., -0.4776, -0.3725, 2.4111],
[-0.2500, -0.2325, -0.3375, ..., -0.7052, -0.6702, 2.3585],
[-0.3025, -0.2850, -0.3901, ..., -0.7402, -0.8102, 2.3761],
...,
[-0.4251, -0.2325, -0.3725, ..., -1.0903, -1.0203, 2.4286],
[-0.3901, -0.2325, -0.4251, ..., -1.2304, -1.2304, 2.4111],
[-0.4076, -0.2850, -0.4776, ..., -1.2829, -1.2829, 2.3410]],
[[-0.6715, -0.9853, -0.8807, ..., -0.9678, -0.6890, 2.3960],
[-0.7238, -1.0724, -0.9678, ..., -1.2467, -1.0201, 2.3263],
[-0.8284, -1.1247, -1.0201, ..., -1.2641, -1.1596, 2.3786],
...,
[-1.2293, -1.4733, -1.3861, ..., -1.5081, -1.2641, 2.5180],
[-1.1944, -1.4559, -1.4210, ..., -1.6476, -1.4733, 2.4308],
[-1.2293, -1.5256, -1.5081, ..., -1.6824, -1.5256, 2.3611]]])
Was it really useful?
What is the shape?
What are the statistics?
Min / max values?
Are any of the values nan or inf?
Is it an image of a man holding a tench?
import lovely_tensors.tensors as lt
lt.monkey_patch()
numbers
'tensor[3, 196, 196] n=115248 x∈[-2.118, 2.640] μ=-0.388 σ=1.073 x=...'
t = numbers.view(-1)[:12].clone()
# A spicy tensor
t[0] *= 10000
t[1] /= 10000
t[2] = float('inf')
t[3] = float('-inf')
t[4] = float('nan')
t.reshape((2,6))
tensor[2, 6] n=12 x∈[-3.541e+03, -3.369e-05] μ=-393.776 σ=1.180e+03 +inf! -inf! nan! x=...
print(torch.zeros(10, 10))
print(torch.zeros(10, 10).to(torch.device("cuda")))
print(torch.zeros(10, 10).to(torch.device("cuda")).requires_grad_(True))
print(torch.zeros(10, 10).to(torch.device("cuda")).requires_grad_(True)+1)
tensor[10, 10] all_zeros
tensor[10, 10] cuda:0 all_zeros
tensor[10, 10] grad cuda:0 all_zeros
tensor[10, 10] n=100 x∈[1.000, 1.000] μ=1.000 σ=0. grad_fn cuda:0 x=...
But back to the numbers. Is it our man?
numbers.rgb
That’s a maybe, let’s fix him up.
in_stats = { "mean": (0.485, 0.456, 0.406),
"std": (0.229, 0.224, 0.225) }
numbers.denorm=in_stats
numbers.rgb
It’s our hero, the Tenchman!
Update 21/10/2022: I’m releasing version 0.0.3 of lovely-tensors.
The changes include:
- Colors! (I’d love to hear your feedback on the choice of colour scheme)
- Layers!
#per-channel stats
numbers.deeper
tensor[3, 196, 196] n=115248 x∈[-2.118, 2.640] μ=-0.388 σ=1.073
tensor[196, 196] n=38416 x∈[-2.118, 2.249] μ=-0.324 σ=1.036
tensor[196, 196] n=38416 x∈[-1.966, 2.429] μ=-0.274 σ=0.973
tensor[196, 196] n=38416 x∈[-1.804, 2.640] μ=-0.567 σ=1.178
# You can go even deeper if you want to
dt = torch.randn(3, 3, 5)
dt.deeper(2)
tensor[3, 3, 5] n=45 x∈[-1.398, 3.680] μ=0.312 σ=1.020
tensor[3, 5] n=15 x∈[-1.198, 3.680] μ=0.289 σ=1.199
tensor[5] x∈[-1.198, 3.680] μ=0.702 σ=1.804 x=[3.680, 0.083, 0.621, 0.322, -1.198]
tensor[5] x∈[-1.122, 0.467] μ=-0.426 σ=0.739 x=[-1.122, 0.255, -1.029, -0.699, 0.467]
tensor[5] x∈[-0.206, 1.124] μ=0.592 σ=0.512 x=[0.678, 1.124, 0.913, -0.206, 0.451]
tensor[3, 5] n=15 x∈[-1.398, 1.547] μ=0.207 σ=0.859
tensor[5] x∈[-0.300, 1.252] μ=0.407 σ=0.597 x=[0.558, 1.252, -0.006, 0.532, -0.300]
tensor[5] x∈[-0.378, 1.547] μ=0.642 σ=0.900 x=[-0.378, 1.547, -0.252, 1.332, 0.964]
tensor[5] x∈[-1.398, 0.654] μ=-0.428 σ=0.791 x=[0.021, -0.833, 0.654, -1.398, -0.583]
tensor[3, 5] n=15 x∈[-0.969, 2.827] μ=0.440 σ=1.033
tensor[5] x∈[-0.744, 1.555] μ=0.373 σ=0.820 x=[0.222, 1.555, -0.744, 0.326, 0.507]
tensor[5] x∈[-0.441, 2.827] μ=1.072 σ=1.291 x=[0.428, 2.827, -0.441, 1.905, 0.641]
tensor[5] x∈[-0.969, 0.825] μ=-0.124 σ=0.704 x=[-0.218, -0.553, -0.969, 0.297, 0.825]
-
A saner way to pass norm statistics to tensor.rgb(). tensor.rgb still works as property.
-
Bug fixes here and there
Update 25/10/2022: I’m releasing version 0.0.5 of lovely-tensors.
The changes include:
-
plots!
-
Bug fixes. It will no longer use significant amounts of GPU memory on huge tensors.
Update 12/11/2022: Version 0.1.3 is out.
I missed a few updates here. The main new features since 0.0.5 are:
.chans
Color-maps and display a tensor [0., 1.] → RGB:
Grouping:
Works for both .rgb and .chans, and with a native PyTorch implementation, it’s very fast!
Not a criticism and I do realize that some people are visual and others are more text-oriented, but basically, on reading the description, I was immediately nodding along and saying “Yep, yep!”
