`grid_sample` in Swift

(Jeremy Howard (Admin)) #1

We’re going to need something like affine_grid and grid_sample from PyTorch. (Which AFAICT is much the same as remap in OpenCV). Whilst we can simply use remap on CPU, that won’t help us on GPU. (Also remap doesn’t support batch transforms.)

It appears that tf doesn’t support this. Although, in classic tf fashion, some half-hearted implementations are dumped in various other places - most notably buried in Spatial Transformer in the official tf repo:

And something in tf.contrib:

https://www.tensorflow.org/api_docs/python/tf/contrib/resampler/resampler

And something in an external lib:

@saeta @clattner is this something that might be doable in Swift (and be performant on GPU)? Perhaps the s4tf team could see what the best approach might be?

0 Likes

(Jeremy Howard (Admin)) #2

Just discovered that cudnn already supports both these operations :slight_smile:

https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html#cudnnSpatialTfSamplerForward

So how do we wrap a cudnn thing in s4tf most conveniently?

0 Likes

(Vova) #3

It seems the latest OpenCV also has cuda implementation of remap (in extra modules list)
https://docs.opencv.org/master/db/d29/group__cudawarping.html

0 Likes