I am currently working on the arxiv version of this idea.
After making some more tweaks, and getting more comprehensive results (more benchmarks), I will submit it to arxiv. It will be great if you could have a look at it and suggest some things that may improve these results or what new benchmarks I should try? I will love to collaborate with someone who finds this idea interesting.
@jeremy: One of the problems I am facing right now is to figure out what problems (image segmentation/object detection etc) and datasets to try this on. And SOTA code of most problems is not even available. If you can have a very quick look through it and suggest something it will be great.
I’ll give it a read and come up with suggestions if possible. Thanks for posting.