I added a new feature to explore a given
ImageDataBunch using PAIR’s Facets Dive exploration. You may have seen the excellent visualization of the Quick, Draw! dataset in Facets Dive. I find it really useful to be able to rapidly evaluate which images are doing worse or better and get an intuitive sense of the images, and Facets Dive helps immensely in that regard.
Here is the diff and the doc notebook for starters. I still need to add tests, so it’s a WIP. I just wanted to announce it to get some feedback about whether you would consider adding this to FastAI or if it is considered out of scope.
A big caveat is that Facets Dive has still not been ported to work with Jupyter Lab. It only works with Jupyter Notebook. There is an outstanding issue in the Facets repo to fix the Jupyter Lab incompatibility.
Any feedback appreciated.
I’d rather have it in its own thread if you don’t mind, the developer chat is for updates on changes in the code, this is better suited for discussion.
I like the idea, but didn’t get a chance to read the code yet. Just one thing though, I think the module should go in fastai.widgets and not fastai.vision.
Did you get a chance to test it on colab? It’s a bit different than Jupyter notebooks, that’s why I ask.
Thanks @sgugger. I’ll move it to widgets. I didn’t test it in colab yet. I’ll see how it works in colab.
Alright, I moved everything to widgets and updated the docs.
And I tried to make it work with Google Colab.
Here’s the summary of how it works with Colab.
Images won’t render in colab
- Facets requires access to image URL. In a local Jupyter notebook any file that’s in a subdirectory of the notebook can be accessed by
localhost:8080/path/to/file.jpg. In other words Jupyter server, serves the image and the image has an associated URL.
- However, if colab runs at
https://colab.research.google.com/drive/xyz, you can’t access an image in a subfolder via
https://colab.research.google.com/drive/xyz/path/to/file.jpg. This is potentially for security reasons. But that puts us in a bind because the Polymer component has to access the image via the URL and then render the various parts appropriately.
- Just to see if there is a hack, I tried to upload the img to imgur and serve via the URL there, but chrome just blocks access to the cross-origin URL because of CORB.
- I googled to figure out if there is a way serve an image via a URL in colab but was not able to find a way.
- Even the official examples of how to use Facets Dive in Colab show a case without a sprite image, which makes me think that they ran into the same limitations.
Interactions still work
- But even with this limitation I feel that it is still partially useful in Colab. You can do all the rest of the plots like faceting by loss, by probability etc.
- This allows you to get an overall idea and if needed dig further by manually exploring images in the folders.
I feel that despite the limitations in Colab, it’s still a pretty strong feature for the majority of us who use Jupyter.
I added tests, more docs, examples and submitted a PR here - https://github.com/fastai/fastai/pull/1206.
just want to let you know, I used your code to integrate Facets Dive into my training loop, works great.
one thing still need to figure out is how to display a link on the right panel. I entered a html tag as value for one of the metadata key, but instead a link, it displayed the raw tag.
I have posted to Facets forum for help, have not received any suggestions yet, wonder if you have any ideas.