Issue viewing images in Voila

[amritv (asvcode)]

Trying to figure out why displaying images via voila do not work in some cases, for example using this:

source = untar_data(URLs.IMAGENETTE_160)
items = get_image_files(source)
img = items[7]
imgs = Image.open(img)

this correctly displays the image in voila:

#display image
print('display original image')
imgs = display(Image.open(img))

and so does this:

#display aug image
print('display transform image with zero padding and resize')
pil_img = PILImage(PILImage.create(img).resize((213,160)))
rsz = Resize(224, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros)
display(show_image(rsz(pil_img)))

But when you do this (both images are to be displayed side by side):

def aug_show():
aug_button = widgets.Button(description='click')
display(aug_button)
aug_out = widgets.Output()
display(aug_out)
def on_aug_button(b):
    with aug_out:
        clear_output()
        j = widgets.Output()
        u = widgets.Output()
        with j:
            print(type(img))
            display(Image.open(img))
        with u:
            pil_img = PILImage(PILImage.create(img).resize((213,160)))
            rsz = Resize(224, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros)
            print(type(pil_img))
            display(show_image(rsz(pil_img)))
        display(HBox([j, u]))
aug_button.on_click(on_aug_button)

This is what is displayed:

You can see that the two blocks are printing as the image type is shown side by side but the second image does not show up. I looked at this thread Use of display after notebook execution does not show · Issue #532 · voila-dashboards/voila · GitHub but have not been able to figure a way around this.

Wondering if there is anyone may know what is going on?

[radek]

I am not sure if this is correct, but I suspect this might be helpful.

Maybe try if the following will work:

hbox = HBox([show_image(rsz(pil_img), Image.open(img), show_image(rsz(pil_img))])
display(hbox)

If yes, then you can add a title to the things on the right by sticking them in their own box.

Not sure if this is going to help - the docs are very good though, probably they can point you in the right direction.

[amritv (asvcode)]

Thanks @radek! for the info and your suggestion.

Unfortunately it did not work resulting in this error:

The thing is the original code does work, I can place two images side by side using Image.open, for example;

def aug_show2():
aug_button = widgets.Button(description='click')
display(aug_button)
aug_out = widgets.Output()
display(aug_out)
def on_aug_button(b):
    with aug_out:
        clear_output()
        j = widgets.Output()
        u = widgets.Output()
        with j:
            print(type(img))
            display(Image.open(img))
        with u:
            print(type(img))
            display(Image.open(img))
        display(HBox([j, u]))
aug_button.on_click(on_aug_button)

I have to dig deeper to see why they don’t render properly. Thanks for your suggestion

[radek]

Ha, thank you, I think we are onto something

If I am reading this error correctly, it is just saying that it wants widget to go into an HBox. I wonder if an HBox can go into an HBox?

If it could, then we could just do something like this maybe:

instead of:

display(show_image(rsz(pil_img)))
display(HBox([j, u]))

maybe

display(VBox([
    display(show_image(rsz(pil_img))),
    HBox([j, u])
])

I am not sure if this should work. My thinking is that maybe display (outside of context of a widgets.Output) should only be called once in a jupyter notebook cell. I am not really sure how that works.

Anyhow, sorry if this is not helpful Just started playing with ipywidgets - didn’t suspect they would be so awesome

[amritv (asvcode)]

I love ipywidgets

It unfortunately did not work. I think the issue is the way voila renders images using widgets.Output() with show_image and show_batch

for example without using widgets.Output() like in this example it works:

#display aug image
print('display transform image with zero padding and resize')
pil_img = PILImage(PILImage.create(img).resize((213,160)))
rsz = Resize(224, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros)
display(show_image(rsz(pil_img)))

but when you use widgets.Output() like in this example:

#display aug image
but = widgets.Button(description='pil_img')
display(but)
outp = widgets.Output()
display(outp)
def butt(b):
    with outp:
        print('display transform image with zero padding and resize')
        pil_img = PILImage(PILImage.create(img).resize((213,160)))
        rsz = Resize(224, method=ResizeMethod.Pad, pad_mode=PadMode.Zeros)
        display(show_image(rsz(pil_img)))
but.on_click(butt)

the output is:

with no image displayed.

both show_image and show_batch use matplotlib methods for plotting and somehow they don’t render in voila(when using Output). (note that they work fine within jupyter)

[amritv (asvcode)]

Update: I had to customize show_image to display without using ax to get it to work. It seems that widget.Output() does not like to display images using ax. Using this instead now works:

def show_imagee(im, **kwargs):
    "Show_image helper for viewing images in Voila"
    # Handle pytorch axis order
    if hasattrs(im, ('data','cpu','permute')):
        im = im.data.cpu()
        if im.shape[0]<5: im=im.permute(1,2,0)
    elif not isinstance(im,np.ndarray): im=array(im)
    # Handle 1-channel images
    if im.shape[-1]==1: im=im[...,0]
    it = Tensor(im)
    img = Image.fromarray(im, 'RGB')
    display(img)

So now it works in voila (the second smaller image was the one that was not showing before)