Open_nsfw (or any custom model) + fast.ai library

Has anyone had any luck getting the open_nsfw model (written in Caffe) working with fast.ai? I’m working on a project to automate image moderation and was hoping to experiment with it.

What I’ve tried

I should preface all of this by saying that I’m very new to this. I’m about halfway through v2 of the course. Here’s what I’ve done so far:

First I converted the Caffe model to a pytorch model using MMdnn from Microsoft. That gave me two weights files (npy and pth) and the python model file. Once I did that, I imported my python model file import open_nsfw.

I then defined the standard get_data function but added an arch parameter:

def get_data(arch, sz, bs):
    tfms = tfms_from_model(arch, sz, aug_tfms=transforms_side_on, max_zoom=1.1)
    data = ImageClassifierData.from_paths(DATA_PATH, num_workers=6,tfms=tfms, bs=bs)
    return data

I then proceeded to create a ConvLearner mostly like normal:

f = open_nsfw.KitModel
data = get_data(f, img_size, batch_size)
# models = ConvnetBuilder(f, data.c, data.is_multi, data.is_reg)
# models.model
learn = ConvLearner.pretrained(f, data, precompute=False)

What I’m seeing

This all worked seemingly well and good. However, when I call learn.fit(0.01, 3), the kernel dies every time without fail. I even decreased the batch size down to 1 just in case.

After adding a whole bunch of log statements to the fast.ai library, I found that the line that’s causing the crash is output = self.m(*xs) inside the model.py file. I’ve done my best to trace back what this means: I believe this is calling the optimizer function that was determined by learner.py.

The only error message I get from jupyter is free(): invalid pointer. I’ve tried searching around but this is understandably a very generic error message.

Any ideas?

I realize this is a longshot, and I’m a bit out of my league here, but I feel like I’m pretty close. When I started working on this, I wasn’t sure if I was going to be able to get anything working. But the fact that the open_nsfw model was pretrained with resnet50 gives me hope.

Any help or guidance of what to try next would be much appreciated!

Hello,

When I tried to import a pretrained model (xception) into fast.ai library, I had the same error message while the kernel died each time.
While searching for a solution, I came across this: https://github.com/pytorch/pytorch/issues/2507. Unfortunately, it did not work.

Did you end up finding a solution for your issue? Thanks.

1 Like

I’m also having the same problem, have you found any solution? Thanks!