Has anyone had any luck getting the open_nsfw model (written in Caffe) working with fast.ai? I’m working on a project to automate image moderation and was hoping to experiment with it.
What I’ve tried
I should preface all of this by saying that I’m very new to this. I’m about halfway through v2 of the course. Here’s what I’ve done so far:
First I converted the Caffe model to a pytorch model using MMdnn from Microsoft. That gave me two weights files (npy and pth) and the python model file. Once I did that, I imported my python model file import open_nsfw
.
I then defined the standard get_data
function but added an arch parameter:
def get_data(arch, sz, bs):
tfms = tfms_from_model(arch, sz, aug_tfms=transforms_side_on, max_zoom=1.1)
data = ImageClassifierData.from_paths(DATA_PATH, num_workers=6,tfms=tfms, bs=bs)
return data
I then proceeded to create a ConvLearner mostly like normal:
f = open_nsfw.KitModel
data = get_data(f, img_size, batch_size)
# models = ConvnetBuilder(f, data.c, data.is_multi, data.is_reg)
# models.model
learn = ConvLearner.pretrained(f, data, precompute=False)
What I’m seeing
This all worked seemingly well and good. However, when I call learn.fit(0.01, 3)
, the kernel dies every time without fail. I even decreased the batch size down to 1 just in case.
After adding a whole bunch of log statements to the fast.ai library, I found that the line that’s causing the crash is output = self.m(*xs)
inside the model.py file. I’ve done my best to trace back what this means: I believe this is calling the optimizer function that was determined by learner.py.
The only error message I get from jupyter is free(): invalid pointer
. I’ve tried searching around but this is understandably a very generic error message.
Any ideas?
I realize this is a longshot, and I’m a bit out of my league here, but I feel like I’m pretty close. When I started working on this, I wasn’t sure if I was going to be able to get anything working. But the fact that the open_nsfw model was pretrained with resnet50 gives me hope.
Any help or guidance of what to try next would be much appreciated!