Converting A PyTorch Model to Tensorflow or Keras for Production

Part 1 (2018)
21

@dhoa @jccj I’m struggling to convert my fastai model using onnx, and could use some help. I’m running everything on Google Colab. I checked to make sure my FastAI version was up to date, 1.0.50, and PyTorch, 1.0.1. I’ve tried loading my model two different ways:

learn = load_learner(’/path’, ‘model.pk1’) and
learn = learn.load(‘model.pth’)

Then I run:

dummy_input = Variable(torch.randn(1, 3, 224, 224))
torch.onnx.export(learn, dummy_input, “model.onnx”)

and get the error: ‘Learner’ object has no attribute ‘state_dict’

any thoughts?

22

What you want to export is the model (learn.model), not the learner object. Thus, using torch.onnx.export(learn.model, dummy_input, “model.onnx”).

1 Like
23

Thanks, that did the trick.

1 Like
24

@jccj Did you get the error: “BatchNormalization version 9 is not implemented.” when trying to convert from a .onnx model to a tensorflow model?

I found this on GitHub NotImplementedError: Cast version 9 is not implemented. #362, but it looks like the problem was resolved.

Any thoughts?

25

Does anyone have any experience with pytorch2keras?

I run:

dummy_input = Variable(torch.randn(1, 3, 224, 224))
k_model = pytorch_to_keras(learn.model, dummy_input, [(3, 224, 224,)], verbose=True)

and it runs for a while doing its thing, then I get this error:

‘1.3.running_mean’

26

hello,
i want to convert my pytorch model to tensorflow, so first i have to convert it to onnx first and then onnx to tensorflow. but when i am converting to onnx i am getting error. can someone solve this error.

this is the code

import torch.onnx

from torch.autograd import Variable
model= open(“model_weights.pth”, “w”)
dummy_input = Variable(torch.randn(1, 3, 137, 137))
// since i am giving a 137*137 image as a input i have written this not sure wheather it is right or not. also i dont know what does third parameter mean “net.onnx” is it just output file name.
torch.onnx.export(model,dummy_input,“net.onnx”)

i am getting error as
AttributeError Traceback (most recent call last)
in ()
3 model= open(“model_weights.pth”, “w”)
4 dummy_input = Variable(torch.randn(1, 3, 137, 137))
----> 5 torch.onnx.export(model,dummy_input,“net.onnx”)

/usr/local/lib/python3.6/dist-packages/torch/onnx/init.py in export(*args, **kwargs)
25 def export(*args, **kwargs):
26 from torch.onnx import utils
—> 27 return utils.export(*args, **kwargs)
28
29

/usr/local/lib/python3.6/dist-packages/torch/onnx/utils.py in export(model, args, f, export_params, verbose, training, input_names, output_names, aten, export_raw_ir, operator_export_type)
102 operator_export_type = OperatorExportTypes.ONNX
103 _export(model, args, f, export_params, verbose, training, input_names, output_names,
–> 104 operator_export_type=operator_export_type)
105
106

/usr/local/lib/python3.6/dist-packages/torch/onnx/utils.py in _export(model, args, f, export_params, verbose, training, input_names, output_names, operator_export_type, export_type, example_outputs, propagate)
279 training, input_names,
280 output_names, operator_export_type,
–> 281 example_outputs, propagate)
282
283 # TODO: Don’t allocate a in-memory string for the protobuf

/usr/local/lib/python3.6/dist-packages/torch/onnx/utils.py in _model_to_graph(model, args, f, verbose, training, input_names, output_names, operator_export_type, example_outputs, propagate)
222 raise RuntimeError(’‘forward’ method must be a script method’)
223 else:
–> 224 graph, torch_out = _trace_and_get_graph_from_model(model, args, training)
225 params = list(_unique_state_dict(model).values())
226

/usr/local/lib/python3.6/dist-packages/torch/onnx/utils.py in _trace_and_get_graph_from_model(model, args, training)
182 # A basic sanity check: make sure the state_dict keys are the same
183 # before and after running the model. Fail fast!
–> 184 orig_state_dict_keys = _unique_state_dict(model).keys()
185
186 # By default, training=False, which is good because running a model in

/usr/local/lib/python3.6/dist-packages/torch/jit/init.py in _unique_state_dict(module, keep_vars)
199
200 def _unique_state_dict(module, keep_vars=False):
–> 201 state_dict = module.state_dict(keep_vars=keep_vars)
202 filtered_dict = type(state_dict)()
203 seen_ids = set()

AttributeError: ‘_io.TextIOWrapper’ object has no attribute ‘state_dict’

27

No, I didn’t. Though I used

from onnx_tf.backend import prepare
model = onnx.load('export.onnx') # Load the ONNX file
tf_rep = prepare(model)
28

@mnandu I’m not sure about what your error means, I don’t have enough experience. This worked for me, so try this:

learn = learn.load(‘model.pth’)
dummy_input = Variable(torch.randn(1, 3, 137, 137))
torch.onnx.export(learn.model, dummy_input, “net.onnx”)

I’m not familiar with

model= open(“model_weights.pth”, “w”)

I haven’t seen it in the fastai library. From what I know, you can can save/export and open a model one of two ways:

learn.export(‘trained_model.pk1’)
learn = load_learner(’/path’, ‘trained_model.pk1’)

or

learn.save = (“trained_model”)
learn = learn.load(“trained_model.pth”)

The third argument, “net.onnx” is your output file name.

I was able to convert my model to a .onnx model, but now I am having trouble trying to convert to TF. I’m looking at other options too: MMdnn and pytorch2keras.

29

If you look here, you can see the modification made. You can either install onnx-tensorflow from master or possibly just update your version of onnx_tf/handlers/backend/batch_normalization.py to add these lines, though I can’t guarantee the latter option will work in your case.

30

You need to install the latest version(from github instead of pypi). They haven’t cut a new release since October :frowning:

pip install git+https://github.com/onnx/onnx-tensorflow.git
31

I’ve created a GitHub gist for BertForSequenceClassification model conversion from PyTorch transformers to TensorFlow.

32

I do not understand what it means to transpose the weight.
I have pytorch model weights and bias. How can I assign to keras/tensorflow model?

33

Hi all, this is a naive question but since fastai has great inference tools is there a tool to convert all other models (TF, Keras etc) to fastai/pytorch? I can’t seem to find such a tool and don’t really understand why there isn’t one?