Exporting and deploy a trained fastai model

fastai
1

BeatConvModel is the name of the model that I have trained and these are some the layers invloved in the model.

BeatConvModel(
   (layers): Sequential(
   (0): Dropout(p=0.5, inplace=False)
   (1): Sequential(
      (0): Conv1d(1, 128, kernel_size=(55,), stride=(1,), padding=(27,), bias=False)
      (1): ReLU(inplace=True)
      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
   )
   (2): MaxPool1d(kernel_size=5, stride=5, padding=0, dilation=1, ceil_mode=False)
   (3): Dropout(p=0.5, inplace=False)
   (4): Sequential(
      (0): Conv1d(128, 128, kernel_size=(25,), stride=(1,), padding=(12,), bias=False)
      (1): ReLU(inplace=True)
      (2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  )

I haved saved the weights and exported the model like this:

learn.save('ECG_model')
learn.export()

But when I try to deploy the model in an application, I get this error.

AttributeError Traceback (most recent call last)
<ipython-input-3-b84658d87aa7> in <module>
  2 ECG = np.loadtxt('text-file.txt')
  3 path = Path('pth-of-the-.pkl-file')
  ----> 4 BeatConvModel = load_learner(path, 'export.pkl')
  5 pred_class,pred_idx,outputs = learn.predict(ECG)

 ~/anaconda3/envs/fastai/lib/python3.7/site-packages/fastai/basic_train.py in 
     load_learner(path, file, test, tfm_y, **db_kwargs)
  614     "Load a `Learner` object saved with `export_state` in `path/file` with empty data, 
    optionally add `test` and load on `cpu`. `file` can be file-like (file or buffer)"
  615     source = Path(path)/file if is_pathlike(file) else file
  --> 616     state = torch.load(source, map_location='cpu') if defaults.device == 
              torch.device('cpu') else torch.load(source)
  617     model = state.pop('model')
  618     src = LabelLists.load_state(path, state.pop('data'))

~/anaconda3/envs/fastai/lib/python3.7/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
527             with _open_zipfile_reader(f) as opened_zipfile:
528                 return _load(opened_zipfile, map_location, pickle_module, 
**pickle_load_args)
--> 529         return _legacy_load(opened_file, map_location, pickle_module, 
**pickle_load_args)
530 
531 

~/anaconda3/envs/fastai/lib/python3.7/site-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
700     unpickler = pickle_module.Unpickler(f, **pickle_load_args)
701     unpickler.persistent_load = persistent_load
--> 702     result = unpickler.load()
703 
704     deserialized_storage_keys = pickle_module.load(f, **pickle_load_args)

AttributeError: Can't get attribute 'BeatConvModel' on <module '__main__'>

I have deployed the model like this:

ECG = np.loadtxt('text-file.txt')
path = Path('pth-of-the-.pkl-file')
learner = load_learner(path, 'export.pkl')
pred_class,pred_idx,outputs = learn.predict(ECG)
print(pred_class)
print(pred_idx)
print(outputs)
2

The best is to define your custom model in a separate module both in your training env and your deployment env, so that the code is at the same place.

Otherwise, a quick fix is to copy and paste your code defining the model in your prod environment. In general, export and serialization saves the function names, not their codes (that is how python works).

3

@sgugger, this can be temporary solution. But my plan is to load the .pkl file from Cloud Storage and deploy it.

4

You still need to reference the custom model somehow, so it should either be stored in the prod environment or as a module as sgugger mentions.

6

So I tried deploying the model using pytorch in a new notebook. I defined the BeatConvModel and deployed it as follows:

PATH = Path(pkl file path)
model = torch.load(PATH)
model = BeatConvModel()
model.eval()
ECG = np.loadtxt(text file path)
ECG_data = torch.from_numpy(ECG).unsqueeze(0).float()
outputs = model.forward(ECG_data)
print(outputs)

Now the weird part is, instead of giving the prediction for 5 classes I am getting the predictions for 10 classes. As per my knowledge, the class info is stored in the .pkl file.

7

Just curious, but is your very first layer dropout ? ( or like you mentioned these are just an example of a few layers ? ) if dropout is the very first then you are never going to be using 50% of your data(p=0.5) is that your intention?
For the output being 10predictions, maybe pasting the whole model could help :slightly_smiling_face:

8

This model is being trained by someone else. I am only trying to deploy. The training of the model is still under progress.