Training Model with Multiple Inputs (i.e. Two stream CNN)

Part 1 (2018)
1

Hello

I’m trying to use fast-ai on top pytorch in order to simplify training a model in pytorch. I have successfully created a learner (learn= ConvLearner(model, data)) and train a model with my own model and data.

However, I am having issues when I am trying to train a model with multiple inputs. I see the internals of fast-ai code use *x or *y and saw an example of a dataset with mutiple outputs.

I have a model with two inputs in the forward() pass. The dataset returns the following: ((input_1, input_2), label). If I change it to only one input in the forward apss and the dataset it works but if I leave it as multiple inputs it gives me an error on learn.fit()

Error: "only 1-element tensors can be converted " "to Python scalars"

I already confirmed that next(iter(md.trn_dl)) return a list of input tensors for input and a tensor for output.

Anyone with experience on fast-ai and multiple inputs. Is fast-ai compatible with multiple inputs.

Regards

2

You could try to pass a Stepper to learner.fit similar to this notebook -https://github.com/fastai/fastai/blob/master/courses/dl2/translate.ipynb

In the Stepper.step you can do the custom forward pass, loss, gradient step and return the loss value.

3

Thanks ramesh.
I can see that you can create or modify Stepper.step in fastai core.py. However, it seems all the logistic is there for it to work multiple inputs data inputs. My forward pass already expects multiple inputs and so does the stepper. I have been debugging the error and the error seems to be due to np.ascontiguousarray giving the error.

It seems that even if it now it is a list islisty(x) returns true. It is still somehow passing the entire list to method V_(o, requires_grad, volatile) which leads to error. It seems to be a bug with the code. I’ll keep debugging to try and fix it.

What you mention is more closer to Sequence2Seq model but my model is just a dataset with two inputs.

4

If this is something you can share or replicate the problem using open datasets, I would suggest you create a gist.github.com so that it will be easier to see the issue.

5

I can mark this problem as solved.

Basically in order to use multiple inputs the dataset output should be as followed:
Dataset class:
return input_1, input_2, ..., input_n, y

I was doing:
return (input_1, input_2), y
But this throws and error because FastAI sees it as a list of list and unable to unwrapped it properly. So all inputs must be past as comma separated in the return of your dataset get_item method.

Thanks.

6 Likes
6

@avn3r could you provide an example of multiple inputs for the BaseDataset/ModelData classes you’re deriving.
I’m confused with fastai’s classes and how to achieve what you’ve just said

1 Like
7

@avn3r nevermind, when all hopes were lost, it seems overriding “def get1item(self, i)” as you said solved the issue :wink:

Thanks for the pointer.

1 Like