So I was able to successfully predict using the above code. I tested this out on the first row of the df, so df.iloc[0,:] and gave a prediction that seems logical.
When I try to do the same on the second row though, and the others after it, I get an error:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-65-8d5362871fe4> in <module>()
----> 1 model(V(cat), V(cont))
~\AppData\Local\conda\conda\envs\fastai-cpu\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
355 result = self._slow_forward(*input, **kwargs)
356 else:
--> 357 result = self.forward(*input, **kwargs)
358 for hook in self._forward_hooks.values():
359 hook_result = hook(self, input, result)
~\AppData\Local\conda\conda\envs\fastai-cpu\lib\site-packages\fastai\column_data.py in forward(self, x_cat, x_cont)
113 def forward(self, x_cat, x_cont):
114 if self.n_emb != 0:
--> 115 x = [e(x_cat[:,i]) for i,e in enumerate(self.embs)]
116 x = torch.cat(x, 1)
117 x = self.emb_drop(x)
~\AppData\Local\conda\conda\envs\fastai-cpu\lib\site-packages\fastai\column_data.py in <listcomp>(.0)
113 def forward(self, x_cat, x_cont):
114 if self.n_emb != 0:
--> 115 x = [e(x_cat[:,i]) for i,e in enumerate(self.embs)]
116 x = torch.cat(x, 1)
117 x = self.emb_drop(x)
~\AppData\Local\conda\conda\envs\fastai-cpu\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
355 result = self._slow_forward(*input, **kwargs)
356 else:
--> 357 result = self.forward(*input, **kwargs)
358 for hook in self._forward_hooks.values():
359 hook_result = hook(self, input, result)
~\AppData\Local\conda\conda\envs\fastai-cpu\lib\site-packages\torch\nn\modules\sparse.py in forward(self, input)
101 input, self.weight,
102 padding_idx, self.max_norm, self.norm_type,
--> 103 self.scale_grad_by_freq, self.sparse
104 )
105
~\AppData\Local\conda\conda\envs\fastai-cpu\lib\site-packages\torch\nn\_functions\thnn\sparse.py in forward(cls, ctx, indices, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
55
56 if indices.dim() == 1:
---> 57 output = torch.index_select(weight, 0, indices)
58 else:
59 output = torch.index_select(weight, 0, indices.view(-1))
RuntimeError: index out of range at c:\miniconda2\conda-bld\pytorch-cpu_1519449358620\work\torch\lib\th\generic/THTensorMath.c:277
Not sure what this is. Of course I should say that my initial set up might be wrong in itself. What I am not sure is how to construct the model and new df and how to pass it through the model. In this case, I passed the (unseen) data through a proc_df in a similar way like we would do for the testing dataset. However, not sure if I should be preprocessing that. Even if I do, given that this data is unseen, how would the model know to assign weights in the specific categorical index these values get?
What I am going to try next is to allow the unseen data as a test or validation set. This way I am hoping that the preprocessing is consistent and that I can use the df created to predict after training. Is this the way I should be doing this? Has anyone else tried to predict on completely unseen data for this method?
Thanks in advance!
Kind regards,
Theodore.