Learn.get_preds() on the training data

Part 1 (2020)
1

Hello,
i want to to get the predictions for the training data, I tried this:
image
image
any fast way to do that?

1 Like
2

Try ds_idx=0 rather than the complicated dl=learn2.dls.train

3 Likes
3

if i want to predict test data, how should I do?

4

Please take a look at this.

5

Can you please explain, what this parameter does?

6

It specifies what dataloader packed in the Learner we run get_preds on. Eg an idx of 0 is the training dataloader, and 1 is the validation dataloader

And passing in a dataloader with the dl parameter allows for any dataloader you want, even if not tied to the Learner directly

1 Like
7

Thank you for your answer! I trained a model, saved it and loaded it back with load_learner, but this way the dls are empty. That is why my code did not run when I specified ds_idx=0.

When I do it the “complicated” way with dl=dls.train I run into an index problem.

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
/tmp/ipykernel_4035/2420368553.py in <module>
----> 1 y_pred, y_true = tower_detector.get_preds(dl=dls.train)

~/miniconda3/envs/pgaiNew/lib/python3.9/site-packages/fastai/learner.py in get_preds(self, ds_idx, dl, with_input, with_decoded, with_loss, act, inner, reorder, cbs, **kwargs)
    258                 res[pred_i] = act(res[pred_i])
    259                 if with_decoded: res.insert(pred_i+2, getattr(self.loss_func, 'decodes', noop)(res[pred_i]))
--> 260             if reorder and hasattr(dl, 'get_idxs'): res = nested_reorder(res, tensor(idxs).argsort())
    261             return tuple(res)
    262         self._end_cleanup()

~/miniconda3/envs/pgaiNew/lib/python3.9/site-packages/fastai/torch_core.py in nested_reorder(t, idxs)
    712     "Reorder all tensors in `t` using `idxs`"
    713     if isinstance(t, (Tensor,L)): return t[idxs]
--> 714     elif is_listy(t): return type(t)(nested_reorder(t_, idxs) for t_ in t)
    715     if t is None: return t
    716     raise TypeError(f"Expected tensor, tuple, list or L but got {type(t)}")

~/miniconda3/envs/pgaiNew/lib/python3.9/site-packages/fastai/torch_core.py in <genexpr>(.0)
    712     "Reorder all tensors in `t` using `idxs`"
    713     if isinstance(t, (Tensor,L)): return t[idxs]
--> 714     elif is_listy(t): return type(t)(nested_reorder(t_, idxs) for t_ in t)
    715     if t is None: return t
    716     raise TypeError(f"Expected tensor, tuple, list or L but got {type(t)}")

~/miniconda3/envs/pgaiNew/lib/python3.9/site-packages/fastai/torch_core.py in nested_reorder(t, idxs)
    711 def nested_reorder(t, idxs):
    712     "Reorder all tensors in `t` using `idxs`"
--> 713     if isinstance(t, (Tensor,L)): return t[idxs]
    714     elif is_listy(t): return type(t)(nested_reorder(t_, idxs) for t_ in t)
    715     if t is None: return t

~/miniconda3/envs/pgaiNew/lib/python3.9/site-packages/fastai/torch_core.py in __torch_function__(self, func, types, args, kwargs)
    338         convert=False
    339         if _torch_handled(args, self._opt, func): convert,types = type(self),(torch.Tensor,)
--> 340         res = super().__torch_function__(func, types, args=args, kwargs=kwargs)
    341         if convert: res = convert(res)
    342         if isinstance(res, TensorBase): res.set_meta(self, as_copy=True)

~/miniconda3/envs/pgaiNew/lib/python3.9/site-packages/torch/tensor.py in __torch_function__(cls, func, types, args, kwargs)
    993 
    994         with _C.DisableTorchFunction():
--> 995             ret = func(*args, **kwargs)
    996             return _convert(ret, cls)
    997 

IndexError: index 57943 is out of bounds for dimension 0 with size 57920
8

I managed to get the predictions by loading my trained model into a new Learner like this:

new_learner = Learner(dls, model=loaded_learner.model)
predictions, labels = my_learner.get_preds(0)