Is it possible to change the batch size for existing data loaders for a language model? I tried changing the dls.bs attribute but the actual batch size doesn’t seem to be affected. Is there a better way to do this without creating the data loaders from scratch?
print(learn.dls.bs)
128
learn.dls.bs = 64
print(learn.dls.bs)
64
x, y = learn.dls.one_batch()
print(x.shape)
torch.Size([128, 72])
@Hmamin try adjusting the learn.dls.train.bs or valid? 
Yep, that fixed it. Thanks!
I’ve made some progress on this. It appears the tensors are getting blanked out on the grad function:
I’ve removed grad from the intrinsic_attention function, and it works now!
Used to be:
attn = emb.grad.squeeze().abs().sum(dim=-1)
Changed to:
attn = emb.squeeze().abs().sum(dim=-1)
Full function:
def intrinsic_attention(learn, text, class_id=None):
"Calculate the intrinsic attention of the input w.r.t to an output `class_id`, or the classification given by the model if `None`."
learn.model.train()
_eval_dropouts(learn.model)
learn.model.zero_grad()
learn.model.reset()
dl = learn.dls.test_dl([text])
batch = dl.one_batch()[0]
emb = learn.model[0].module.encoder(batch).detach().requires_grad_(True)
lstm = learn.model[0].module(emb, True)
learn.model.eval()
cl = learn.model[1]((lstm, torch.zeros_like(batch).bool(),))[0].softmax(dim=-1)
if class_id is None: class_id = cl.argmax()
cl[0][class_id].backward()
attn = emb.squeeze().abs().sum(dim=-1)
attn /= attn.max()
tok, _ = learn.dls.decode_batch((*tuplify(batch), *tuplify(cl)))[0]
return tok, attn
It concerns me that the earlier emb line has “requires_grad_(True)”, but I changed it to false and the result was the same.
Is the result inherently flawed without .grad on emb?
Wanted to post this quick example of running inference after you’ve created a text classifier using FastAI v2. It’s a little different than v1 and took me a little bit of time to find out.
from fastai2.text.all import *
defaults.device = torch.device('cpu')
path = Path('.')
learner = load_learner("./export.pkl")
f = open("/tmp/test.txt", "r")
test_file_contents = f.read()
_, _, losses = learner.predict(test_file_contents)
cats = [learner.dls.categorize.decode(i) for i in range(len(losses))]
predictions = sorted(
zip(cats, map(float, losses)),
key=lambda p: p[1],
reverse=True
)
print(predictions)
I can add a pull request to put this somewhere in the docs if wanted, but I’m not sure where it should go.
This might sound dumb but can’t this be done with
items = pd.read_csv("/tmp/test.txt", sep = '\t')
test_dl = learner.dls.test_dl(items.values)
learner.get_preds(dl=test_dl, with_decoded=False)
I should clarify, my example is for classifying one item at a time, for say, a webservice. It gets you a response for that item that looks something like:
{"predictions":[["positive",0.8328688740730286],["negative",0.06257648020982742],["neutral",0.054476577788591385]]}
So the response includes the category names as well. Pulling out the category names was the part that I had difficulty with: The magic is in learner.dls.categorize.decode(i), which gets you the name of the ith category.
I just updated to the latest fastai2 and fastcore files on git. I’m loading a learner I trained last week, and I’m getting:
Traceback (most recent call last):
File "./newfile.py", line 392, in <module>
learn = load_learner('fastai2_05_23.pkl', cpu=False)
File "/home/paperspace/git_packages/fastai2/fastai2/learner.py", line 526, in load_learner
res = torch.load(fname, map_location='cpu' if cpu else None)
File "/home/paperspace/environments/fastai2/lib/python3.6/site-packages/torch/serialization.py", line 593, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/paperspace/environments/fastai2/lib/python3.6/site-packages/torch/serialization.py", line 773, in _legacy_load
result = unpickler.load()
AttributeError: Can't get attribute 'ModelReseter' on <module 'fastai2.callback.rnn' from '/home/paperspace/git_packages/fastai2/fastai2/callback/rnn.py'>
I’m still investigating, but has anyone else seen this?
Note, the original learner doesn’t have that callback:
callbacks = [SaveModelCallback(monitor = 'accuracy',
every_epoch = False),
EarlyStoppingCallback(monitor = 'accuracy', patience=25),
ReduceLROnPlateau(factor=10, patience=1000, min_lr = 0.000001)]
learn = text_classifier_learner(dbunch_class, AWD_LSTM, drop_mult=0.5, metrics=[accuracy, Perplexity()], cbs=callbacks, wd=0.1).to_fp16()
learn = learn.load_encoder('finetuned_67.pkl')
Update:
When I load the saved learner, instead of the exported learner, I get a different error:
>>> learn.load('fastai2_learner_05_23_2')
<fastai2.text.learner.TextLearner object at 0x7f249d3e4d30>
>>>
>>>
>>> learn.predict("Superman")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/paperspace/git_packages/fastai2/fastai2/learner.py", line 251, in predict
dec = self.dls.decode_batch(inp + tuplify(dec_preds))[0]
File "/home/paperspace/git_packages/fastai2/fastai2/data/core.py", line 78, in decode_batch
def decode_batch(self, b, max_n=9, full=True): return self._decode_batch(self.decode(b), max_n, full)
File "/home/paperspace/git_packages/fastai2/fastai2/data/core.py", line 83, in _decode_batch
return L(batch_to_samples(b, max_n=max_n)).map(f)
File "/home/paperspace/git_packages/fastcore/fastcore/foundation.py", line 377, in map
return self._new(map(g, self))
File "/home/paperspace/git_packages/fastcore/fastcore/foundation.py", line 327, in _new
def _new(self, items, *args, **kwargs): return type(self)(items, *args, use_list=None, **kwargs)
File "/home/paperspace/git_packages/fastcore/fastcore/foundation.py", line 47, in __call__
res = super().__call__(*((x,) + args), **kwargs)
File "/home/paperspace/git_packages/fastcore/fastcore/foundation.py", line 318, in __init__
items = list(items) if use_list else _listify(items)
File "/home/paperspace/git_packages/fastcore/fastcore/foundation.py", line 254, in _listify
if is_iter(o): return list(o)
File "/home/paperspace/git_packages/fastcore/fastcore/foundation.py", line 220, in __call__
return self.fn(*fargs, **kwargs)
File "/home/paperspace/git_packages/fastcore/fastcore/utils.py", line 349, in _inner
for f in L(funcs): x = f(x, *args, **kwargs)
File "/home/paperspace/git_packages/fastai2/fastai2/data/core.py", line 297, in decode
def decode(self, o, full=True): return tuple(tl.decode(o_, full=full) for o_,tl in zip(o,tuplify(self.tls, match=o)))
File "/home/paperspace/git_packages/fastai2/fastai2/data/core.py", line 297, in <genexpr>
def decode(self, o, full=True): return tuple(tl.decode(o_, full=full) for o_,tl in zip(o,tuplify(self.tls, match=o)))
File "/home/paperspace/git_packages/fastai2/fastai2/data/core.py", line 233, in decode
def decode(self, o, **kwargs): return self.tfms.decode(o, **kwargs)
File "/home/paperspace/git_packages/fastcore/fastcore/transform.py", line 195, in decode
if full: return compose_tfms(o, tfms=self.fs, is_enc=False, reverse=True, split_idx=self.split_idx)
File "/home/paperspace/git_packages/fastcore/fastcore/transform.py", line 140, in compose_tfms
x = f(x, **kwargs)
File "/home/paperspace/git_packages/fastcore/fastcore/transform.py", line 73, in decode
def decode (self, x, **kwargs): return self._call('decodes', x, **kwargs)
File "/home/paperspace/git_packages/fastcore/fastcore/transform.py", line 82, in _call
return self._do_call(getattr(self, fn), x, **kwargs)
File "/home/paperspace/git_packages/fastcore/fastcore/transform.py", line 86, in _do_call
return x if f is None else retain_type(f(x, **kwargs), x, f.returns_none(x))
File "/home/paperspace/git_packages/fastcore/fastcore/dispatch.py", line 98, in __call__
return f(*args, **kwargs)
File "/home/paperspace/git_packages/fastai2/fastai2/text/data.py", line 46, in decodes
def decodes(self, o): return L(self.vocab[o_] for o_ in o if self.vocab[o_] != self.pad_tok)
File "/home/paperspace/git_packages/fastcore/fastcore/foundation.py", line 47, in __call__
res = super().__call__(*((x,) + args), **kwargs)
File "/home/paperspace/git_packages/fastcore/fastcore/foundation.py", line 318, in __init__
items = list(items) if use_list else _listify(items)
File "/home/paperspace/git_packages/fastcore/fastcore/foundation.py", line 254, in _listify
if is_iter(o): return list(o)
File "/home/paperspace/git_packages/fastai2/fastai2/text/data.py", line 46, in <genexpr>
def decodes(self, o): return L(self.vocab[o_] for o_ in o if self.vocab[o_] != self.pad_tok)
AttributeError: 'Numericalize' object has no attribute 'pad_tok'
>>>
Update 2: Ended up reverting to an older fastai2 build, no longer seeing this. Let me know if there’s anything you’d like me to test!
Hi chess Hope you are having a wnderful day!
I am not sure if it is related but I and others person have been having issues with callbacks on Google colab.
Until I saw your post I had assumed it was Google Colab related. However having seen your email and you are using Paperspace It looks more like a library issue.
I don’t know if its been resolved, but but the user was going to raise a PR to resolve the following issue “savemodelcallback not working for collaborative filtering”
I have checked this site and have not seen a PR or Issue for it pending.
It might be worth raising a PR or Issue.
Cheers mrfabulous1 
Thanks for the help!
That link gives me an:
“# Oops! That page doesn’t exist or is private.“
Is that the right link?
Hi chess the previous link was private So I updated my previous post.
Cheers mrfabulous1 
Thanks for the update! I have the same issue on my local machine. Installed from scratch with Paperspace and saw it there too.
I suspect I might just have to retrain all my models. Will try that when I have time, and submit an issue if it doesn’t help.
Thanks!
Confirmed, it works fine after retraining the models.
FYI, in case people ever run into an issue of “where is my vocab or name of my classes on a text model”, it lives in learn.dls.categorize.vocab, considering the fact we have a vocab that lives in learn.dls.vocab
im not seeing categorize in learner.dls. Just learner.dls.cat? I’ve just updated fastaiv2 today as well, which may be the issue.
How did you build your DataLoader? Categorize is the transform
I just used
dl= learner.dls.test_dl(df[‘Message’])
I’m seeing it just fine here:
dls = TextDataLoaders.from_csv(path, 'texts.csv', text_col='text', label_col='label')
dls.categorize
And on the individual DataLoaders as well
(this is the IMDB_Sample and I’m running 0.0.17)
Ahhhh. You aren’t labeling it. You should also pass in with_labels=True to your call to test_dl
That’s it. Sorry about that! That makes much more sense.