Even i tried bs thing. But got same problem. Didn’t find any solution till now
I have written a brief blog post based on my notes for lesson 4. Do have a look, and suggest if there are some errors/ missing points.
I can’t run this model on GCP with fastai 1.0.24.
I’ve tried reducing bs and bptt as small as I can and I still get CUDA out of memory on the 8GB GPU.
from fastai import *
from fastai.text import *
path = untar_data(URLs.IMDB)
data_lm = TextLMDataBunch.load(path, 'tmp_lm')
data_lm.bs = 1
learn = language_model_learner(data_lm, pretrained_model=URLs.WT103, drop_mult=0.3, bptt=1)
learn.fit(1)
I’ll try it in Gradient and if the problem persists I’ll maybe try to reduce the vocabulary size or look into how to debug CUDA memory usage.
I think, of all the differential learning rates he tried on Various NLP techniques, he found that particular hyperparameter kind of to be more appropriate.
In the lesson he did say that to explain the 2.6, but he also said that the ** 4 had another explanation that he never did explain.
If 2.6 ** 4 was found by just testing different learning rates, why is it 2.6 ** 4 and not the real value, 45,6976 ?
How to fine tune the language model and evaluate its performance?
It seems the model requires 9~10GB of memory. So you’ll need at least that much.
I don’t know what you can change to reduce this requirement.
Can someone tell me the approach to solve this type of problem?
Try running with bptt = 4. Worked fine for me on GCP.
In lesson3-imdb notebook, when trying to save the TextDataBunch I’m getting this error:
ValueError Traceback (most recent call last)
<ipython-input-9-e4b0eb2f0ee7> in <module>
----> 1 data_lm.save()
~/projects/fastai/fastai/text/data.py in save(self, cache_name)
106 cache_path = self.path/cache_name
107 pickle.dump(self.train_ds.vocab.itos, open(cache_path/f'itos.pkl', 'wb'))
--> 108 np.save(cache_path/f'train_ids.npy', self.train_ds.x.items)
109 np.save(cache_path/f'train_lbl.npy', self.train_ds.y.items)
110 np.save(cache_path/f'valid_ids.npy', self.valid_ds.x.items)
~/miniconda3/envs/fastai/lib/python3.6/site-packages/numpy/lib/npyio.py in save(file, arr, allow_pickle, fix_imports)
517
518 try:
--> 519 arr = np.asanyarray(arr)
520 format.write_array(fid, arr, allow_pickle=allow_pickle,
521 pickle_kwargs=pickle_kwargs)
~/miniconda3/envs/fastai/lib/python3.6/site-packages/numpy/core/numeric.py in asanyarray(a, dtype, order)
551
552 """
--> 553 return array(a, dtype, copy=False, order=order, subok=True)
554
555
ValueError: only one element tensors can be converted to Python scalars
In this notebook I can skip saving and continue, but I’m trying to train a language model from scratch (spanish wiki corpus) and I’m having the same problem.
I’d appreciate some help.
I have 11GB cards (personal DL box) and still cant get through the notebook as written, adding “empty_cache”, or changing “bs” up to this point. Parts where I have made it through later fail the following day. I will try more combinations as the day goes on, but I cannot get through it and it has been a week
You should create your own Tokenizer class then. Check the docs.
1 Like
Do I have to pass any argument in the data block API to specify I want a regression modeling when dealing with tabular data?
I’m trying to use fastai on the Kaggle House Price prediction dataset and even though I am typing the label column as float, the class of the databuch ys are CategoryList and the loss function cross_entropy
Here’s the outputs (I’m not running fastai on Google Colab): https://colab.research.google.com/drive/158oy-agO7TtSF29FReO7pM74IpovQR-8
Hello Jeremy ^^ so you said your vim rc is almost bare in this lesson but would you mind sharing your vim setup via Github anyways? For vim beginners, that might be a good starting point or good enough !
@lesscomfortable
I’m building a model on the belgian traffic light dataset but when I do recorder.plot(), all I get is an empty plot. I reduced the batch size to 32 but the plot is still empty. Any clues?
His vim config file was shared by Rachel in this thread during the lesson, you should be able to find it.
1 Like
@PierreO oh OK, thanks !
Is there currently a way to do regression tasks on tabular data? I was able to create a bunch on Tuesday before the library got updated, but there doesn’t seem to be a way to do regression with the datablock api.
1 Like
It was possible for me to create a databunch for regression problems up until Wednesday by setting c = 1. I assume fastai was updated on Wedesday? Because that method doesn’t work anymore, and the docs seems to be different as well. Does this mean we can’t do any regression with fastai at the moment?
It is entirely possible and will be covered in next class. See my answer here.
2 Likes