Lesson 8, notebook 10_nlp, OSError: Not found: unknown field name “minloglevel” in TrainerSpec

Part 1 (2020)
1

Hi All,
Apologies in advance if I am submitting beginner questions in the wrong category.
Going through the lesson 8 video and running notebook 10_nlp alongside. I get an error trace (below) after cell # 12 which runs

subword(1000)

Having searched thru the forums, I suspect I need to install the latest version of fastai2 and fastcore but I’m not really sure and any guidance and clarification would be appreciated. I am running a notebook on Paperspace.

OSError Traceback (most recent call last)
in
----> 1 subword(1000)

in subword(sz)
1 def subword(sz):
2 sp = SubwordTokenizer(vocab_sz=sz)
----> 3 sp.setup(txts)
4 return ’ '.join(first(sp([txt]))[:40])

/opt/conda/envs/fastai/lib/python3.7/site-packages/fastai2/text/core.py in setup(self, items, rules)
366 for t in progress_bar(maps(*rules, items), total=len(items), leave=False):
367 f.write(f’{t}\n’)
→ 368 sp_model = self.train(raw_text_path)
369 self.tok = SentencePieceProcessor()
370 self.tok.Load(str(sp_model))

/opt/conda/envs/fastai/lib/python3.7/site-packages/fastai2/text/core.py in train(self, raw_text_path)
354 f"–character_coverage={self.char_coverage} --model_type={self.model_type}“,
355 f”–unk_id={len(spec_tokens)} --pad_id=-1 --bos_id=-1 --eos_id=-1 --minloglevel=2",
→ 356 f"–user_defined_symbols={‘,’.join(spec_tokens)}"]))
357 raw_text_path.unlink()
358 return self.cache_dir/‘spm.model’

/opt/conda/envs/fastai/lib/python3.7/site-packages/sentencepiece.py in _sentencepiece_trainer_train(arg, **kwargs)
387 “”“Train Sentencepiece model. Accept both kwargs and legacy string arg.”“”
388 if arg is not None and type(arg) is str:
→ 389 return SentencePieceTrainer.TrainFromString(arg)
390
391 def _encode(value):

/opt/conda/envs/fastai/lib/python3.7/site-packages/sentencepiece.py in TrainFromString(arg)
195 @staticmethod
196 def TrainFromString(arg):
→ 197 return _sentencepiece.SentencePieceTrainer_TrainFromString(arg)
198
199 @staticmethod

OSError: Not found: unknown field name “minloglevel” in TrainerSpec.

1 Like
2

Hi rj9 Hope all is well!

I am getting the exact same error, have you managed to resolve it.

Cheers mrfabulous1 :smiley: :smiley:

1 Like
3

I just ran the notebook, latest version of fastai2 libs (fastai2, fastcore, course-v4 repos), and cell #12 [subword(1000)] ran fine. Can you check your versions of fastcore/fastai2/sentencepiece, and update/upgrade where needed, then try again?

Some of my relevant library versions listed below. Note that my fastcore and fastai2 are both with ‘editable pip install’, and are the latest version (having just done git pull) as of right now. sentencepiece seems to have a 0.1.90 (on their Github), but mine is older.

Yijin

Name: fastcore
Version: 0.1.18

Name: fastai2
Version: 0.0.18

Name: sentencepiece
Version: 0.1.83
2 Likes
4

Hi utkb I hope are also having a wonderful day!

Thank you very much for you reply it worked perfectly, I normally record all my library versions once I get a working notebook, so I know when somethings changed, but this is my first run through the notebook. Your help saved me a lot of time.

Fastai2 and Fastcore were fine, but sentencepiece was ver 0.1.90.

Thanks again mrfabulous1 :grinning: :grinning:

1 Like
5

Hi utkb, thanks for the info!

Checking out the library versions, I have:

sentencepiece – 0.1.90
fastcore – 0.1.18
fastai2 – 0.0.18

However, I don’t believe my fastcore and fastai2 were done with an editable install. Trying now to move forward with an editable install but I may be doing it wrong. I have tried the following:

!pip install -e “.[dev]” git+https://github.com/fastai/fastcore.git
!pip install -e “.[dev]” git+https://github.com/fastai/fastai2.git

as well as:

!git clone GitHub - fastai/fastcore: Python supercharged for the fastai library
!cd fastcore
!pip install -e “.[dev]”

!git clone GitHub - fastai/fastai2: Temporary home for fastai v2 while it's being developed
!cd fastai2
!pip install -e “.[dev]”

And both attempts yield the following error:

ERROR: File “setup.py” not found. Directory cannot be installed in editable mode: /notebooks/course-v4/nbs
ERROR: File “setup.py” not found. Directory cannot be installed in editable mode: /notebooks/course-v4/nbs

What I tried next was the editable install via terminal, which seems to install just fine. However, when I reopen and run the notebook, I get the same OSError error noted above.

Appreciate you helping this noob out!

6

Hi mrfabulous1!

Could you tell me how you record all your library versions with a working notebook? As a beginner, this would be really useful and would save me from a lot of heartache! :+1:

7

According to @mrfabulous1’s reply above, the OSError seems to be caused by sentencepiece 0.1.90. Try downgrading it back to 0.1.83, which seems to work fine?

Yijin

3 Likes
8

HI rj9 :grinning: Sorry to hear your still experiencing the problem.

The way I save my library versions is I have a cell in my notebook which has the following statement

!pip freeze > notebook_10_lib_versions_20200518.txt

I have to use the ! as I am using Google colab notebooks.
I do this when I start a notebook and when I complete a notebook!
and save the file. So I know if a library has changed, its not totally foolproof
as sometimes a minor change is made to a library without a version number change.

I use a repository created by a user to configure my colab session to run any notebooks. which configures the notebook for use on google colab. Fastai2 and fastcore were configured with the right version.

sentencepiece was configured for 0.1.90

I just added the following line in my notebook after the initial imports.

# This version  0.1.90 of sentencepiece breaks the sentence piece beloW
#!pip install sentencepiece # required to use subword tokenization 20:00 on video

!pip install sentencepiece==0.1.83

Every time I run this notebook I will have to run this cell as like I said earlier I use a repository which configures a lot of other things on colab for me.

One other point google colab always asks me to reboot the kernal before I can use sentencepiece==0.1.83.

like utkb also said just try downgraiding sentence piece. I would of thought you could of left everything else as it is.

Hope this helps mrfabulous1 :grinning: :grinning:

1 Like
9

Thanks @utkb and @mrfabulous1, that seemed to do the trick! Downgraded to sentencepiece==0.1.83 and the cells are now running. Appreciate you getting me over this hurdle.

1 Like
10

Thanks for letting me have a look at how you do this! I’ve been using paperspace since the course started in March but I am considering migrating to google colab soon. Hope you don’t mind me reaching out to you in the future!

For now, I’m just glad I can move forward with the lesson video and notes. :grinning:

1 Like
11

I still have the same problem. I’m running on a Kaggle notebook. And I have installed sentencepiece==0.1.83, as it was suggested.

12

Hi utkb,
How do you check the version of these libraries? I am running the notebook on Paperspace. I only know how to print the fastai version. Thanks!