LanguageModelLoader

(Yesu Feng) #1

Hi,

I find both the text.py and nlp.py implement the LanguageModelLoader, but with different constructors, which is confusing, I guess in the imdb notebook, we are using text.py, just curious, what is the other implementation in the nlp.py for?

1 Like

(Jeremy Howard (Admin)) #2

fastai.nlp is the deprecated approach that relies on torchtext. fastai.text is a new approach that tries to be similar to the previous API where possible.

2 Likes

(Apil Tamang) #3

I was about to ask the same question. I think we should annotate the methods/classes in the file with a @deprecated annotation, explaining this situation.

@jeremy Have you thought of a versioning scheme/schedule for fastai that would allow me to submit a PR with the right information ?

thanks …

0 Likes

(Jeremy Howard (Admin)) #4

We can’t do that yet since fastai.nlp is what we’re still using for part 1, and part 2 isn’t even out yet!

My view is that we shouldn’t deprecate anything where there’s a current course that uses it - that would be awfully confusing to new students! Instead, we can just add docs to each module explaining that one is for use with torchtext and one is standalone - and that folks doing part 1 should stick with fastai.nlp.

1 Like

(Apil Tamang) #5

certainly, thanks!

0 Likes

(Yesu Feng) #6

having docs is certainly helpful, can we start to do it, @jeremy you mentioned some centralized docs in class, but I think it is still helpful to have doc in the method, such that e.g., shift + tab in jupyter notebook can easily view the docs. Is there some way to allow the community to contribute and you can review the docs?

0 Likes

(Arvind Nagaraj) #7

The imdb markdown notes I just added in the latest PR clearly calls this out.

1 Like

(Jeremy Howard (Admin)) #8

Yes we’ll have a 1-line docstring, just like the python std library.

Absolutely. There’s already stuff happening in the doc/ directory! :slight_smile:

1 Like