How to adjust the input and output when finetuning ULMFit?

fastai
1

Hi,
I was looking for tutorials on ULMFit(Universal Language Model Fine-tuning) and I’m new to fast-ai. .
I know that if you want to finetune a model, you need to adjust the input and output.
for example:
1- you need to change the size of the input and output based on the number tokens in the vocabulary
2- you need to change the vocabulary(add the tokens that exist only in the target data, remove the tokens that exist in the source data only, keep the tokens that exist in both source and target data)
In the tutorials that I saw, they did this

learn = language_model_learner(data_lm,AWD_LSTM, pretrained = True, drop_mult=0.3)
learn.fit_one_cycle(4,1e-2,moms=(0.8,0.7))

This code doesn’t adjust the input and output.
Where are the steps that adjust the input and output when finetuning?
also, what does pretrained parameter do? some tutorials use it and some don’t

2

Have you checked out the NLP lesson in the most recent course? Fast AI Video Viewer

3

thanks, I just did that. but I didn’t find what I’m looking for.
I saw this

Our vocab will consist of a mix of common words that are already in the vocabulary of our pretrained model and new words specific to our corpus (cinematographic terms or actors names, for instance). Our embedding matrix will be built accordingly: for words that are in the vocabulary of our pretrained model, we will take the corresponding row in the embedding matrix of the pretrained model; but for new words we won’t have anything, so we will just initialize the corresponding row with a random vector.

I don’t know which part of the code does that?
Is it this?

learn = language_model_learner(data_lm,AWD_LSTM, pretrained = True, drop_mult=0.3)