Thanks, it’s fixed now.
What are your thoughts on adding a new doc page to explain fastai.imports? I’m guessing that familiarizing myself with the imports and abbreviations that the fastai authors find useful will make me more productive, but there is not a quick way to see what purpose each import serves. I could write a doc page with a table that lists each import, a brief description, and a link to the homepage for the import if you’d entertain such a pull request.
1 Like
Thank you @jpizarrom for pointing that out it is very good catch.
Maybe it explains why increasing the dropmult had inverse effects on regularisation, in my experiments as the output dropout was increased more than 2 fold
with bug: input_p=0.2 , output_p=0.25, weight_p=0.15, embed_p=0.02, hidden_p=0.1,
correct : input_p=0.25, output_p=0.1, weight_p=0.2, embed_p=0.02, hidden_p=0.15
0.5 , +1.5 , -0.5 , 0 , -0.05
I wonder how have you noticed the change?
Hello,
I’ve started to read about the model two weeks ago in blogs and also reading code of some use cases in github.
When I was trying to use ULMFIT to finetune with my own dataset, I was always overfitting(kind of) the language model and the text classifier, always getting lower training loss than validation loss. I was trying with drop_mult in the range 0.3 to 0.7 because that was the common values.
After lots of trial and fail, and because I realice the api changed recently, I decided to review the ULMFIT implementation and recents changes.
1 Like
Hi @stas
in order me to use suggested : from fastai.callbacks.mem import PeakMemMetric i had to update fastai library which somehow messed my whole python environment that I have been dealing with it fix it for almost two working days. Anyhow after every thing seems to be alright I got this error when trying to train a model
create_cnnis deprecated and is now namedcnn_learner`.
when switched it to “cnn_learner” and running it that jupyter_notebook cell remains busy forever without actually constructing a model and training it. any solution for this?
thanks for your help 
this is the code I am using :
arch = models.resnet18
aName = '_resNet18_test'
epochS = 10
maxLR=1e-02
data = ImageDataBunch.from_folder(f'{d_path}'+"LC_B_5", ds_tfms=(tfms, []),valid ='test', bs=8)
mName_S = str('bestModel_' +str(5)+'_S_'+ aName)
learnS = cnn_learner(data, arch,pretrained=True,
metrics=[accuracy, error_rate],
callback_fns=[partial(CSVLogger, filename =str('stat_' +str(tr)+'_S_'+ aName))
, ShowGraph,PeakMemMetric,
partial(SaveModelCallback, monitor ='val_loss', mode ='auto',name = mName_S ),
partial(ReduceLROnPlateauCallback, monitor='val_loss', min_delta=0.01, patience=3),
partial(EarlyStoppingCallback, monitor='val_loss', min_delta=0.01, patience=5) ])
learnS.fit_one_cycle(epochS, max_lr=maxLR, moms =[0.95, 0.85], div_factor = 25.0)
Perhaps some of the callbacks you use weren’t correctly ported to the new API, try to disable them one a time and see if you find the culprit, then post which one gives you the trouble.
I thought of it too and removed all the callbacks but still the same.
If there are no callbacks, it shouldn’t be a problem, cnn_create was just renamed to cnn_learner. Perhaps something is messed up in your python setup as you were saying?
Try a fresh conda env, with nothing in it and just conda install -c pytorch -c fastai fastai (and jupyter)?
Thanks @stas. I will try your suggestion and will let you know you how did it go.
but last time unless i would use :
pip install git+https://github.com/fastai/fastai.git
i would get module not fount error for :
from fastai.callbacks.mem import PeakMemMetric
You probably had a very old fastai, when you install the last released fastai 1.0.48, it’s there: https://github.com/fastai/fastai/blob/1.0.48/fastai/callbacks/mem.py
sadly creating new conda environment also didn’t work. perhaps I have to have a software engineer take a look at my system as it seems that one thing get fix and others go wrong despite the fact that I have removed all python and conda environment and dependencies and re-installed them again yesterday to have everything up to date.
re fastai version, i just installed the latest version yesterday which I suppose it is the latest version.
Thanks for the help anyway. I will be in touch if I did any progress in it.
sadly creating new conda environment also didn’t work.
It’s not possible for us to help unless you say specifically what doesn’t work
Are you saying that cnn_learner is still stuck? What’s your full environment, see: https://docs.fast.ai/support.html?
and it looks like there is a bug in pytorch on windows, which now has a workaround in fastai-1.0.49 - just released. So update and try again please.
I’ve been looking closely at parallel (fastai.core). It takes a function (func) and a collection (arr) and calls func on each element of arr (in parallel).
Does anyone here know why forces the function you provide func to accept both the value and index of each element in arr? This means you have to write a new function that is a copy of your old one, but accepts an additional input index that it never uses. In the source code it calls
ProcessPoolExecutor(max_workers=max_workers).submit()
which I looked up here: Python 3 Library and it doesn’t seem to use the index argument.
Could parallel possibly be reworked to drop the index argument?
1 Like
No, we need the index argument for its use in verify_images. Adding it your function and ignoring it shouldn’t be too painful.
1 Like
Thank you, that’s what I’ve been doing but wasn’t sure if a better way was possible. I’ve also noticed it fails more or less silently when you pass in a function that hasn’t been altered to accept an index. The progress bar completes to 100% in 0:00 but the function doesn’t get run. The full documentation makes it really clear, but the in notebook documentation doesn’t mention the existence of an index. I can add these as issues, but I’m really new and wanted to make sure this wasn’t expected behavior.
This next question is a bit off topic and I know you’re incredibly busy, so no worries if you don’t have time to reply. Any advice on best practice for using parallel when you have a function that takes more than one argument? My approach has been if the other arguments are static, to use a partial, but if the other arguments are variable, to change the function to accept one argument that is actually a tuple of arguments (and the index of course), and then pack and pass in a collection of tuples. I’ve written a post on the nuances of parallel to teach others to use it and I’d really like to get it right. Thank you.
That seems the right way to do it, yes. Also note that this is just a convenience function for our internal use, which just takes a few lines of code, so you can rewrite your own version whenever it’s needed 
1 Like
have you though about using lambda , adopting parameters to match the intereface seems to be quite good usecase for them.
parallel(lambda v,_: your_func(v), data, max_workers=3)
1 Like
You can also use a dict containing your function arguments (similar to **kwargs)
1 Like