Fastai_v1, adding features

(Jeremy Howard (Admin)) #83

You’re looking at the sig of DataBunch, which I believe can take arbitrary Datasets. However create_cnn needs an ImageDataBunch, which needs c defined.

(Ilia) #84

Yeah, that’s correct. I was thinking that if ImageDataBunch is OK with accepting arbitrary Dataset instances then it works with other library classes all the way down to training loops. It was a bit unexpected to me that it is not the case :smile:

Ok, sorry for the misinterpretation.

(Giles Strong) #85

I’m wondering if the tabular data class could be extended to provide the ability to use sample weights for each entry when computing losses?
For the data I work with (high energy physics) these weights are necessary in order to allow the simulated data we train on to match reality.
Currently I use Keras for my work, which has such a feature, but having followed the DL courses I’m looking to move to using the Fast.AI library.

Having looked on the forums, there only seems to be a few topics on balancing classes via hard-coded weights and a custom loss function, but with sample weights it depends on the batch of data being passed to the loss function, so it’s a bit more tricky.


(Christian Werner) #86

Adding an option to auto-save model state after each fit_one_cycle() cycle

I am wondering if an option to fit_one_cycle would be useful to autosave the model state after each cycle? I had the problem on several cloud providers that my notebook died when training a domain language model and so I reran the 10-cycle step like 4 times…
Maybe one could specify a flag autosave=True or autosave=“modelname” and the function would dump the state automatically into modelname-01.pth, modelname-02.pth, …

If you think this sounds worthwhile I might take a look at it? Hints where to start (hooks?) welcome…

(Michael Schuldes) #87

Hi Christian,
Have you taken a look at the Save Model Callback? It might be already what you need.

(Christian Werner) #88

Did not know that one!

And yes, seems like it does just that!

(Fabrizio) #89

Hi, and sorry if this request has been already discussed somewhere else!
I think that fastai is fantastic, but as allenNlp user I find quite convenient to instantiate objects from Jsonnet blobs. In ablation studies such declarative syntax allows to specify an entire experiment using json, moreover it allows to change architectures without changing code. It would be great to have experiment configuration files in fastai too. Thanks!!
For the unfamiliar reader, check this out to have an idea :

(Gidi Shperber) #90

I’m using ImageDataBunch.from_df, which randomly splits the data frame to train/test, with no option to pass random seed.

I would like to add a seed=None argument, which will be passed to random_split_by_pct, and allow reproducible split. Do you think it’s necessary?


You can do it with the data block API (which you should learn since it’s more flexible than the factory methods).

(zw) #92

how to build custom layer, is there any tutorial?

(Pierre Ouannes) #93

I’m currently working on Object Detection and I’m thinking about developing an object similar to fastai’s ClassificationInterpretation to interpret the model.

Would that be a welcome addition to fastai? If so, do you have any particular request with respect to the features?

(originally posted on Developer chat, removed to post here)


It would be welcome, yes.

(Pawan Sasanka) #95

Adding the transformer model to fastai V1.
Can we do this? although probably not top priority, They have shown massive improvements in a variety of NLP tasks.


That’s actually what I’ve been working on these past few days. Transformer and transformer XL should be in the library before the end of the week.

(Pawan Sasanka) #97

If you need any help, do let me know. In my personal experience though. Transformers have a lot of small things to be careful about to get them to train

(Kevin L.) #98

Hello, very minor feature request. Would it be possible to add a boolean parameter to the .from_folder() method in the data blocks API to the recurse through symlinks. If this is something that would be in the spirit of the data blocks API I could submit a pull request.

It looks as simple as carrying a boolean value over to the get_files() call in the .from_folder() method and then placing it in os.walk()


A PR on this would be welcome, yes.

(Alex Rigler) #100

Hi @sgugger, the transformer and tranformer-xl look great! I am doing some research on transformers for language modelling including trying out the evolved transformer .

All of these papers used word-piece tokenization. Would PRs be welcome for a SentencePieceTokenizer class?


Minor feature request: in DataBunch.create() and any other places where this makes sense, add a parameter (e.g. val_bs=None, if None then just use bs) to be able to specify a different batch size for validation set.

Great work overall!


Good idea! Just added it in master.