Fastai_v1, adding features

jeremy · November 16, 2018, 11:59pm

You’re looking at the sig of DataBunch, which I believe can take arbitrary Datasets. However create_cnn needs an ImageDataBunch, which needs c defined.

devforfu · November 17, 2018, 4:07am

Yeah, that’s correct. I was thinking that if ImageDataBunch is OK with accepting arbitrary Dataset instances then it works with other library classes all the way down to training loops. It was a bit unexpected to me that it is not the case

Ok, sorry for the misinterpretation.

GilesStrong · November 22, 2018, 3:06pm

Hi,
I’m wondering if the tabular data class could be extended to provide the ability to use sample weights for each entry when computing losses?
For the data I work with (high energy physics) these weights are necessary in order to allow the simulated data we train on to match reality.
Currently I use Keras for my work, which has such a feature, but having followed the DL courses I’m looking to move to using the Fast.AI library.

Having looked on the forums, there only seems to be a few topics on balancing classes via hard-coded weights and a custom loss function, but with sample weights it depends on the batch of data being passed to the loss function, so it’s a bit more tricky.

Cheers

cwerner · November 24, 2018, 8:09pm

Adding an option to auto-save model state after each fit_one_cycle() cycle

Hi,
I am wondering if an option to fit_one_cycle would be useful to autosave the model state after each cycle? I had the problem on several cloud providers that my notebook died when training a domain language model and so I reran the 10-cycle step like 4 times…
Maybe one could specify a flag autosave=True or autosave=“modelname” and the function would dump the state automatically into modelname-01.pth, modelname-02.pth, …

If you think this sounds worthwhile I might take a look at it? Hints where to start (hooks?) welcome…

msandroid · November 25, 2018, 9:51am

Hi Christian,
Have you taken a look at the Save Model Callback? It might be already what you need.
https://docs.fast.ai/callbacks.tracker.html#SaveModelCallback

cwerner · November 25, 2018, 11:23pm

Did not know that one!

And yes, seems like it does just that!

github.com

fastai/fastai_docs/blob/master/dev_nb/experiments/wikitext103-qrnn.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fastai.text import * \n",
    "from fastai import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = Path('data/wikitext-103')"
   ]

This file has been truncated. show original

fabris · November 26, 2018, 4:09pm

Hi, and sorry if this request has been already discussed somewhere else!
I think that fastai is fantastic, but as allenNlp user I find quite convenient to instantiate objects from Jsonnet blobs. In ablation studies such declarative syntax allows to specify an entire experiment using json, moreover it allows to change architectures without changing code. It would be great to have experiment configuration files in fastai too. Thanks!!
For the unfamiliar reader, check this out to have an idea :

github.com

allenai/allennlp/blob/master/tutorials/getting_started/walk_through_allennlp/configuration.md

# Configuring Experiments

Now that we know how to train and evaluate models,
let's take a deeper look at our experiment configuration file,
[tutorials/getting_started/walk_through_allennlp/simple_tagger.json](simple_tagger.json).

The configuration is a [Jsonnet](https://jsonnet.org/) file
that defines all the parameters for our experiment and model. Don't worry if you're not familiar
with Jsonnet, any JSON file is valid Jsonnet; indeed, the configuration file we use in this tutorial
is just JSON.

In this tutorial we'll go through
each section of the configuration file in detail, explaining what all the parameters mean.

## A preliminary: Registrable and from_params

Most AllenNLP classes inherit from the
[`Registrable`](http://docs.allennlp.org/en/latest/api/allennlp.common.html#allennlp.common.registrable.Registrable)
base class,
which gives them a named registry for their subclasses. This means that if we had

This file has been truncated. show original

shgidi · December 31, 2018, 11:50am

I’m using ImageDataBunch.from_df, which randomly splits the data frame to train/test, with no option to pass random seed.

I would like to add a seed=None argument, which will be passed to random_split_by_pct, and allow reproducible split. Do you think it’s necessary?

sgugger · December 31, 2018, 12:00pm

You can do it with the data block API (which you should learn since it’s more flexible than the factory methods).

zhangweida2080 · January 27, 2019, 2:20pm

how to build custom layer, is there any tutorial?

PierreO · January 27, 2019, 6:31pm

I’m currently working on Object Detection and I’m thinking about developing an object similar to fastai’s ClassificationInterpretation to interpret the model.

Would that be a welcome addition to fastai? If so, do you have any particular request with respect to the features?

(originally posted on Developer chat, removed to post here)

sgugger · January 27, 2019, 7:56pm

It would be welcome, yes.

paws · January 30, 2019, 5:34am

Adding the transformer model to fastai V1.
Can we do this? although probably not top priority, They have shown massive improvements in a variety of NLP tasks.

sgugger · January 30, 2019, 2:19pm

That’s actually what I’ve been working on these past few days. Transformer and transformer XL should be in the library before the end of the week.

paws · January 30, 2019, 2:51pm

If you need any help, do let me know. In my personal experience though. Transformers have a lot of small things to be careful about to get them to train

loftiskg · January 31, 2019, 9:36pm

Hello, very minor feature request. Would it be possible to add a boolean parameter to the .from_folder() method in the data blocks API to the recurse through symlinks. If this is something that would be in the spirit of the data blocks API I could submit a pull request.

It looks as simple as carrying a boolean value over to the get_files() call in the .from_folder() method and then placing it in os.walk()

sgugger · February 4, 2019, 2:43pm

A PR on this would be welcome, yes.

alexrigler · February 4, 2019, 6:35pm

Hi @sgugger, the transformer and tranformer-xl look great! I am doing some research on transformers for language modelling including trying out the evolved transformer .

All of these papers used word-piece tokenization. Would PRs be welcome for a SentencePieceTokenizer class?

irustandi · February 6, 2019, 4:07pm

Minor feature request: in DataBunch.create() and any other places where this makes sense, add a parameter (e.g. val_bs=None, if None then just use bs) to be able to specify a different batch size for validation set.

Great work overall!

sgugger · February 6, 2019, 7:37pm

Good idea! Just added it in master.