ULMFiT for regression problem

mizzourah2006 · February 4, 2019, 9:27pm

I had a question about using ULMFiT. Let’s say I am trying to predict a rating for a specific essay response rating between 1-5. I don’t necessarily need the exact class score, because if the argmax was a 2 and the true value was a 3 that’s a lot better than the argmax being a 1 and the true value being a 3, etc. I was wondering if there was a way to make a small change to the architecture to turn a classification model into a regression based model. I’ve seen people scale the the score down to be between 0 and 1 and then use a sigmoid activation function, but I’m still having trouble understanding how this works conceptually. It’s still a number between 0 and 1, it’s not a probability of a class for 0 and 1 which is what the sigmoid provides.

Any help would be much appreciated.

Thanks!

Nick

ste · February 4, 2019, 10:17pm

Try to pass label_cls=float to label_from_list or other labellig step in data block api or simply cast your labels to float or np.float32

See get_label_cls definition and usage in:

github.com

fastai/fastai/blob/6c0959b8d760fbe1459dfcf99d644c9898d8e288/fastai/data_block.py

from .torch_core import *
from .basic_data import *
from .layers import *

__all__ = ['ItemList', 'CategoryList', 'MultiCategoryList', 'MultiCategoryProcessor', 'LabelList', 'ItemLists', 'get_files',
           'PreProcessor', 'LabelLists', 'FloatList', 'CategoryProcessor', 'EmptyLabelList']

def _decode(df):
    return np.array([[df.columns[i] for i,t in enumerate(x) if t==1] for x in df.values], dtype=np.object)

def _maybe_squeeze(arr): return (arr if is1d(arr) else np.squeeze(arr))

def _get_files(parent, p, f, extensions):
    p = Path(p)#.relative_to(parent)
    res = [p/o for o in f if not o.startswith('.')
           and (extensions is None or f'.{o.split(".")[-1].lower()}' in extensions)]
    return res

def get_files(path:PathOrStr, extensions:Collection[str]=None, recurse:bool=False,
              include:Optional[Collection[str]]=None)->FilePathList:

This file has been truncated. show original

mizzourah2006 · February 5, 2019, 1:28am

Thanks! I’ll take a look. I appreciate it!