Any reason why metric would stay constant on tabular models?

Part 2 2022/23
1

Hi all, I’ve just finished Chapter 3 in the course and the associated work. To solidify some of the skills, I’m trying to replicate some of the dataset prep and model building work on the Titanic classification problem mentioned in the videos. I’ve written some code that uses the Tabular classes provided by FastAI, and although I’m seeing my loss function go down, my metric stays constant regardless of my epochs or learning rate. In fact, I can give it a massive learning rate, that increases the loss and the accuracy stays the same. I’ve tried checking other examples on the web, and it seems like others also have gotten the same result. Can somebody help me understand why?

train = pd.read_csv('./titanic/train.csv')
train = train.drop(columns=['PassengerId','Name','Ticket', 'Cabin'])

train['Category_encoded'] = label_encoder.fit_transform(train['Embarked'])
train['Sex_encoded'] = label_encoder.fit_transform(train['Sex'])
train = train.drop(columns=['Embarked', 'Sex'])

procs = [Categorify,FillMissing,Normalize]

dls = TabularDataLoaders.from_df(
    df=train,
    procs=procs,
    cat_names=['Pclass', 'SibSp', 'Parch', 'Embarked_encoded', 'Sex_encoded'],
    cont_names=['Age', 'Fare'],
    y_names='Survived'
)

learner = tabular_learner(dls, metrics=accuracy)
learner.fit(10, .1)

epoch	train_loss	valid_loss	accuracy	time
0	     4.165205	 0.874836	0.628272	00:00
1	     2.066422	 0.519062	0.628272	00:00
2	     1.327172	 0.210924	0.628272	00:00
3	     0.954734	 0.137054	0.628272	00:00
4	     0.728373	 0.173381	0.628272	00:00
5	     0.585556	 0.279527	0.628272	00:00
6	     0.492400	 0.166058	0.628272	00:00
7	     0.429404	 0.150508	0.628272	00:00
8	     0.374902	 0.157832	0.628272	00:00
9	     0.334262	 0.205877	0.628272	00:00
10	     0.306443	 0.152892	0.628272	00:00
2

I needed to specify the y_block in my Dataloader.

dls = TabularDataLoaders.from_df(
    df=train,
    procs=procs,
    cat_names=['Pclass', 'SibSp', 'Parch', 'Category_encoded', 'Sex_encoded'],
    cont_names=['Age', 'Fare'],
    y_names='Survived',
    y_block=CategoryBlock
)
3

Hello,

You are encountering a common issue that can arise with classification tasks, especially in cases where your metric remains constant despite changes in the loss. Given that your accuracy metric is stuck and does not change. Ensure that your data preparation steps (such as dropping columns and encoding) are correctly implemented. Double-check that the target variable Survived is correctly included in your data processing pipeline and not inadvertently dropped or altered.