Lesson 3-CamVid super bad accuracy

daisy · March 21, 2019, 1:37pm

Before I tried to do semantic segmentation for satellite images, I run the official CamVid notebook out for understanding how things work out.

However, I found that after I change the batch size to 4 (the only change I made to the official notebook because of “out of memory” ), I found that the accuracy of camvid gone really bad.

I tried to change the learning rate to 5e-3 because I noticed that the training error is bigger than the validation error. but things didn’t work out and the prediction is really bad as well.

I am not sure if this is because of different batch size since I don’t have enough GPU memory(Tesla P4 7611MB)

So I would really love to know what is going on and I will try different learning rate and more epochs by myself as well.

here is the exact notebook I was running

github.com

shiyutang/Lesson3/blob/master/lesson3-camvid.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Image segmentation with CamVid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline"
   ]
  },

This file has been truncated. show original

gevezex · March 21, 2019, 2:21pm

Did you do a git pull to get the latest versions of the notebooks?
As you have some lines missing from the original. Especially the part where you can see how much memory leftovers you have.

daisy · March 22, 2019, 1:40am

Thanks for your advice which really didn’t come to my mind.
I think it is a wise move because somehow “git pull” updates me with some new docs and code which shows I didn’t have the latest version.

(now I updated)

However, when I look at the updated code, it doesn’t have much difference and I trained the learner exactly as the notebook, the result still not improving

gevezex · March 22, 2019, 6:30am

Probably because your files are newer (with your own edits) then the github files. You need to:

git fetch --all
git reset --hard origin/master

But keep in mind that all your changes in the dir will be overwritten. So if you have important changes, back them up first.

daisy · March 29, 2019, 1:11am

Thank you. I will try it later.
And could you please send your version of this jupyter note book such that I can try out?

daisy · April 22, 2019, 10:46am

I tried

git checkoout .
git pull

update fastai using:

sudo /opt/anaconda3/bin/conda install -c fastai fastai

and then increase lr = 1e-2,decrease wd = 1e-4.
then trainning for 12 epoches and solve the problem.