CNN Densenet Model always predicting the first category

Part 1 (2019)
1

Hello folks,

I am trying to build a model to get buy and sell prediction on time series. I’ve generated my data and put it in separated folders:

  • train
    • buy
    • sell
  • valid
    • buy
    • sell

To parse the data to use with fastai i’ve written a custom collate function to generate the databunch. Here is the custom function:

def collate(samples:BatchSamples):
  input_tensor = []
  labels = torch.zeros(len(samples),1,dtype=torch.long)
  for i,s in enumerate(samples):
     matrix = pd.read_pickle(s[0]).values
     matrix = np.stack((matrix,)*3, axis=0).astype('float32')
     input_tensor.append(matrix)
     labels[i] = 0 if (str(s[1]) == "buy") else 1
 return tensor(input_tensor), tensor(labels)

My call to construct the data bunch object:

data = (ItemList.from_folder(matrix_path)
             .split_by_folder()
             .label_from_folder()
             .databunch(bs=128, collate_fn=collate))

The problem is when I call fit_one_cycle all predictions are 0(buy). Here is the confusion matrix:

Screenshot%20from%202019-06-24%2008-17-24
Screenshot from 2019-06-24 08-17-24.png638×594 21.3 KB

Can anyone shed some light where the error is?

2

Oof, that’s fairly abstract, and we’re only seeing parts of your code, so making a solid prediction on where the mistake could be is a shot in the dark. It looks like your data set is balanced between the two classes, so I suppose the issue is not there. My go-to technique for when things are completely broken is to step through the fastai pipeline. Here’s what I would propose:

  • Can you grab a single batch of your training set?
  • Can you grab a single item from that batch and inspect it? Does it look the way it should (dimensions, data it contains etc). In other words, can you take a look at an individual x and y, and see if they look the way they should? These steps check whether your items “survived” the data block API correctly.
  • If that’s fine, can you put your x-batch through your model (simply do out = learn.model(batch), and inspect the out variable? Does everything there look the way it should (correct shape, contents make sense etc).
  • Lastly, take a look at your loss function. Could it be that for some reason, it only rewards “buy” predictions?
2 Likes
3

Thank you very much for the help! I will try the steps suggested.