Color sensitive image classification

I am trying to implement image classification for fish fillets (as a computerized quality control system)

the system has no issue in differentiating between a tuna and salmon
but its struggling to differentiate between different grades of fish and its freshness, for example:
red salmon, reddish orange salmon and orange salmon
they for all intent and purposes has practically the same shape, features but only different in color

can anyone suggest what i could probably do to improve?
should i use transformations on the training data? or should i instead transform the images during inference? what sort of transformation would help in this problem?

on side note, the training so far i use are images but for inference i use opencv webcam video

thank you

Do different colour representations show the difference more fully? I would look across RGB, LAB, YCrCb to see if there are better combinations of channels to feed in to the network than rgb. Maybe some white balance investigation. If you are doing light/colour augmentation, visibly make sure the augmentation isn’t actually changing the apparent class of the image.

Interesting… let’s try different color space to see…

Also, would you suggest augmenting both the training data and the test data ?

Or only one of them?

is well to see thanks dude! راه اندازی کافی شاپ

You will just have to try and see what works best. It’s not a lot of effort.
Re colorspace, you don’t have to use a full set. I was working on a task where I simply replaced the G in RGB with the Y of YCrCb and this worked best. It was a task like yours with very similar images separated by colour. Intuitively might not make a lot of sense, but experimentation at all stages of a pipeline should be part of ones process.

Thank you so much for the help sir

I will try the suggested methods and play around… I’ll also see if increasing contrast would help

Thank you

Anthell, I am running my own experiment right now and I wrote a function go through and try all combinations between RGB and YCrCb. If you’d be interested in that let me know.

Hi sir, I would be glad to receive your help

Hey anthell! Absolutely. Here is a function I wrote that can make a new image based off of a color schema you define. Was slightly more complicated than I had hoped but it does the trick for what we’re wanting. You could very easily just do a predetermined channel if you want, the key is the.split() and the .merge() :

import re
from PIL import Image

def customColor(img:PIL.Image, layers:str='RGB'):
  chnl = re.findall('[A-Z][^A-Z]*', layers)
  im_rgb = img.convert('RGB')
  im_ycbcr = img.convert('YCbCr')
  
  R, G, B = im_rgb.split()
  Y, Cr, Cb = im_ycrcb.split()
  lrs = []
  for x in range(len(chnl)):
    if chnl[x] == 'R':
      lrs.append(R)
    elif chnl[x] == 'G':
      lrs.append(G)
    elif chnl[x] == 'B':
      lrs.append(B)
    elif chnl[x] == 'Y':
      lrs.append(Y)
    elif chnl[x] == 'Cr':
      lrs.append(Cr)
    elif chnl[x] == 'Cb':
      lrs.append(Cb)
  return Image.merge('RGB', lrs)

I am so greatful for your help, if I’ll share the result of the code you shared, I hope to be able to contribute to the community too for those facing similar kssues