Is it possible to do multiple image classification (and sub-classification) in 1 Model?

Hey Fast.AI members.

I finished Jeremy’s season 1 and 2 classes… holy awesomeness. Made everythign so clear

Here’s my dilemma - I’m trying to detect a person, face / ethnicity (if it can be seen in camera) and clothing color of people walking by. With release of tensorflow object detection API, i’m even more confused.

Here’s my pseudo code

while True:

	img = capture_video_frame()

	# detect object using Tensorflow object detection API 

	object = detect_object(img) 

		face = detect_face() # using facenet

		if(face is not None):
			ethnicity = classify_ethnicity() # my own classifier
			ethnicity = None

		# use  Kmeans to determine the "dominant" clothing colors 

		clothing_colors = detect_dominant_color(img) 

		record(face, ethnicity, clothing_colors)

Now, several issues

  1. I am using 3 models to do something that I think that should be done with 1. Detecting person object, face, ethnicity. I have a feeling this is the wrong way to do it. For one, performance would be a major issue. Is there a way to “merge” 3 models into one? Any tutorials you can point me to?

  2. is using TF object detection API unnecessary here? my thought process for wanting to use it was that I’d want to classify other things down the road… like hat, glasses, etc.

  3. KMeans for doing dominant color analysis seems like overkill, not to mention it’s insanely slow. Do you guys have any suggestions? Any suggestions on how I might be able to separate the torso from the body? I am looking for clothing color

Issue 1
Yes it is possible predict several classes. You can re-use first layers, and have different last layers for different classes. See Keras sample in my SO answer: