Determining fabric on Sofa

Hi,I was going through videos that explained how transfer learning works. It briefly described how the architecture of the networks can help us in separating content and style. So, I am currently trying to tackle a problem of extracting texture on an object from a image. For eg: If I have an image of a house which has a sofa in it. Now I want to compare the color/texture of the sofa with a bunch of texture files I have and determine which is the closest to it.
This is an example image from which I would want to determine the cushion fabric the matches closest to the one in my repository

Bunch of fabrics that I have. Right now its a single image but I do have them in separate images as well.

Any pointers on how to approach this problem would be great!