Control/Sham architecture for compare with CNN. Make sense?


I am doing binary image classification. As a matter of comparing models in order to see if the CNN is actually learning, I am wondering if building a “Sham CNN” would be useful. I was thinking on using, as architecture a linear regression (y=x*a+b) and see how it works but I am not sure is this is the proper control.

Also, I am not sure on the rationale behind the use of a “sham CNN” but I am kind of skeptical on the use of a random guess (0.5) as a baseline. Any thoughts on this?