I am learning about what exactly a convolution is, and the definition used by wikipedia here (Convolution - Wikipedia) is not the same as the convolution we learn about in deep learning. In my opinion, according to Wikipedia’s definition, the kernels in a deep learning “convolution” should first be flipped to really be convolutions. Right now, “convolutions” in deep learning look more like cross-correlation (Cross-correlation - Wikipedia) to me.

Hello,

You are correct - so-called convolutions in deep learning implement cross-correlation and are not equivalent to the convolution operation mathematicians are familiar with. However, using cross-correlations in a network, as is the norm, or flipping the kernels first to correctly emulate the mathematical convolution would yield identical results because in case of the latter, the model would simply learn a flipped version of the weights of the neural net trained using cross-correlation.

Please let me know if you require further clarifications.

2 Likes