You should try Total Variation Regularization. The relevant section and paper below:
https://arxiv.org/pdf/1603.08155.pdf
Section 3.3:
Total Variation Regularization. To encourage spatial smoothness in the
output image ˆy, we follow prior work on feature inversion [6,20] and superresolution
[48,49] and make use of total variation regularizer `T V (ˆy).