@resdntalien I tried minimizing f_original using the original image as a starting point instead of noise (results below), it diverges a bit to loss=0.128 (why?) then goes back to much lower loss values. All of these iterations are almost visually indistinguishable from the original though.
Clearly, bfgs does not find anywhere close to a global optimum if it starts from noise… where it starts from determines where it ends up.
This got me to wondering: what happens if we start from something in between the original and pure noise? Turns out we end up in a different local minimum, much closer to the original, but still not all that close (loss=0.373). Also it takes a lot more iterations before it gets stuck in that local minimum.
Food for thought:
- How to go to loss=0 starting from pure noise or from any other initialization?
- Can a better initialization be constructed without knowing anything about the original image?
- Are there other images (different from the original) which have loss=0? What is the most pixel-wise-different image like that?
- What does the family / manifold of those images look like?