Yep, having the same seed for both was crucial since otherwise, as you mentioned, the noise varied. Even then, if you took the final frames, there was way too much variance between the two images generated to get a proper mask. I spent most of today just trying out things with the final frames to see how much of a difference there was and if you subtract the final frames of horse and zebra, there’s too much noise to be able to get a good mask at all … well, at least for me 
1 Like
I was seeing some mask results that didn’t seem great when averaging my noise channels together so I decided to dig in a little bit. Here is what my 4 noise masks looked like:
Two that have the horse area as bright and two that have the horse area as dark. What I really want here is to combine these so I decided to try to subtract 0.5 and then take the absolute value from my noise channels then add the 0.5 back so that I make sure I am seeing everything going the same way. It seemed to be helpful.
Here is what they look like after the absolute value piece:
so all of the higher pixel values are now around the horse/zebra which is helpful when averaging the 4 channels together.
Here is a comparison of the two once they are averaged together:
I’d also love to join
1 Like
Welcome! I’ll start a new thread today after the lesson if we get a new paper to work on
2 Likes
Did you finish implementing diffedit? I saw the updated notebook but wasn’t sure
Sorry, wasn’t sure who you were asking, or if you were asking generally I believe lots of people finished their own implementations and quite a few are linked in the [ Lesson 11 official topic](Lesson 11 official topic - #152) thread …
Personally, I didn’t finish it as a completed solution (just did bits and pieces) since I got sidetracked by the masking and figuring out various aspects of using OpenCV and then working on a separate solution which did not involve masking at all.
1 Like
Here is my finished implementation with a little bit of code refactoring from my original: diffedit implementation (github.com). Still planning on making it a little more polished and pushing it out as a blog post, but won’t get to that until tomorrow.
3 Likes
Yea I like this idea of centralizing the work for a specific project/paper to one area. Hehe, we had conversations going on here, show your work, and lesson 11 . Imagine also if after a few days of working individually we had just jumped on a zoom call for an hour to discuss. That would be cool but could be tricky with time zones etc. Anyway, I’m interested in the ongoing discussions
2 Likes
I wonder if it would be helpful to have sort of a retrospective here after we’re done with our individual efforts. Kind of talk about what we learnt, what surprised us, what we found interesting etc.
For my part, I was really interested to implement DiffEdit but what frustrated me the most was getting the the mask to work But in the process, I did learn a lot about the options that OpenCV offers (the Python tutorials here are very, very helpful) and spent like a couple of days trying out various masking techniques …
I think where I went off the road (and diverged significantly from the spirit of DiffEdit) was in using inpainting. The inpainting results can be a little unpredictable and doesn’t always do a perfect replacement on the original image — at least in my results. That’s when I kind of lost steam and simply started working again on my own implementation of PromptEdit since that gave me way better replacement results at the cost of not always keeping the background intact. For my purposes, that worked better than my results from DiffEdit …
I feel as if I link to PromptEdit way too much and so am hesitant to link to it yet again, but if anybody wants to take a look, it’s there in my repo along with other notebooks with various experiments from this course
3 Likes
The automatic mask generation is what got me all excited for this paper. But i still can’t generate the mask robustly across different images for it to work satisfactorily.
2 Likes
And that’s what kind of leached away my initial excitement — the fact that the mask is finicky. If we can’t get the mask to work consistently using a single technique and have to mess around with each image to modify the mask accordingly, the approach isn’t as useful, at least for me.
But maybe there’s a solution and I’m missing it. I wish the folks who did the DiffEdit paper published their code so that we could know for certain if we are on the right track or not, but I guess that’s too much to ask for?
1 Like
Yeah being robust with a fixed set of prams(which the paper actually claims to have) is what I’m looking for too. I’m converting mine to a blog so will post it soon. May be you should rename this topic to Paper discussion: DiffEdit. So we don’t mix chats about different papers
1 Like
Good idea, will do!
I found that increasing the noise strength and the noise samples helps in making the mask more robust. I have posted in the lecture 11 few examples.
This week I’ll try to run my idea against a set of different images. (Maybe 20ish)
1 Like
This is very cool, thanks for sharing 
Looking forward to reading it — I’m always interested to learn how others approached a certain problem and what their takeaway from it all was
Nice, I am catching up on the blogs and slowly build up to DiffEdit Aayush Agrawal - Introduction to Stable diffusion using 
Hugging Face (aayushmnit.com) Here is is first one in the series. I intend to do a lot of catching up / writing on weekend.
1 Like
Ah, I remember the heady days of first getting started on Stable Diffusion I wrote a bunch of stuff back then but now just don’t have the time to write about it since I’m too busy working with it … Bookmarking your post to read tomorrow but it’s bedtime for me now.
I decided to try out the model mentioned here: Lesson 11 official topic - #63
From my very initial test, it does seem to improve the results a little bit, but not significantly at least in my horse to zebra case. This is probably the best one:
From:
To:
2 Likes