One that I use and have tested, is to uninstall HF diffusers and install a custom version of diffusers by using
pip install --user git+https://github.com/huggingface/diffusers.git@e895952816dc4a9a04d1d82fb5f8bdee59341c06
One more method which I have tested is to disable the code for safety check in installed packages manually show by @cjmills. More details here