This project helps you do prompt-based inpainting without having to paint the mask – using Stable Diffusion and Clipseg
It’s currently a notebook based project but we can convert it into a Gradio Web UI.
It takes 3 mandatory inputs.
- Input Image URL
- Prompt of the part in the input image that you want to replace
- Output Prompt
There are certain parameters that you can tune
- Mask Precision
- Stable Diffusion Generation Strength
Step-by-Step Tutorial – https://www.youtube.com/watch?v=bv1Wur3DvZ0
Inital Req:
- If you are using Stable Diffusion from Hugging Face
? for the first time, You need to accept ToS on the Model Page and Get your Token from your user profile
Requires:
- GPU (runs fine on Google Colab Tesla T4)
- Clipseg Model Weights – https://github.com/timojl/clipseg
- Stable Diffusion In-painting – https://www.youtube.com/watch?v=N913hReVxMM
