I would love to be proven wrong though.Įdit: To make my above statement more concrete, these are non-cherry picked outputs directly from Clipdrop's Uncrop. If I were to throw in my own two cents, I do think that ultimately no current publicly available models can fully handle illustrated artwork very well (not even Waifu Diffusion), so I don't think it's a fault of the system but the base models/data. Using the image w/o modifications seems like the obvious way the reference preprocessor should use the image but I'm guessing that's a limitation of the method (I'm not 100% familiar with how it functions, so if that actually could work, that should probably be fixed). Just Resize and Resize and Fill don't make much sense to use either since the former introduces stretching, and the latter introduces borders, which affect the generated image to then take those 'artifacts' into account. That same crop behavior occurrs for the anime illustration example as well though, so I don't think it necessarily explains how it helps in one case but not another. I'm guessing this has something to do with the reference image becoming expanded and cropped to a small, wide area in the center (due to the Crop and Resize resize mode). In this case, it actually makes the results worse.

OP example, with reference_adain+attn 0.5 strength, 0.5 style fidelity I verified I could reproduce the original results beforehand, so the only difference in this example below is the addition of the reference preprocessor. I also tried using the same settings for the example in the OP but with the addition of the reference preprocessor, just to see if it helps at all. Inpaint_only+lama AND reference_adain+attn 0.5 strength, 0.5 style fidelity (much better results, imo) Inpaint_only AND reference_adain+attn 0.5 strength, 0.5 style fidelity Lower res version below just to preview here. It's easier to see if you a/b the examples below in a new tab.Īll examples directly below use the same parameters above, with the exception of the input images, and these settings for the model: In my short testing so far it's moreso a problem for illustration/anime artwork (in contrast, your example of a photo works perfectly fine). When outpainting using only inpaint_only+lama it seems more inclined to produce more blobby/blurry shapes to fill in the remaining picture, rather than trying to make something more believeable. You need at least 1.1.222 to use this feature.īeta Was this translation helpful? Give feedback.Īs with the last update, I still seem to be getting nicer results when using this in combination with the reference preprocessor. You can see that inpaint_only+lama is a bit 'cleaner' than inpaint_only (and also a little bit more robust). (The Resize Mode 'Resize and Fill' enables outpaint) Input image (this image is from Stability's post about Clipdrop) This makes inpaint_only+lama suitable for image outpainting or object removal. The results from inpaint_only+lama usually looks similar to inpaint_only but a bit “cleaner”: less complicated, more consistent, and fewer random objects. When you use the new inpaint_only+lama preprocessor, your image will be first processed with the model LAMA, and then the lama image will be encoded by your vae and blended to the initial noise of Stable Diffusion to guide the generating. The basic idea of 'inpaint_only+lama' is inspired by Automaic1111’s upscaler design: use some other neural networks (like super resolution GANs) to process images and then use Stable Diffusion to refine and generate the final image. Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, Victor Lempitsky LaMa: Resolution-robust Large Mask Inpainting with Fourier Convolutions (Apache-2.0 license) 1.1.222 added a new inpaint preprocessor: inpaint_only+lama

Inpaint for ios instal