But doesn't that defeat the purpose of upscaling? I see a lot of people promoting Magnific AI (which costs $40 a month) but it's not really an upscaler. It's more of an AI repainter. It replaces most details by re-imagining them instead of guessing the original details. I think we should be clear about what an AI Upscaler is and what an AI re-painter is.
ESRGAN is the same thing. You cannot recover signal that are already lost without additional information (in Diffusion case, model's prior). Upscaler are hallucinators by that definition.
But results differ in perception. ESRGAN doesn't hallucinate whole [sub]objects and that puts it into a different category. Latent/denoising i2i re-scale is different because its parameter range doesn't really intersect with what ESRGAN does.
What @cchance said. Also, Diffusion models have much more capacity than ESRGAN. Unable to hallucinate whole subjects is a limitation of model capacity and training data, not the reverse (i.e. a high capacity model can mimic what you want in a low capacity model if you steer it the right way). An unsuspicious drop of water can have a whole world inside it under a microscope.
It can only hallucinate if your i2i noise level is high enough, if you're adding enough noise that the latent is basically filling in giant swaths of noise areas then of course its going to make up new stuff, thats not hallucinations as much as you asking the AI to draw that entire area as something new.
