Stable Diffusion loves VRAM. The larger and more complex the images you're trying to produce, the more it'll eat.

My line of thinking is that if you have a slower GPU it'll generate slower, sure, but if you run out of VRAM it'll straight up fail and shout at you.

I'm not an expert in this field though, so grain of salt, YMMV, all that.