Some things i can think of (for Wayland)
-
plymouth will either display in a very low res or not at all on nvidia systems
-
no smooth dm-de transitions
-
occasional graphical errors (although these barely exist on my hardware anymore)
-
taints kernel by loading an out-of-tree module, which breaks some stuff
-
for foss purists, oss drivers aren't as fast as proprietary by a large margin (although nvk looks promising)