Annoying for me as well, I will probably switch back to android once they have a similarly small factor phone that has a camera I like, for now there is none.
But this annoyance is less worrisome for me because at the end of the day, even though it makes little sense and will be more difficult you will remember for each app you use, what is the random back gesture that was chosen for each window.
And yes the system is slower in terms of animations, I heard people explaining it as apple wanting you to “feel the weight”, this might be an excuse but it makes me feel better because I do feel the phone is more buttery, slow but buttery smooth than android which is quick and functional