The recoil IS relative. It pushes the gun back towards you more than anything. Shooting a pistol sideways usually pushes it back into your palm. Idk it it's anatomy or muscle memory but it always feels like my arm wants to go up no matter how I hold a pistol.
A rifle or shotgun when held like the meme, generally feels like it's just pushing straight back relative to the barrel, wonder if that's all the effect of a second hand.