I bet no one truly understands the mathematical machineries, Renormalization is hell of a drug.

For the first question, I would recommend reading the philosopher and physicist Francois-Igor Pris who not only seems to understand the deep philosophical origins of the problem, but also provides probably the simplest solution to it. Pris points out that we cannot treat the philosophical ramification in isolation, as if the difficulty in understanding quantum physics originates from quantum physics itself. It must originate from a framework in which we are trying to apply to quantum physics that just breaks down, and therefore it must originate from preconceived philosophical notions people have before even learning of quantum physics.

In other words, you have to go back to the drawing board, question very foundational philosophical notions. He believes that it originates from the belief in metaphysical realism in the traditional sense, which is the idea that there is an objective reality but it is purely metaphysical, i.e. entirely invisible because what we perceive is merely an illusion created by the conscious mind, but somehow it is given rise to by equivalent objects that are impossible to see. For example, if you have a concept of a rock in your mind, that concept "reflects" a rock that is impossible to see, what Kant had called the thing-in-itself. How can a reality that is impossible to observe ever "give rise to" what we observe? This is basically the mind-body problem.

Most academics refuse to put forward a coherent answer to this, and in a Newtonian framework it can be ignored. This problem resurfaces in quantum physics, because you have the same kind of problem yet again. What is a measurement if not an observation, and what is an observation if not an experience? You have a whole world of invisible waves floating around in Hilbert space that suddenly transform themselves into something we can observe (i.e. experience) the moment we attempt to look at them, i.e. they transform themselves suddenly into observable particles in spacetime the moment we look.

His point is ultimately that, because people push off coming up with a philosophical solution to the mind-body problem, when it resurfaces as the measurement problem, people have no idea how to even approach it. However, he also points out that any approach you do take ultimately parallels whatever solution you would take to the mind-body problem.

For example, eliminative materialists say the visible world does not actually exist but only the nonvisible world and that our belief we can experience things is an illusion. This parallels the Many Worlds Interpretation which gets rid of physical particles and thus gets rid of all observables and only has waves evolving in Hilbert space without observables. Idealists argue in favor of getting rid of invisible reality and just speak of the mind, which if you read the philosophical literature you will indeed find a lot of academics who are idealists who try to justify it with quantum mechanics.

Both of these positions are, in my view, problematic, and I like Pris' his own solution based on Jocelyn Benoist's philosophy of contextual realism which is in turn based off of Ludwig Wittgenstein's writings. Benoist has written extensively against all the arguments claiming that reality is invisible and has instead argued that what we experience is objective reality as it is exists independent of the observer but dependent upon the context of the observation. Thus he is critical of pretty much all of modern philosophers who overwhelmingly adhere either to metaphysical realism or to idealism. There is no mind-body problem under this framework because reality was never invisible to begin with, so there is no "explanatory gap."

Apply this thinking to quantum mechanics then it also provides a solution to the measurement problem that is probably the simplest and most intuitive and is very similar to Carlo Rovelli's interpretation. Reality depends upon context all the way down, meaning that the properties of systems must be context variant. And that's really the end of the story, no spooky action at a distance, no multiverse, no particles in two places at once, no language of observer-dependence, etc.

Whenever you describe physical reality, you have to pick a coordinate system as reality depends upon context and is not "absolute," or as Rovelli would say, reality depends upon the relations of a system to every other system. Hence, if you want to describe a system, you have to pick a coordinate system under which it will be "observed," kind of like a reference frame, but the object you choose as the basis of the coordinate system has to actually interact with the other object. The wave function then is just a way for accounting for the system's context as it incorporates the relations between the system being used as the basis of the reference frame and the object that it will interact with.

Basically, it is not much different from Copenhagen, except "observer-dependence" is replaced by "context-dependence" as the properties of systems are context variant and any physical system, even a rock, can be used as the basis of the coordinate system. But, of course, if you want to predict what you will observe, then you always implicitly use your own context as the basis of the coordinate system. This is a realist stance, but not a metaphysical realist stance, because the states of particles are not absolute, there is no thing-in-itself, and the reality is precisely what you perceive and not some waves in Hilbert space beyond it (these are instead treated as tools for predicting what the value will be when you measure it, and not itself an entity). Although, it is only whether or not they have a property at all that is context variant.

If two observers have interacted with the same particle, they will agree as to its state, as you do not get disagreements of the actual values of those particles, only whether or not they have a state at all. They would not be verbal disagreements either, because if an observer measures the state of a particle then goes and tells it to someone else, then it also indirectly enters their context as they would become correlated with that particle through their friend. You only get disagreements if there is no contact. For example, Wigner's friend paradox, where his friend has measured the particle but has not told him the results nor has he measured it himself, from his context it would indeed have no state.

The "collapse" would then not be a collapse of a physical "wave" but, again, reality is context variant, and so if you interact with a system, then it changes your relation to it, so you have to update the wave function to account for a change in context, kind of like if you change your reference frame in Galilean relativity. Everything is interpreted through this lens whereby nature is treated as context variant in this way, and it resolves all the paradoxes without introducing anything else. So if you can accept that one premise then everything else is explained. By abandoning metaphysical realism, it also simultaneously solves the other philosophical problems that originate from that point of view, i.e. the "hard problem" does not even make sense in a contextual realist framework and is not applicable.