Linux phones try to build from upstream Linux, and the major phone SoC vendors HATE upstreaming their code.
They believe every character in their source code is absolutely top secret.
A middle ground I wish was considered more is taking Google's kernel and the vendors DLKM partition/DTB/DTBO for hardware support, and putting a GNU userspace on top.
This has had problems in the past, because vendors would modify syscall tables such that they don't match userspace anymore, but with GKI, I think we're closer to that being a possibility