Building the kernel is not that hard, as you have kernel-devel which has all the sources.

Yeah. Some myth that it's hard to do is not why we end up with monolithic kernels. Like any case where you find yourself thinking "it doesn't look that hard; I could do that easily", it's either harder than it looks or it's done a certain way for an entirely different reason you haven't figured out.

You should learn that reason.