I think the desktops need to better protect users' secrets by accepting rogue user processes are part of the threat model now. There are lots of mechanism to lock known future user processes into a protected area (containers, chroot, etc), but none to lock existing and future unknown user processes out of a protected area.

Your idea of a yubikey access protected file area makes a lot of sense and I don't think it exists yet. Then a user could throw their existing ssh keys in there and immediately get physical protection on them.

It would need to be carefully controlled. Something like gocryptfs on FUSE as you suggest but with a stronger threat model layered on top as theirs is short: https://nuetzlich.net/gocryptfs/threat_model/