I just found out about AppImageLauncher, a package handler for AppImages. It organizes them, creates desktop files for you and handles updates and removal.

Integrate AppImages to your application launcher with one click, and manage, update and remove them from there. Double-click AppImages to open them, without having to make them executable first.

Much better than having to create all the desktop files myself, and having to figure out what to put in them for it to work correctly (I'm looking at you, qBittorrent and magnet links).