The iCalendar standard also includes tasks and notes and Stalwart hasn’t implemented those yet.

That's incorrect. Stalwart supports every single iCalendar IANA registered component and that certainly includes VTODO and VJOURNAL.

That's probably because you were using RocksDB as a backend, which does not work well on mechanical HDDs. Try using PostgreSQL instead.

Only the iCal/vCard parser and serializer, it is available at https://github.com/stalwartlabs/calcard