So does me forgetting to lock the door.
I once left my garage doors open all weekend with thousands of dollars of tools in it, along with my motorcycle with the key in it. My garage at the time was quite visible from a very busy street. I got lucky and nothing was stolen. My garage doors are smart now, so that doesn't happen any more. Sure, someone could hack my home assistant server and open the garage, but that's way less likely than my continued pattern of forgetfulness.
Based on that logic, my house also has smart locks now that automatically lock when I leave. I will say that they aren't cheapo who-knows fingerprint locks, they're made by a real company and just have a keypad with a lengthy entry code. The only added vulnerabilities should be my home assistant server and the entry code.