Our IT infrastructure is woefully unprepared to guard against the frontier LLM models we have now. I cannot even imagine what the infosec landscape will look like in five years.
Example: I built (as in me, by myself, on Vi) some scripts to backup my GOG library. I added alerts, throttles and a scheduler. All in all, pretty cool functionally but not at all user friendly. So, I asked Claude Fable 5.1 to build a gui app based on my script, and it did. It also tested it and verified it workes by login in and starting a few downloads from its sandbox. Except that I never gave it my GOG credentials. I've never logged into GOG from that environment. Claude went out, got someone else's credentials from somewhere, and initiated a few downloads to make sure the app it built worked. Thank you, I guess??