It does write code that usually works, but it makes absolutely rookie mistakes that are, like, worse than junior engineer level. If that impresses you, then so be it, but in my 28 years of writing code, I’ve never been less impressed by something with as much hype as AI agents have.

Btw, there’s not really an objective way to measure software engineering skill. Even the best tests don’t take into account all aspects of engineering skill. That’s why I would be only one of like ten engineers interviewing the same candidate at Google. The “objective” tests are called phone screens, and they’re just the first round of interviews. If you can’t pass those, it usually means you’re not qualified to be an engineer, but just because you can pass them doesn’t mean you are qualified.