The thing I have to constantly adjust for is: it does things at 10x my speed (on average, some things it does slower, but some things it does 100x faster), and about 80% of my accuracy. So, I have to really aggressively apply the bug catching methodologies that a lot of times I would get away without "because I'm that good..." not really, but there are some things I do that I have justifiably high confidence in, others that definitely need testing which I try to do before handing over to the test team. The reason I don't apply those bug catching methodologies to ALL my code is because that slows me down 3-5x, and it seems to slow down the LLMs about 2-3x, so they're only 3-5x faster than me, on average, when they are writing all the tests and multiply reviewing all the docs and code, compared to me "winging it."

Where I think all this LLM stuff is going to pay off bigger is 10 years in the future, looking back at old code, yeah all that documentation is a bit of overkill, but we tended to underkill the documentation 10 years ago and we're still not doing as much as we probably should when we do things the "just before LLMs came on the scene" ways.

Well... That's why you test the public API and not the internals. And if you need to change the API you should have a good enough reason that changing the tests is okay too.