Where is this coming from? I don't think an LLM can code at the level of a recent cs grad unless it's piloted by a cs grad.

Maybe you've had much better luck than me, but coding LLMs seem largely useless without prior coding knowledge.