I'm curious what you are using. The free versions of chatgpt have been like that for me, but even Gemini flash with extended thinking, also free for a while longer, is giving me pretty reliable results as long as there training data out there to derive an answer from. The higher (paid) Claude models will one shot most coding tasks.

Generally I've found that:

If the facts are painfully obvious from a simple web search, then GenAI has a decent chance of getting it right. This can be useful if you can't recall any "key" words well and the GenAI can craft several searches and get there.

However, if it does mess up the facts, the result looks superficially the same as "correct". So while it can give accurate data, you always have to double check. This can still be useful, as finding the right search terms can be a decent help.

In coding, sometimes in some situations, you can have requirements that are absolutely testable, and thus you can have the models retry and retry until it works. This isn't always feasible. Even when it seems feasible, you may screw up the criteria, or the GenAI when enough freedom disables a probablematic test rather than solve it, and it likely will generate code that's not really fit to modify. There are a lot of situations where this is useful, but it is infuriating that non technical people and even some low skill technical people assume this is always the case.

Then when you get away from facts mattering, it gets "better". Example, someone jokingly asked for one to "make gta6". After a while it came back with a GTA 1 clone. Lots of people were impressed, because whatever it did could be considered a success even as it obviously didn't match the expectation. The operators also like to GenAI some webcomic, where it is a fiction. They almost always didn't have any interesting thought going in so they tend to be crap, but "correctness" didn't matter.

Of course, also making fakes. Supreme case where looking correct matters but being factually correct does not matter at all. GenAI above all else "seems" correct.

I've been able to find a workflow where it's mostly correct and can handle most of my gamedev related coding without making too many mistakes. I still have to actually read through the code and pay some attention to what it's doing, and if it misses something and goes on a wild goose chase, it's unusable and I have to start over (so someone who didn't know what they are doing would be cooked), but whatever.

Sure, it does require a lot of looping adversarial reviews, and my average token cost is around 3000$ (4B tokens) a month (we have unlimited budgets and a pretty accurate tracking, and also probably cheaper than consumer prices per token with how large company it is), which is actually more than my monthly salary, but it's just a job, for a company and on a product I don't really care about, and I can 1) keep slacking in my job while doing my own coding stuff and projects, and keep seeing how absolutely unreasonable the prices are if you want to get at least semi-submitable results.

You could ask if it's necessary to spend so much tokens - but so far most adversary review rounds do find serious blocker bugs the first implementation had, most of them being some hidden stuff that would be difficult to reasonably find by hand unless you really understand everything around the code, which you should, but the point of AI is that you shouldn't have to.

Is it worth it? Lol, no. The whole team is loosing codebase knowledge, we're getting bottle-necked by pending PR reviews that are just stacking up and no one wants to do, so we're not even more effective, the cost is absolutely absurd and in no way near sustainable or worth it. And the longer it goes on, the less I can realize it's spewing bullshit and I should stop it and nudge it into a different direction in our codebase, because I'm slowly loosing touch with it, while for now I'm still relying on what I remember from before we went all-in on AI.

And that's while the whole industry is in the "Uber pricing" phase, so it will get a lot worse. But yeah, if you can burn 100-200$ per a simple implementation task, then it can have a pretty usable results. And that 3000$ a month does not include our CI review bot, that does additional rounds of multi-agent council reviews. And I'm not even working on anything complex, mostly just various menu screens UI. I can imagine the bill getting a lot higher if you get into more involved systems.

Yeah, every time I say this to someone they say it's fine when they use it because they pay for the latest expensive model. Except they've been saying that for the last couple of years, so I guess they were lying back then...?

from my point of view it feels like most people are willing to trade their cognitive function for being lazy, which is a boundary i never want to cross.

AI will produce a lot of code but most of it is pretty poor quality as most training data is going to be poor quality code, there's just always going to be more bad code than good to begin with just due to how difficult quality code really is to produce. I'll give a hint, good code is usually small and succinct.

since my work started pushing AI live site issues have increased dramatically. turns out the person using AI looks like they have a ton more productivity but in reality that just shifts to whoever is reviewing the code. and to those who will say its the developer's responsibility to review the code, yeah no shit, but if you ever work corporate you realize most don't care as long as their managers think they are productive and just blame others for being bottlenecks.

Overall it just enlightened me to how bad the average developer really is, but i guess obtaining mediocrity is the sacrifice to make in the name of "productivity".

That doesn't mean he's wrong about AI though.

Whatever it was it was bad enough for his wife to divorce him

"allegedly underage" oh sure, ole Willy Gates fucked Epstein's old lady slaves. Which he was known for.

He's working too hard. We just care about his well being. He doesn't need to be doing everything by himself.

Yeah really. Also buying up farmland.

The video where he gets asked about Jeffrey, the micro expressions, eyes all over the place, fidgeting, tonal changes, stammering, indirectly stating being involved by bringing up slight details without being asked directly about them, etc--it's easier to watch the video. He lost his cool so fast it makes Clinton look like a smooth criminal.

https://www.youtube.com/watch?v=9LffGJNOZUE

“Millions of people around the world will soon consider large models ‘hoovering up’ all their work to be an astonishing theft of unprecedented proportions,” an internal Microsoft document admitted.

https://futurism.com/artificial-intelligence/ai-is-creating-doom-loops-all-over-the-economy

Prices will definitely rise to cover operating costs and to recoup all the investment costs.

I honestly hope the whole thing crashes once people realize they can run their own LLM on their own hardware.

A friend who’d just bought a new car spent 10 min on the phone with Geico trying to speak to a human. Their Sam’s Cola version of ChatGPT convinced me to never consider them for insurance. It was fuuuucked.

In the end If you don’t have workers you don’t have consumers.

He repeats the "better than humans at coding" which is super situational. Executives got so used to "code is code" mentality they can't seem to understand that just because they can prompt up straightforward apps and other folks have done stunts like "make a knock off of this compiler" does not mean it's just better than people.

Also very bizarre to say they have perfect memory, they are pretty spotty at that.

On the matter of support, it will generally always such because it's rigged for the company in all matters where helping the customer would go against the company's interests. Has been a long time. Why pay warranty when you can make it so painful they will give up?

The problem is he should, but he definitely won't.