694
Sounds like a good way to move around money real and imagined.
Just to make things clear: API access to most models is charged per input tokens + output tokens. It means that the longer your conversation is, the more you pay for every new answer. Single prompt with no context and 100 tokens of answer is cheap. Single prompt with 100k tokens of context and 100 tokens of answer is NOT cheap.
Extremely long conversations with most expensive top of the line models can absolutely demolish your budget.
does it give the full history to the LLM each time?
Last time I tried implementing something like this, it suggested to have a rolling window of history so that it takes into account your last X messages but not the entire conversation.
(I guess this is what ollama calls "context length"?)
Most agent harnesses do something called "compaction." For example, here's how Pi does compaction
You send the entire history for that conversation every time and likely more if its getting info from tools. If its not in the context the model dose not see it unless you have a memory system that dose something like feeding in summaries of past conversations that also takes up tokens and context. Rolling drops old messages to not reach context limits but you can lose important info or get odd results. If the history gets bigger than the context things break or slow way down.
presumably this is why Claude periodically writes its conclusions so far into a text file that it can read later instead of having to remember everything. Sounds like an interesting approach.
> Be a corporate executive
> Tell your employees to use more AI in their workflows
> Punish employees who don't use enough AI, while rewarding those who use it the most, irrespective of actual outcomes
> Be shocked when your company blows through an absurd amount of tokens in one month
Don't know why bosses are universally this out of touch in literally every single industry
The paradox of promotions based on performance in a previous roll is that you end up with incompetent managers unable to move upwards anymore.
Because this system rewards incompetence as long as it comes with dark triad traits and a heaping dose of nepotism.
I'm really jealous of these types of guys' ability to lie without feeling anything. If I lied like that, I'd be embarrassed because my words sounded like bullshit. How do they do it?
Cocaine helps with that
Damn
It's probably a lot easier when they have themselves fooled as hard as any other
See, I don't know! I used to think that too! But then I actually met some people like this, and a lot of them absolutely do not believe the shit they say! They're just really good at convincing other people that they do.
The more recent report says corporate AI adoption has found several issues with AI, with human workers turning to automating dreary and mundane tasks they don't like doing, rather than valuable or meaningful work.
Thank god we have consulting companies to tell us what humans like!
stop feeding it data centers and you have a better chance at controlling it
Maybe AI will finally negatively impact some CEO jobs.
Nah they'll make up the costs by laying off workers, determined by some bs metric
What's funnier is that typically the AI providers lose money on every query their customers make. So, this may have cost some company $500m to Anthropic, but it cost Anthropic a whole lot more than that.
What a brilliant business model.
maybe they are planning ahead for the business model in a few years time, when nobody can do any work without claude, and they get to charge their preferred "monopoly enshittification" price?
Absolutely this is the plan.
They make it up in volume.
(Volume being how loudly they shout about how it's going to change the world and dupe more people into investing.)
load more comments
(1 replies)
I just want to know what are the best things to type into these ai chat boxes that will cost the most. If my company wants me to use this garbage then I want to make it as expensive as possible and when their liscenses need to be repurchased I want it to be as expensive as possible to continue to force this garbage on us
Edit. Hey everyone lots of great replies here, please keep the suggestions, fixes, corrections etc coming!
These high prices are not from people talking to chatbots.
They're using agentic tools where their prompt spawns a lot of bots which talk to themselves/the other bots and they keep going until someone (usually a higher quality reasoning model) decides that they've met the goals of the task that they were assigned.
So instead of 1 prompt and 1 response, you get 1 prompt and 800 responses across 5 different bots each using really large context windows.
"Continue modifying this code until all unit-tests pass"
(gives it conflicting unit tests)
load more comments
(10 replies)
load more comments
(2 replies)
When you owe Claude half a million, you've got a problem.
When you owe Claude half a billion, Anthropic has a problem
It's probably Amazon. They can absolutely afford it.
But if we are to uncritically believe what the AI peddlers told us, that means this mystery company should be reaping $10 billion in additional revenue or quantifiable gains in productivity!
load more comments
(1 replies)
Either I have some inside knowledge of that exact thing happening and I know the company (not saying who) or this is probably a common things that happened to a lot of major companies (more likely). To be fair, I do not have privy on how far it went and how much it cost before they realize the problem, and it may not have been this much. Which further suggests it's a thing everywhere.
Most companies can't eat a half billion dollar loss so who ends up paying this? AI queries burn actual energy so the AI company would have to charge I would think.
Most companies can’t eat a half billion dollar loss so who ends up paying this?
Taxpaying proles will foot the bill somehow.
load more comments
(1 replies)
view more: next ›
this post was submitted on 29 May 2026
694 points (98.6% liked)
Technology
84998 readers
3496 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS