Not entirely wrong, but tokens are not just "fake" in the way, for example, an in-game currency is. They're the fundamental "units" of data, both input and output, processed by the model. For most models, tokens are just a certain number of characters or words. So they're not completely untethered from the model. If we're both using Clankerbot v5.1: Sloppy Logic Edition™️, your tokens are defined in the same way mine are.

This is near the edge of my limited understanding, but AFAIK, yeah they can mess with token costs and billing schemes all they want. They could theoretically charge us 2 different costs per token, or do surge pricing or some shit.

if they wanted to change the actual size/definition of what a token is though, that would require a whole new model (or at least a major revision).

Tokens are well-defined groups of bytes ranged by frequency of occurrence in texts to efficiently translate them into a sequence of 32 or 64-bit binary integers, an LLM-optimised form if compression. They are well-known, you can play with them here: https://gpt-tokenizer.dev/

Maybe you're confusing tokens with the "credits" you pay for. Tokens have a technical meaning, but some companies are charging per AI credit, where they don't tell you the conversion rate of credits to tokens, so they can change this at any time, or vary it between models, etc.

It's not like that. Tokens are an inherent computational property of how a model calculates the probabilities and such to generate text.

Having said that, what a token means in terms of computation varies wildly between models and is not directly comparable. So attributing a money value to tokens in general, independently of the model, is weird by nature.

And even within a model, the number of tokens needed to generate a response is very variable too, depending of the model itself and the parameters with which it has been configured (thinking mode, temperature, etc.).

So yeah, companies can pretty much set any price they want and there's not much anyone can do about it.

People are arguing with you that tokens are "real" - they miss the point. You cannot predict how much "tokens" you will spend for a given prompt.

That's the problem you're highlighting, we are charged for a metric we cannot estimate before buying to make an informed decision how many we want to spend vs the quality of the outcome.

The charge and quality is arbitrary, and we can't trace if we actually spent as much as we were charged for.

Anyone who waffles about tokens being real - I believe missed your point.

“just made up” if you mean arbitrarily defined, sure. It’s not like a “bit” that has an irreducible objective definition.

However it does have a definition in whatever context you’re looking at and is very real, so I can’t really agree with your whole comment.

Yes the definition could be changed to jack up prices, but prices can also just be changed.

Dollars are “just made up” too and have varying value in different contexts to different parties. Are they “not real?” Would we hand wave away the entire financial section of the news and say bah, these “dollars” are fake anyway?

This has 3 upvotes at time of writing in a technology community when it's so obviously ignorant of the actual technology that it should be an object of pity or mockery depending on the vibe.

Ignorance is a problem only if you are made aware of it and nothing changes.

https://blogs.nvidia.com/blog/ai-tokens-explained/

For subscriptions they use a black box metric nobody knows. For usage credits, tokens are very measurable.

The subscriptions are much cheaper than usage credits but have been nerfed in the past and will be nerfed again in the future

What do you use that file for? I see Claude.md thrown around and I'm a bit curious.