so a model trained in 2 year old data is just as good? as a model trained today?
Yes, assuming a plateau in LLM capability the only reason you'd train on an updated corpus is to get fresh info. That's not worth it because:
- The model will most likely be wrapped in a harness that has search capabilities, so it can use that to fetch fresh info
- 2 year old data may actually be better as the Internet becomes increasingly tainted with LLM content
Most of the improvement from new models isn't coming from expanding or updating the corpus, its coming from increasing the parameter size and reinforcement learning.
So they are just choosing to lose money?
No, they are competing. If open AI decides to stop training new models right now then anthropic will and take all there business as the switching cost for models is low. That's why they want the government to regulate it, so they can have a ceasefire to start taking in profits from inference without worrying about their competitor making a new better model and eating there lunch.