Yeah that's exactly what I'm expecting as well. The real difference in philosophy is that Americans companies treat the model as the product, while Chinese companies see models at infrastructure you build products on top of. You amortize the cost of deploying it at scale by sharing knowledge and iterating quickly to bring the cost down.

Models themselves are general purpose tools, so it's not where the money is going to be long term. There's a reason everybody isn't rolling their own operating systems for example. It makes a lot more sense to treat models as shared infrastructure everyone contributes to. We'll likely converge on a handful of common architectures because there's really not much difference between them at the end of the day. Everybody curating their own model is a huge duplication of effort with no clear benefit.

If you treat the model as the product, then it makes sense to keep it closed. You have some secret sauce that nobody else has, and you sell it. But the reality is that nobody has a magic formula that's significantly better than what other people can figure out. You might get an advantage for a few months tops, and then other models start catching up.

And this creates involution where you just have a race to the bottom where nobody makes any money. On the other hand, if you treat models as infrastructure, and everybody contributes to the same pool of knowledge, then you amortize the cost of making a better model. The money comes from actual products that can genuinely differentiate themselves. Companies are going to seek niches they can dominate where they do a specific thing really well. That's a much more realistic path towards long term sustainability.

And continuing to work in the open with the rest of the world means getting the benefit of having a global community of researchers helping advance this tech forward. It's not just altruism or clout. American companies working on closed models have to foot the bill for all the research, and they're limited to the brainpower within the company while they're competing with Chinese companies which have much bigger research community contributing to developing their models.

If the model itself is not the product, then American companies find themselves in a situation where they're spending a ton of resources on something that's not their core business.

Isn’t this just mixture-of-experts architecture?