you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 4 months ago

Often they do that within the model's architecture itself. You can save drastically on the filesize of the model itself by using this Mixture of Experts (MoE) strategy. Its how deepseek was able to break into the market.

  • source
  • parent