Basically the title.

For me, a dense model around 20b, with an additional engram table of 10-20b, or a sparse MoE around 80b would be amazing.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 day ago

Something that would make good use of a 128GB spark. GPT-OSS-120b is good but shows its age

  • source