Basically the title.

For me, a dense model around 20b, with an additional engram table of 10-20b, or a sparse MoE around 80b would be amazing.

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 5 days ago

Agree, and upvoted.

Another good size would be 160 gb for 96 gb vram systems. After that I would like to see 225 to 250 gb.

  • source
  • parent