you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 1 month ago

Good point, that should theoretically be double the decode speed, though not sure how the prefill would differ since it’s mainly compute-bound.

  • source
  • parent