you are viewing a single comment's thread
view the rest of the comments
[–] 11 points 7 months ago (1 child)

while this is true in isolation, the amount of users means that inference now uses more power than training for the large actors.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 8 points 7 months ago*

    The question is about per-prompt, so number of users is not relevant. What may be more relevant is number of tokens in and out.

    If anything, number of users will decrease power use per prompt due to economy of scale.

  • source
  • parent