While it may seem counterintuitive, cloud LLMs use less electricity than local LLMs. When serving a single user, an inference engine spends very little time and power doing calculations, and most of it reading in the model (weights) from memory

Cloud LLMs serve multiple users and therefore batch requests, so a model that has been copied once from memory can be used to generate hundreds of tokens