-6
FitMyLLM — Independent benchmarks for self-hosted AI (www.fitmyllm.com)
submitted 3 months ago by [M] to c/homelab@programming.dev
5 comments fedilink hide all child comments
 

Check what can you use and at what rate of token per seconds would it be... It has examples of many models and quantization levels. Huge resource!

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 3 months ago

By small, I mean GPUs like outdated ones, laptop GPUs, or like GPUs with only 4GB or 6GB of VRAM.

  • permalink
  • fedilink
  • source
  • parent