Trying something new, going to pin this thread as a place for beginners to ask what may or may not be stupid questions, to encourage both the asking and answering.

Depending on activity level I'll either make a new one once in awhile or I'll just leave this one up forever to be a place to learn and ask.

When asking a question, try to make it clear what your current knowledge level is and where you may have gaps, should help people provide more useful concise answers!

you are viewing a single comment's thread
view the rest of the comments
[–] 5 points 3 years ago (3 children)

What can I run on a 1080ti and how does it compare to what's available in general?

  • source
  • hideshow 3 child comments
  • [–] 8 points 3 years ago (1 child)

    On Huggingface is a space where you can select the model and your graphics card and see if you can run it, or how many cards you need to run it. https://huggingface.co/spaces/Vokturz/can-it-run-llm

    You should be able to do inference on all 7b or smaller models with quantization.

  • source
  • parent
  • hideshow 1 child comment