You shouldn't let that discourage you. VLLM is really finicky, but there are easier backends to use (though which one depends on your hardware).
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
You shouldn't let that discourage you. VLLM is really finicky, but there are easier backends to use (though which one depends on your hardware).