Yes, but in terms of speed you don't need the same parameters and quantization for the secondary layers.
If you haven't seen it, see how fast a very capable model can actually be: https://groq.com/
Yes, but in terms of speed you don't need the same parameters and quantization for the secondary layers.
If you haven't seen it, see how fast a very capable model can actually be: https://groq.com/