quantization is always a problem; what quant were you using?
Q8_0 on my laptop, hope LiquidAI team can give some feedback on what model they offer in the playground!
alt Hacker News
Q8_0 on my laptop, hope LiquidAI team can give some feedback on what model they offer in the playground!