logoalt Hacker News

NegativeLatency • today at 4:23 PM • 0 replies • view on HN

https://github.com/microsoft/LLMLingua one such library I’m aware of

Tried porting it to swift, was hoping it would make mobile local models more useful for a project I was working on but it was hurting the accuracy too much. Can be very task dependent on how much you can compress. https://github.com/nburns/llmlingua-swift

My specific use case was using a LLM on segments of labeled data from a different faster model as a final pass.