For email you can use a classifier

One way: separately embed sender, recipients, subject, body - then use the embedding vectors as input to a logistic classifier

With that setup, I get 95% accuracy on email classification, training on 50-100 base examples. The model trains on CPU in under 1min, and it does inference in under 20ms (most of it is running the embeddings, so you can make it faster if you train your own embeddings model)

Here’s a gist with some sample code: https://gist.github.com/nicobrenner/056a5aaff5d0119c0032ecda...

That code applies the embeddings + classifier setup on the Banking77 dataset. It gets 93-94% accuracy depending on the embeddings you use (SOTA for this is ~95%, with much bigger and slower models)

> hold your horses to paint it as dirt cheap

For a moment I thought this was going to be a metaphor — maybe an ancient Chinese proverb about how paint brushes are made from horsehair and how you can't hold the horse to paint before you've turned the hair into a brush.

How are you benefiting from prompt caching for simple classification?

Are you getting better performance from an LLM than a Bayesian classifier?

How many requests per second do you have for spam that you are reliably hitting the Luna cache?

Is that just because the Jev implementation is less mature? Couldn't it also implement prompt caching?

What are the costs compared to an ML model?

[dead]