gte-small outscores all-MiniLM-L6 on MTEB (~61 vs ~56 avg per the GTE paper). MiniLM is ternlight's teacher (ternlight holds 0.84 Spearman fidelity to teacher).
I haven't run a head-to-head yet; STS-B/MTEB numbers are on the roadmap. Also on the roadmap is to distill gte-small as teacher.
alt Hacker News
gte-small outscores all-MiniLM-L6 on MTEB (~61 vs ~56 avg per the GTE paper). MiniLM is ternlight's teacher (ternlight holds 0.84 Spearman fidelity to teacher). I haven't run a head-to-head yet; STS-B/MTEB numbers are on the roadmap. Also on the roadmap is to distill gte-small as teacher.