It doesn't typically get run millions of times per day because in most regular uses it's trivial to cache the glyphs. I use it for my terminal, and it's not in the hot path at all for rendering, as its only run the first time any glyph is rendered at a new size. If you want to add hinting and ligatures etc., it complicates the caching, but I have no interest in that for my use, and then it turns out rendering TrueType fonts is really easy:

https://github.com/vidarh/skrift

(Note that this is a port of the C-based renderer libschrift; the Ruby version is smaller, but much less so than "usual" when converting C code - libscrift itself is very compact)