One way to beat this in terms of coding efficiency would be to use something like rANS (explanation [1], code [2]).

You would have a more complicated decoder with rANS though, even if you removed the frequency table lookup (because always 1/3) and you'd only produce one output at a time instead of 5 doing this with a lookup table.

It might be possible to use a combination of both techniques with e.g. a 16 bit accumulator, multiplying by 243 to get the value to lookup into bits 23-16 and then bias and shift the incoming 8 bits into the accumulator.

[1] https://fgiesen.wordpress.com/2014/02/18/rans-with-static-pr...

[2] https://github.com/rygorous/ryg_rans