These are trits, which provide their own efficiencies.
Interestingly, a trit x float multiplier is cheaper than a trit x integer multiplier in hardware if you're willing to ignore things like NaNs.
0 and 1 are trivial, just a mux for identity and zero. But because floats are sign-magnitude, multiply by -1 is just an inverter for the sign bit, where as for integers you need a bitwise inverter and full incrermenter.
Do you know a good reference to learn more about this (quantizing weigths to 1.58 bits, and trit arithmetic)?