> don’t understand “number of books read” as a metric of smartness or literacy or intelligence

Because it’s not. Your plumber is smart and intelligent. They are not literate. This constrains their intelligence nevertheless.