That is ONE elite. There have been many elites over history with different levels of wealth and different things they did. Those who could afford a lot of slaves were not out in the field, but not all elite were that wealthy. I'm sure Jefferson wasn't picking cotton, but that doesn't mean no elite were out in the fields.