That is because time and energy are limited resources, and measuring merit accurately is very costly. Measuring appearance is far less costly, and might serve as an acceptable proxy. And often times it might not.