Expensive mp3 players are for audiophiles. They have better much DACs than a phone, hi-fi headphone jacks in addition to regular ones. Probably other hi-fi features I don't know or care about. They can easily run over $400.

Inexpensive mp3 players are for people who want a distraction-free device, who want a headphone jack, or like physical buttons, I guess, and they also often have somewhat better DACs than a phone, not that my busted hearing can tell. I think decent ones are in the $50-100 range. I have a $70 that I like fine. It's metal, it plays most media formats (not opus, which annoys me, and it flakes out on some specific ogg vorbis). Only thing I don't like is switching between linear and shuffle play is too many button pushes.