The specs and quality of the panel, backlighting (if applicable), and image processing. These days, the few "dumb" TVs that are still sold are either cheap and bad or are designed for signage use and aren't well suited for TV/movies/games relative to their mass-market smart cousins.

A smart TV used as a dumb TV alongside a quality streaming box (Apple TV or Nvidia Shield TV) or console gets you the best overall experience.