They have your location, and can detect your interests from what you watch, and fingerprint you. Things like this are pieces of a larger, comprehensive picture of your life.
Also, how do you pay for streaming channels, etc., if you don't give it any information?
The assumption here is that we don't give the TV any personal information; it's just used as a monitor. The content comes from some other device connected to an input (e.g. HDMI1). (That device may raise privacy issues, but we are concerned with the TV).
Without either doing the fingerprinting from surreptitious audio recording in the room that other commenters mention, or else matching the unit serial number to the purchaser (requiring the retailer to leak personal info to the manufacturer), they can't match the unit's location to our identity.