I think a CPE could (be/include) a router, but usually it refers to the demarc between the provider's network and the customer's (no matter who owns/manages it).
For a Linux box to be a true CPE you'd likely need somewhat of a specialized card, one that can communicate directly to the next device up the line (e.g, take commercial fiber or cable in, ISDN modem, etc).
If it just shoots out ethernet into some other box next to it, it's likely not a CPE.
Plenty of isps that provide internet over regular ethernet. But it's a ye olde telecom provider term that referred to the phone, that you also didnt own yourself. Doesn't always apply cleanly these days.