The worst epubs are bad because some jackass took some poorly OCRed text and dumped it into the format. The best (retail) epubs are on par with the best PDFs except you don't have to pan-and-scan to read a fucking page. It just reflows.
For novels I want and prefer epubs, but also non-novels if they were released in the last 5 years or so. PDF isn't magic, and there are bad pdfs out there too, scans of photo-copied books and other nonsense.
I oversee ebook production for a uni press so I am familiar with how the proverbial sausage is made. Which is why I still mainly prefer print books.
There is a mode for PDF files that reflows and is logically similar to EPUB in that there is an HTML-derived data model and you have images embedded in the PDF much as they are embedded in the EPUB. Of course if you hate how complex PDF is it is more to hate.