I'm no expert in this stuff but:

creata's comment [0] mentions the works of Agner Fog, which seem very good, and are freely available.

I haven't read C++ High Performance [1] but it looks like it covers the sorts of topics you'd expect, although it looks like it doesn't cover computer architecture in detail e.g. branch prediction. There are books on that too, of course.

[0] https://news.ycombinator.com/item?id=49868657

[1] https://www.packtpub.com/en-us/product/c-high-performance-97...