The older construction is also very easy to distinguish from the Inca construction. And the Inca themselves know this history in their community. Brien Foerster has a lot about the Inca culture. https://www.youtube.com/@brienfoerster/search?query=inca

The older construction is made of very big stones of hard granite, that fit perfectly together. Assuming they had some concrete, it is easy how they were able to make them fit so perfectly. If you have a source of materials, concrete is not difficult to make. See https://www.geopolymer.org/

People were not stupid, and technologies were invented and forgotten. And just like Roman technologies were lost in the middle ages, this building technology was lost to the Incas.

The Incas build their houses and temples on top of the existing ones. They used smaller stones that did not fit well together. Still a great culture, but with different technologies.

South America has a lot of cultures that disappeared. They had no written history and a lot of stuff was destroyed by later cultures (including the Spanish). So it is impossible for historians to get it right.

For example there were also people with elongated skulls and red hair in Peru. Could be a result of inbreeding as they also had some other physiological differences. Maybe exterminated by another tribe. https://www.youtube.com/watch?v=5dfpLN3FbQs

History is often full with conflicts, but presented as if it is all known. There are often conflicts with engineers who point out different technologies used for buildings and such. These technologies do not fit in the simplified timeline of mainstream history.

This difference in technology is obvious regarding the extremely accurate Egyptian granite vases https://www.youtube.com/watch?v=7BlmFKSGBzI and granite boxes.