The main 'Bletchley Park' exhibition is good, but it focuses on the human experience of the code breakers. Head around the corner from the car park to the National Museum of Computing (also on the Bletchley Park site) to see more technical exhibitions: they give proper demonstrations of the machines invented at Bletchley, as well as the oldest working computer in the world (which was computing prime numbers when I visited).
I wouldn't skip the main exhibition area. In an era where people were called computers, the human experience was at the heart of the Bletchley Park machine. In the main area, you learn about the makeup of this apparatus: the different roles people had, how information flowed within and between the huts, and much more. There's also a little museum with fascinating artifacts and an area dedicated to Turing. Don't miss it.
Agreed, it is well worth visiting both.
Also, and not obvious, because these two entities are distinct despite occupying the same site: they're not always open at the same time. So if you want to see both, even if you plan to spend more time at one than the other, check they're both open.
Whether something is the first computer is - inevitably - a definitional argument, but TNMOC has several candidates (though not all of them) including (a modern reproduction of, the original was destroyed as a secret) Colossus which is famous because of its involvement in the war.
Bletchley Park is also still an actual stateley home, all the war stuff was built on somebody's grounds - there's a good chance you either don't care about stately homes or you're intending to visit a more interesting one (or indeed one of the Royal Palaces), in which case no need to care, but that's a third distinct thing on the same site.
[Edited to make clear there is no original Colossus, we destroyed it because it was a secret]