I wonder how Waymos know that the traffic lights are out.

A human can combine a ton of context clues. Like, "Well, we just had a storm, and it was really windy, and the office buildings are all dark, and that Exxon sign is normally lit up but not right now, and everything seems oddly quiet. Evidently, a power outage is the reason I don't see the traffic light lit up. Also other drivers are going through the intersection one by one, as if they think the light is not working."

It's not enough to just analyze the camera data and see neither green nor yellow nor red. Other things can cause that, like a burned out bulb, a sensor hardware problem, a visual obstruction (bird on a utility cable), or one of those louvers that makes the traffic light visible only from certain specific angles.

Since the rules are different depending on whether the light is functioning or not, you really need to know the answer, but it seems hard to be confident. And you probably want to err on the side of the most common situation, which is that the lights are working.

That's what I thought. Then I walked buy Waymos stuck in the middle of the block with nobody in front of them.

I miss the time when "confused" for a computer program was meant in a humorous way.

Would have hoped they trained for this but at least now they likely will be.

That sounds plausible. Humans for the most part can usually navigate that situation to a point. It wouldn't surprise me if Waymo cars weren't even trained for this scenario.