You are correct, most people who were trying to get from California to Maine or the reverse for the purpose of getting there would probably prefer to fly. Unless a ticket were significantly cheaper than air travel almost everyone doing the full trip has at least one train enthusiast in their party who is doing it because they want to do it.

The thing is, a high speed corridor enabling that trip would, as you note, involve many stops of which each pairing represents another potential flight that the same train might replace. Many of those pairings will be short enough flights that a proper high speed rail corridor would easily compete with if not destroy the short-haul flight offerings.

To use your example, it looks like there are currently two flights between California and Maine, both run weekly by United between PWM and either LAX or SFO flying east on Saturday mornings and returning west on Sunday morning. A hypothetical high speed rail corridor connecting coast-to-coast between either of those city pairs might not replace a lot of flights across the whole country but one train running along that full route might have replaced one from Portland to Buffalo, another from Boston to Columbus, another from Cleveland to St. Louis, et cetera.