Hi there, ISRG co-founder and current board member here. In brief, shorter lifetimes force people to automate (which, e.g., avoids outages from manual processes) and mitigates the broken state of revocation in the Web PKI. That latter point especially is what I understand to be driving the Web PKI toward ever-shorter lifetimes.

I actually remember the discussion we had in ~2014 about what the default certificate lifetime should be. My opening bid was two weeks -- roughly the lifetime of an OCSP response. The choice to issue certificates with 90 day lifetimes was still quite aggressive in 2015, but it was a compromise with an even more aggressive position.

Because big companies have a habit of growing layers of bureaucracy. If a cert is valid for three years, a decent bunch of them will invent a three-month process around cert renewal, involving two dozen stakeholders, several meetings, and sign-off from the CTO.

The side-effect of this is that they become incapable of doing it any faster during an emergency. Private key compromised? Renewal takes two months, so better hope the attackers can't do too much damage before that. CAs in turn have large (=profitable) customers which such processes who they really don't want to lose, so historically when they've failed to renew in time during incidents CAs have granted those customers exceptions on the revocation rules because they are "business critical" and doing it by-the-book would cause "significant harm". No CA is willing to be strict, because they'd lose their most valuable customers to their competition.

The only way to solve this is to force companies into adopting efficient renewal processes via an industry-wide reduction of certificate validity time. When you have to renew once a month you can't afford to have a complicated process, so you end up automating it, so there's no reason for CAs to delay cert revocation during incidents, so the internet is more secure. And because every CA is doing it, companies don't gain anything by switching to more lenient CAs, so the individual CAs have no incentive to violate the industry rules by delaying revocation.

Lets Encrypt are doing is because of the decision that CAs and browser makers made that it needs to be reduced (browsers have been reducing the length of certs that they trust).

The why is because it's safer: it reduces the validity period of private keys that could be used in a MITM attack if they're leaked. It also encourages automation of cert renewal which is also more secure. It also makes responding to incidents at certificate authorities more practical.