98% Isn't Much

In the early days of commercial optical-character-recognition software, vendors would brag about 99% accuracy.

But a single-spaced-typewritten page has about 500 words, so you were looking at five typos every single page. It was good at the time, but you still had to manually check every single word.

Alternatively, 98% is plenty.

If your business plan requires you to capitalize on more than 98% of the market, it's already a failure. It'll never happen.

As always, it's an "it depends" situation. If your userbase is largely luddites, then maybe you need to support 10+ year old browsers that can't be updated. Otherwise, you can probably just worry about people who are using computers new enough to actually update their browser once a year or better.

The tradeoff is code complexity and engineering time, vs having a larger market. And that's going to be an individual situation for every company.

After Christmas this year, I removed the tree from our living room, and in the process of being moved, it shed of needles everywhere. I swept them up, but I missed a few areas on my first pass. So I did a second pass, but when I looked again, I saw there were still a handful left. It struck me how removing >99% of the needles was nowhere near acceptable! Lots of cleaning jobs are like this, I suppose, because even a tiny mess can be visually distinct. In fact, as you approach 100%, the remaining mess stands out more.

If a fancy new feature can’t degrade gracefully, then 98% isn’t “widely supported”.

Close, but the other way round. Don't avoid a feature because it lacks good enough support. Write code to progressively enhance the experience if the feature is supported in the user's browser. If you're not willing to do that, then don't use the feature.

Progressive enhancement today means you can use pretty much any browser feature you want. You just have to do a bit of legwork with some @supports or JS prototype checking after doing the basic version first. It's not really much extra work.

While I agree with the general sentiment, the problem here isn't developers not being familiar with statistics, it's the simple fact all of this is profit driven most of the time.

I tried to purchase tickets for an event last week. I had to go through Ticketmaster as it was the only official way. They forced me to verify my account using a phone number, but whoever they were using for messages wasn't able to deliver a code to my number. I tried a few numbers from our household and they all failed.

Searching for this issue yielded a bunch of results, so it was definitely a known issue, but there wasn't anything I could do, really. To them, it's simple math. Another SMS provider that covers my (tiny EU) country might be more expensive. They might be avoiding scammers that used my mobile operator in the past. Whatever it is, it would probably cost them more than they lose in ticket sales.

Without some government entity to force them, they don't give a shit about me being able to see an event.

The broader point is that percentages can be misleading, and are often because of that. It makes things sound better. But usually, the more accurate thing to do is use odds-notation ("1 in 50" instead of 98%). Percentages have a kind of singularity at the edges, where small numerical changes have massive real effects. Going from a success rate of 98% to 99% doesn't sound like much, but that's failing 1 in 50 vs 1 in 100. You've doubled the efficacy.

Reminds me of the Meat Loaf song “Two Out of Three Ain’t Bad” which was released in Japan as 66%の誘惑 “66% is Good Enough” etc https://www.discogs.com/release/8303076

I think there are broadly two types of problems - ones where you get partial credit, and ones where you don't. The restaurant example is one where you don't get partial credit - 98% of food being safe isn't enough, it's all or nothing. Paying your employees - all or nothing, you miss a paycheck once, it's a huge problem.

CSS on a website, however, you CAN get partial credit (to an extent). It may not be perfect, but it's at least theoretically still providing some value partially.

I think knowing what kind of problem you're facing is really important when it comes to measuring percentage of "complete".

Reminds me of statistic where most customers already stop going to a particular supermarket if it stops carrying a handful of the items they want to buy

It may sound like a small deal to not carry one brand of chocolate paste but if some customers then also don't buy the 12 other groceries they need at your shop, it doesn't really matter that it's only 2% of products that were discontinued. Supermarket products are so stable not because nobody ever makes anything new but because changing a small percentage frustrates customers

If I can't order from some website, I stop trying. Maybe next week the ESR browser gets an update and it works again but by then I'm not clicking those links in the search results

I don't like treating people like numbers. 98% isn't much and it isn't little. It's just wrong.

If I'm one of the 2% then that's everything for me. Maybe I have good reasons to be in the 2%. And maybe, not caring about that is wrong.

I would rather have a website that only works for 2% of people for the right reason, than a website that doesn't work for 2% for the wrong reason.

Part of the problem is The US Government (and UK Government) use the "2% rule" on their websites and only officially support 98%.

I mentioned 3 years ago that Firefox at 2.2% is dangerously close the being unsupported on government websites, and at this point it's now at 1.9%.

https://news.ycombinator.com/item?id=36776603

https://analytics.usa.gov/ says "There were 1.66 billion sessions in the last 30 days." - so 2% is 33 million sessions if I did my math right.

One thing I wonder regarding browser market share is always: How is it collected?

I assume Firefox users over proportionally use privacy extensions.Thus they overproportionally won't appear on Google Analytics and similar places, which for some statistics reduces the numbers even more than reality.

I was heading to dinner with a friend who worked in infra. Google maps said we could bike across town in 20 minutes. He suggested we leave 40 minutes ahead of time and grab a drink at the bar if we got there early. When I raised an eyebrow, he goes:

"What, do you not live your life based on 99th percentiles?"

I tend to think of work as upside-based on downside-based. Most feature work is upside. 10% lift on conversions is great, 40% adoption is winning, and you're playing for the moonshot of 10x. Infra work is downside-based. 98% secure, 98% available, 98% acceptable performance -- that'll all failure. Winning means the thing works as expected and nobody notices.

Not everything sorts cleanly into upside vs. downside, but a lot does. Allocate your risk accordingly.

There's likely always a line somewhere where effort becomes way out of proportion compared to getting that last mile effort.

Arguably, if you only have a website, that won't work for anyone without access to the Internet. So then you should have a physical presence in each of those people's location, and arguably you shouldn't provide any improvements that give me more than physical presence does, so you should not have the website in the first place, since people without the Internet can't use it or you have to keep your website without any improvements over the physical office.

If you only have a website, arguably 2+ billion people currently wouldn't be able to access it.

And it seems odd to bring 150 mil people as an example, when the baseline should be at least 2+ billion with website only.

> Can you imagine a venue refusing entry to former clients 2% of the time just because they’ve “improved their experience”?

On the flipside, if a client enters enough venues that refuse entry to them because of something the client can fix on their end, eventually the client will probably change themselves -- "If you meet one asshole during your day" and all that.

To bring the analogy back to browsers, if a website works fine for a client, they'll have no pressure to change anything on their end -- why upgrade from Windows XP when the site looks fine in IE6? Eventually the client is forced to upgrade -- normally by their operating system. That works, but what if the operating system adds another 2 years to their end of life -- do you just hang on and hope the shim / hacks you added hold?

The other thing to keep in mind is that if you have a policy of considering 98% to be "close enough", then it only takes 35 of those decisions to remove over half the population. And it'll be exceptionally difficult to work your way back up, because each improvement will be minimal!

(Of course, this assumes that each decision is independent, which, when you're talking about browser support for CSS, is certainly not the case.)

The close-to-home example that came to my mind while reading this is GPU programming, where the percentage multiplies. Maybe there are other similar examples where a large sounding percent needs an exponent and shrinks?

With CUDA you try to keep all threads doing the same thing. Sometimes that’s very difficult, but if each thread does the same thing 98% of the time, is that enough? Well since there are warps of 32 connected threads, you might expect the probability that any thread in the warp diverges to be .98^32, or 50% of the time spent with one thread in the diverged code. 50% still doesn’t sound that bad unless threads diverge at different times, and then 50% warp divergence might mean a 16x slowdown overall. 98% isn’t enough in this case.

Its hard to find these stats now (need to use Android Studio), but about 10% of android users are on Android v9 and below. Android 9 support was recently discontinued by Chromium, such that they cannot update past Chromium 138.

So, 10% of android users dont have web features beyond, at best, June 2025.

caniuse.com does not track this - they lump all Chrome for android together in the latest version.

This is painful as someone who wants to make use of some very useful, powerful new features, but is targeting people who are most likely to have old, slow, not-updated devices...

> Can you imagine a venue refusing entry to former clients 2% of the time just because they’ve “improved their experience”?

The web has been doing this since, when, the 90's?

I use Firefox. In the last year, suddenly quite a few web sites have just stopped working on it for me. Firefox is over 2% market share.

It reminds me of when people argued against covid restrictions saying that the virus had a 99% survival rate. A disease with a 1% infection fatality rate is a terrible disease!

Full disclosure: I also argued against covid restrictions, but not with this terrible argument.

Serving a website is different than serving food or providing safety features. Web design can use progressive enhancement and detect available features to use as they are available.

With a website, you can have the "real" layout, but when someone is blocking your JS, you can fallback and still provide content.

We won't get every mobile application working on old browsers, but we can offer something to the end user. Even a page that lets them know they are missing out.

But CSS Nesting? We can have that and a fallback.

Pragmatically, often users without new browsers and OSses are not the best clients. In ideal world, sure, I want to support everyone. In a world with limited resources, I would better spend my time elsewhere.

I usually go by CanIUse's global percentage when deciding if I can utilize a new browser feature, and right now it's 90.81% (https://caniuse.com/css-nesting)

That's a bit lower than I would be comfortable with, however not that bad, we have been even considering switching all our images to AVIF: https://caniuse.com/?search=AVIF

And google uses Webp all over the place and that's sitting at 96% https://caniuse.com/webp

Author's 98% take is a bit misguided.

Decisions are about tradeoffs. 2% of users staring at a broken screen is bad, of course. But what is the _cost_ of not using nested CSS? The responsible way to make a decision is to consider both sides of the tradeoff.

This concept is missed so much in AI research and is quite frustrating.

The biggest thing missing from this analysis is "is there a business case for supporting those 2% of users?". (Maybe, maybe not.)

The second biggest thing is progressive enhancement. The author picked a CSS feature (nesting) that is basically all-or-nothing: the site will basically be entirely broken for those 2% if you swap Sass for native nesting. Most features aren't like that; maybe the site won't look pixel-perfect on old browsers, or one bit of functionality won't work, but by and large it will still be functional. In those cases, I think it's a much easier decision in terms of where to draw the cutoff.

There are cases when providing service for remaining 2% isn't profitable. It's better just say "sorry".

I used to work in company where we have spent a lot of time making custom fixes for our software in order to work-around wired hardware/software bugs on machines of individual customers. Yes, we provided service for remaining 2% or so, but in cost of slowing-down overall product development and not making our product better for remaining 98%.

If 2 out of 100 people I know see a broken website, depending on the website, that's fine, that doesn't sound like a big deal. Now, if out of 10 power users, all ten of them see a broken site once every 50 logins? Thats a much bigger deal. 98% can be more than enough or not remotely enough depending on the units involved but there are plenty of cases where it's fine to not support the last 2 internet explorer users and stuff

This applies to AI too.

Your classifier might be 98% accurate and it may sound like a lot.

But if it sits inside a car, making thousands of decisions during every trip then you may be in deep trouble.

On the other hand, if you design something to include 100% of people, you will fail, and you will not give your core audience as good of an experience as they could have gotten.

So I like the opposite approach. If there's literally one guy on the planet, and this article/app/idea changes his life... what would that look like?

> If a website uses fancy new browser features and works for 98% of the population, that means that it won’t work for ~150 million people.

Yes, because all of us have website serving, checks notes, the entire world...

98% is probably in the neighborhood of 8% more than I'd require for browsers compatibility before deciding to use a new feature. At a certain people need to update and we do ourselves and our other 98% of customers no favors by catering to the lowest common denominator.

See also, the story of YouTube adding a banner pushing people to upgrade off old IE version being a large factor in people upgrading. Now, obviously, we aren't all running YouTube-sized sites but building for ancient browsers is, very often, a large waste of time.

In the age of evergreen browsers you have to go out of your way to be using something that's not at most a couple years out of date.

What’s much and what isn’t depends on the context. One hair on your head isn’t much but one hair in your soup is.

That framing is setting the question so you immediately are forced to compare pears to apples.

Of course 98% of sterilization is not enough for surgery or for precision in calculating your account balance but the category of landing page conversion a 98% would be astronomically high.

Isn't this obvious?

In some categories, certainty and percentages make a lot of difference--surgeries, accidents. In some, they don't--surveys, grades.

It just depends on the category.

This is akin to saying something as obvious as more percentages are more than less percentages.

I like stuff that takes a statistic that we view one way and explains how to see it another way

in this sense, related https://danluu.com/p95-skill/

That reminds me of an old comic where a guy picks a milk carton from the grocery store shelf and reads in the box: "Now with 0.01% less semen." and he does not know if he's happy or sad about it.

The author is confusing visitors with customers. Refusing entry to 2% visitors? No, no! Forgetting about non-interesting 2% visitors? Not even a blink!

If a plane did not crash 98% of the time, you wouldn't step on it.

Wasn't there something in statistics to describe something like this? i.e. gaussian distributions vs something that's modeled on sparse occurrences, etc?

I thought this was gonna be about uptime.

Well I wasn't very far off I guess! Perhaps "5 nines" is a good threshold for new CSS features too?

That's why things never get better. I mean, i'm all out for retrocompatibility, but if removing something makes my experience much better and unfortunately the thing stops working for 2% of the people with outdated devices/browsers, it's not that sad, but the tradeoffs need to be measured.

I've been in the two ends of this situation, in the 2% with older iPhones and Android devices, and in the 98% with new devices. The 2% cannot hold a tyranny over the absolute majority, and vice versa. Everything must be evaluated on a case-by-case basis.

I guarantee you, if your product is a mobile app, you're excluding more than 2% of the population.

Some things are measured from 0, some are measured from 100. Depending on Expectation.

When expectation is 100%, telling me 98% success rate isn't enough. An example where the argument happens on Reddit, Macrumours and even on HN. When Apple's butterfly keyboard have issues. Apple Supporter was quick to dismiss the issue and point out the double entry is such a small issue because it is working 99.9% of the time. What they don't realise keyboard before that was practically 100%. That 0.1% error rate is infinitely more than 0%.

Another example is Internet connection When you are used to perfect Internet connection, just a small beep in disconnecting turns to be major annoyance. There are plenty of these examples especially with DOCSIS Cable modem. The modem theoretically is working 99,95% of the time, hence cable companies won't fix it. But Disconnecting 10 to 30 seconds every day is annoying enough.

I am not sure if there is a word or terminology for it so this could be better explained to people.

On the other hand, there are plenty of things where 80% is good enough, or doing above and beyond at 96% by getting 80% out of the original remaining 20%.

But is it? Addressing 98%of TAM, is?

Suppose 98% users have not had any sessions crash. You want to build an addon feature that 10% of your users will buy and which will increase the revenue from those users by 30%.

Do you spend time building the feature, or trying to understand why 2% of users sometimes see crashes?

> Can you imagine a venue refusing entry to former clients 2% of the time just because they’ve “improved their experience”?

This reminds me almost precisely of the dynamics of pro sports in the US and how fans are getting priced out of attending games or even watching teams on TV as organizations shift to bespoke streaming platforms.

Software isn't the product though.

Just like the article says, it depends on if the product is an essential or a dessert.

If your product is a "essential necessity" one, then 98% is terrible for your software.

If your product is a "dessert", then for it's software 98% is awesome.

What about those 30% of audience to update their browser? On our web platform, the team currently displays a message along the lines of: 'Please update your browser; this site relies on features incompatible with your current version'.

Great point, but if the 2% are served graceful degradation rather than a broken site, that's probably okay.

Jesse, we need to refactor the edge cases Jesse

And the last 2% is often the hardest part. The low-hanging fruit has been picked, so you're left with these tricky edge cases that may not have a straightforward solution.