logoalt Hacker News

jeroenhdtoday at 4:13 PM3 repliesview on HN

By their own industry data (https://store.steampowered.com/news/app/553850/view/49158394...), deduplication causes a 5x performance increase loading data from HDD. There's a reason so many games are huge, and it's not because they're mining your HDD for HDDCoin.

The "problem" is a feature. The "so it remains unfixed until someone with pride in their work finally carves out the time to do it" mindset suggests that they were simply too lazy to ever run fdupes over their install directory, which is simply not the case. The duplication was intentional, and is still intentional in many other games that could but likely won't apply the same data minimization.

I'll gladly take this update because considerable effort was spent on measuring the impact, but not one of those "everyone around me is so lazy, I'll just be the noble hero to sacrifice my time to deduplicate the game files" updates.


Replies

hinkleytoday at 5:45 PM

> In the worst cases, a 5x difference was reported between instances that used duplication and those that did not. We were being very conservative and doubled that projection again to account for unknown unknowns.

That makes no goddamn sense. I’ve read it three times and to paraphrase Babbage, I cannot apprehend the confusion of thought that would lead to such a conclusion.

5x gets resources to investigate, not assumed to be correct and then doubled. Orders of magnitude change implementations, as we see here. And it sounds like they just manufactured one out of thin air.

hinkleytoday at 5:26 PM

Seems to me that most of these situations have an 80/20 rule and it would be worth someone’s time to figure out what that is.

Get rid of 80% of that duplication for a 2x instead of a 5x slowdown would be something.

JohnBootytoday at 4:52 PM

I expect better from HN, where most of us are engineers or engineer-adjacent. It's fair to question Arrowhead's priorities but...

    too lazy
Really? I think the PC install size probably should have been addressed sooner too, but... which do you think is more likely?

Arrowhead is a whole company full of "lazy" developers who just don't like to work very hard?

Or do you think they had their hands full with other optimizations, bug fixes, and a large amount of new content while running a complex multiplatform live service game for millions of players? (Also consider that management was probably deciding priorities there and not the developers)

I put hundreds of hours into HD2 and had a tremendous amount of fun. It's not the product of "lazy" people...