logoalt Hacker News

kccqzytoday at 2:03 AM2 repliesview on HN

The ICU library. It’s so ubiquitous that it’s somewhat difficult to avoid this library if you are doing anything advanced with human text.


Replies

ks2048today at 2:33 AM

And ICU uses data from CLDR, which is mentioned in the blog. Here, there are 380 xml files: https://github.com/unicode-org/cldr/tree/main/common/transfo...

Yes, ICU is ubiquitous. But, some NLP projects use various other libraries, such as uroman (just for romanization - to Latin script).

poizan42today at 5:07 AM

Windows has its own library built-in: NLS. There have been a lot of back-and-forth about whether to use NLS or ICU in dotnet on Windows because it resulted in different behavior between Windows and other platforms. I think they ended up using ICU by default from .NET 5 on.