logoalt Hacker News

perrygeotoday at 1:58 PM0 repliesview on HN

For me, the superpower of polars is production stability.

Pandas tends to push all problems to runtime, with all sorts of hidden heuristics. Particularly around column types and missing values. It's very hard to know if you've tested all the edge cases. The only way to test your code is to throw all variations of data at it. Fine if you're sitting at a notebook and have the patience to validate and "clean" the data on its behalf. Not so fine if you get paged at 3am because your data pipeline failed when it expected an int column but got float.

Polars is more strict by default and front-loads costs through its planner. The resulting apps are noticeably more stable in production. You can test code and reasonable assurance that it will work on data in the wild.

I don't really have any interest in the API ergonomics or syntax - both are fine. It's all about how they deal with data variation at runtime. Can you write general code that doesn't break on variants? Pandas, not a chance. Polars, absolutely!

Bonus round: polars has a Rust API too, the compiler can effectively prove that your program handles every edge case. It's common to write rust polars apps that run unattended for years.