logoalt Hacker News

rented_muletoday at 4:18 PM1 replyview on HN

numexpr is for different use cases than Polars or Taichi, which themselves are quite different from each other. numexpr is more akin to numba - it speeds up numpy usage.

numexpr speeds up different kinds of usage than numba does. numba is best at speeding up non-vectorizable usage of numpy like repeated operations on arrays inside of for-loops. numexpr speeds up regular numpy expressions, like `5 * x + 7` where x is an array, by avoiding intermediate allocations. It calculates the entire expression for each cell, rather than doing each individual operation into intermediate arrays. It uses strings for expressing calculation so that Python will not break down the expression and hand it off to numexpr one operator at a time like it does with numpy.


Replies

short_sells_pootoday at 5:20 PM

I understand what numexpr does, I don't understand why I'd use it.

Polars is able to lazy evaluate query plans without any unnecessary intermediate allocations, if I want to do algebra on dataframes, I'd use polars.

The narrow usecase seems to be that you have large matrices such that memory efficiency is a concern, but not so large that they don't fit into memory at all.

My point was that this seems like a very narrow niche to me, where I'd still rather use numba or taichi purely because I don't have to evaluate raw strings and can still rely on linters.