> Yes, every time I write df[df.sth = val], a tiny part of me dies.
That's because it's a bad way to use Pandas, even though it is the most popular and often times recommended way. But the thing is, you can just write "safe" immutable Pandas code with method chaining and lambda expressions, resulting in very Polars-like code. For example:
df = (
pd
.read_csv("./file.csv")
.rename(columns={"value":"x"})
.assign(y=lambda d: d["x"] * 2)
.loc[lambda d: d["y"] > 0.5]
)
Plus nowadays with the latest Pandas versions supporting Arrow datatypes, Polars performance improvements over Pandas are considerably less impressive.Column-level name checking would be awesome, but unfortunately no python library supports that, and it will likely never be possible unless some big changes are made in the Python type hint system.
Using `lambda` without care is dangerous because it risks being not vectorized at all. It risks being super slow, operating one row at a time. Is `d` a single row or the entire series or the entire dataframe?
Agreed 100%. I am using this method-chaining style all the time and it works like a charm.
I mean, yes there’s arrow data types, but it’s got a long way to go before it’s got full parity with the numpy version.
I’m not really sure why you think
Is stylistically superior to I agree it comes in handy quite often, but that still doesn’t make it great to write compared to what sql or dplyr offers in terms of choosing columns to filter on (`where y > 0.5`, for sql and `filter(y > 0.5)`, for dplyr)