Interesting. I have the same machine w/ same specs. I find it sluggish. I think it depends on what you're doing.
What are you doing with yours? I recently started using Android/iOS emulators locally and combined with anything else my M1 Pro 32 GB feels unusable.
Definitely grabbing 128 GB + for the next machine.
You might want to do a full restore or something, my M1 Max isn't sluggish at all.