alt Hacker NewsWhy are we comparing a programing language and a GPU. This is a category error. Programing languages do not do any operations. They perform no FLOPs, they are the thing the FLOPs are performing.
"The I7-4770K and preform 20k more Flops than C++" is an equally sensible statement (i.e. not)
Replies
the sentence is ambiguous because "Python" can mean python + a certain library and even a different Python implementation
but I find it illuminating to compare what a certain hardware can do in principle (what is possible) vs what I can "reach" as programmer within a certain system/setup
in this case NVIDIA A100 vs "Python" that does not reach a A100 (without the help of CUDA and PyTorch)
another analogy:
I find it useful to be able to compare what the fastest known way is to move a container from A to B using a certain vehicle (e.g. truck) and how that compares to how fast a person that can not drive that truck can do it + variants of it (on foot, using a cargo bike, using a boat via waterway, …)
I'm also interested in how much energy is needed, how much the hw costs and so on
Often there are many ways to do things, comparing is a great starting point for learning more
> Why are we comparing a programing language and a GPU.
You are taking the statement too literally and forgetting it's a figure of speech, specifically metonymy.
When the author says it's millions of flops faster in a gpu than in an interpreteted programming language, it's not comparing them directly, but algorithms that run in them, so the substitution is the algorithms for the tools used to implement/run them.
It makes sense if you say "running similar logic -- like multiplying vectors and matrices -- on the CPU is millions of flops slower then on the GPU". There is no category error there.