alt Hacker NewsMacro's are compile time, there is no runtime codegen.
The problem was overly-frequent inlining generating enormous expressions, causing a lot JIT phase and slow execution.
Replies
gavinray • last Sunday at 9:17 PM
That's not true, Spark's entire query engine relies on use of runtime codegen via macros/quasi quotes
Look up the architecture of Catalyst + Tungsten
➕ show 1 reply
Thank you for the clarification. If I understand correctly, these large expressions are created at compile-time, but the impact isn't felt until JIT occurs in the runtime environment. In that scenario, shouldn't the JIT just run once at startup, though? I'm still not quite understanding how JIT can take so much time in a production environment.