so, i've gone through a bunch of iterations on how to interview in days of LLMs and i came up with a process that is quite good:

first, a coding problem that is solved live (right now, online) -- it's not a hard one and most LLMs can easily one shot it.

BUT: if you are not a senior developer, you need to do it without LLM, and if you are a senior+ and do it via LLM, the bar is super raised and now you need to explain in details everything the LLM wrote as if you wrote it yourself.

the main component here is to know if the candidate understands coding in general, understands problem solving and how to dive into an existing (although quite small) codebase. solving the problem is not really the goal -- in fact, most candidates we hired did not solve it 100%, but were able to understand and explain everything that they put it on code. this usually takes 1h30.

(we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).

the second is an architecture interview, where we build a system together BUT instead of doing the whole spec, miro board, whatever, we do it in more of a free way: we start with a really simple, MVP level, spec and we ask how the candidate would build it. then, we add another feature, and we ask how they would go for it. during this whole thing, i ask random questions like "why would you use x instead of y?" or "why do you prefer this approach?".

one important thing: we allow the candidate to say "i don't know" and we tell the candidate that there's no gotcha questions (which is true, i hate these kind of questions). this lets the candidate be super comfortable (feedback we had from real people).

in general, this flow helped us a ton and i really really like running it :)

(edit: spelling)