That's fair, though even that could just be a matter of time, as people build tools that interface LLMs to the physical world. I wonder how something like Bus Pirate could be used with an LLM (maybe a more powerful version of it?) to grok and poke hardware all over the place.
I forsee issues with really getting use out of any commodity language model in the hardware security context, because hardware systems notoriously lack standardization. And often times, the technical knowledge (datasheets, app notes) is locked behind vendor NDAs, or straight up not documented, only existing in the minds of engineers. The implementations of said designs are similarly highly proprietary, with little public "real" systems to train models on.
So the issue is two-fold:
* The knowledge must be documented and accessible for training.
* A bespoke model must be trained this documentation.
It is unlikely that both of these things happen in the general model context. Perhaps individual chip vendors will eventually pursue this, but I suspect it is just not a priority for them.