Ok the next steps beyond predicting a sequence would then be modelling the structure - which requires something like AlfaFold or 'traditional' (as it were) software for molecular mechanics and dynamics.
How useful the predicted sequence is for the predicted structure - and predicted function - no idea! There are already several layers of prediction here.
Sorry, I definitely do not know enough about LLMs to understand the analogy to weights. You can predict the effects of mutations on sequences, I think? As in, if I change this Q to an E will it make the enzyme faster (??). Perhaps you could LLM-generate a plausible member of a family of enzymes with a particular function, given fragments of an existing protein (???).
No idea how the data got in the model. It seems like the study that Xinming and Chen have done was designed to eliminate sources from the general corpus of knowledge - like any old science papers - b choosing obscure and rare sequences that might only be found in a sequence db.