That's a counterargument to a different thing.

Iteratively measuring loss is a way to reconstruct values. That's trivial to show for a single value If 5 gives you a loss of 2 and 9 gives you a loss of 2 then you know the missing value is 7.

A model with enough parameters can memorise the training set in a similar manner. Technically the model hasn't seen that data by direct input either, but the mechanism provides the means to determine the what the data was. In that respect it is reasonable to say the model has seen the data.

Performing well on examples not in the training set is doing something else.

Any attempt to characterise that as having been seen before negates any distinction between taking in data and reasoning about that data.