This argument would be difficult to reconcile if we didn't have billions of examples of humans forming memories plus modern evolutionary biology to trivially demonstrate such cases are rare pathologies but still human beings.