Why use AI when duplicate finders have been a thing for decades?
For the pictures, I set up a batch script the one time that used exiftool to read all of my photo's exif data (for creation date), rename the file to that date / time, and put it in a year/month folder structure. Once that's normalized it was pretty straightforward to identify and delete duplicates (assuming the exif data on duplicates has the same date/time in it).
You have kids? /s
Duplicates are easy, that gets sorted quite fast. Picking 1-2 out of those 5 slightly different pictures of the same occasion is another task.