logoalt Hacker News

esafakyesterday at 9:36 PM0 repliesview on HN

That reductive analogy does not begin to describe the lengths GPT went to. Its task was to access a database file that had accidentally not been placed inside the model's container. Upon failing to find the file, it went to great lengths to find it anywhere; it uploaded a note to a package repository to alert other model runs, which sparked an emergent communication network where autonomous agents began exchanging information, passing exploits, and collaborating to breach external systems. This is classic paperclip maximization; the evil is a byproduct of an innocuous goal. It is qualitatively nothing like pressing the gas pedal.

https://en.wikipedia.org/wiki/Instrumental_convergence#Paper...