ChatGPT is adding real cartoonists' signatures to fake New Yorker cartoons

The problem is not that ChatGPT is doing that, the problem is that it's not being sued into oblivion after.

Plagiarism as a Service.

We can all try really hard to pretend that's not the business model, but that's totally the business model.

We live in confusing times.

Steal one mp3 and you might get fined thousands, steal a book from your local shoppe and the police would come visit you. Forge a signature and you would also be in trouble. Hack a government website and you will have to answer some questions.

Steal all the books in the world, forge millions and this story begins to tell and nothing happens.

The part of the AI generating the mashup doesn’t really understand what the signature means, or how it may be interpreted, it’s just a visual part of a New Yorker cartoon. Elsewhere in ChatGPT there is plenty of information about signatures and what they mean, and what plagiarism is.

It’s not organized like a human brain, it shouldn’t be surprising that unusual results occur. They are approximating human intelligence from a different angle. It’s interesting to see the improvements in areas like this that require introspection that isn’t fully wired up yet.

[edit] I should add that a human making a New Yorker cartoon is extremely iterative and introspective. Current generative AI is meant to push it out, and you can do the iteration and introspection yourself.

This has been a perennial problem with my own generated comics with both Nano Banana Pro and ChatGPT (all generations). I often have to put in an extra edit to erase the false signature. It is annoying and I'm unsurprised most users don't bother.

I'm certain if I drew a cartoon and used the signature that looks just like a real cartoonists', I would get sued and liable for the damage.

The same should apply to LLM vendors.

The copyright washing machine strikes again.

Engineering manager at my company put a comic at the end of our sprint demo that was signed bloper. Except it wasn’t funny at all, and kind of weird. I asked him, sure enough it was ChatGPT and he didn’t notice the signature.

Of course it does this. The training data is full of examples that associate e.g. "New Yorker-style cartoon" with Loper's signature in the corner, because Loper's signature is in the corner of a lot of them. There's nothing to make it treat the signature as anything special by default; that would have to be trained in explicitly.

One would hope that the person prompting ChatGPT would notice this sort of thing and do something about it before sharing it publicly, out of a genuine desire not to cause confusion etc. But I guess that's way more personal responsibility than we can expect average people to take on nowadays.

Nice. This demolishes the "LLMs can reason" (but not enough to avoid this sort of basic error) and "humans make mistakes too" (not like this) talking points from the LLM promoters.

this is trivial to reproduce ever since early stable diffusion models, it's not really an oai exclusive issue.

for example if you ask any of these models (just about any image-gen) to produce Japanese ukiyo-e art they will almost always produce it with a hanko[0] that has been seen a lot in historical art pieces, usually having nothing to do with the era or style of the replica but seen so often in 'Japanese artwork' that it's just permanently tokenized into it as a defining characteristic.

[0]: https://theartofzen.org/the-hanko-in-japanese-art-and-ukiyo-...

This reminds me my early attempts to use GitHub Copilot when it just straight added some guy's name in a javadoc copyright note in the code it generated.

The entire company is based on stealing everyone else’s creative content and then passing it off as their own. So this isn’t a surprise.

> “It’s like somebody attributed a quote to me that I didn’t say.”

> Katzenstein considers the reproduction of his signature by ChatGPT to be more than just a violation of intellectual property; to him, it’s closer to false impersonation. “[ChatGPT] is attaching my name to work that I do not endorse or like. It’s slop, and unlike the other slop that I’ve encountered, this is slop that’s pretending to be me.”

> “I’ve had people hack my credit card,” said Joe Dator, a New Yorker contributor for the past 20 years. “That feels like less of a violation than this. When they hacked my credit card, they didn’t dress up like me.”

So this has morphed from plagiarism and copyright infringement (bad) to impersonation (also bad, arguably worse, and maybe more provable in court). It’s chilling to think of the implications of having one’s signature attached to a document or to words that are not one’s own.

New Yorker cartoonists could have grounds for a “right of publicity” case, a body of state-based law that protects individuals from the unauthorized use of their name or identity. For that, there would need to be proof that the AI-generated cartoons were used for a commercial purpose, not just as a joke or gag.

I would have thought the fact OpenAI is commercially selling these forgeries (in exchange for subscription payments) would make that fairly straightforward to prove.

signatures in, signatures out

It seems like all the criticisms of Gen AI and LLM seems to concentrate on OpenAI and their products over products from anthropic and others. I am pretty sure that this faking of signature can be done by Gemini, claude as easily as chatgpt.

This should be illegal.

Wait, isn't it already illegal to forge someone's signature?

Whatever the original intention, this is clearly a bug and should be fixed. But should ChatGPT sign its cartoons with its own name or leave them unsigned?

Makes sense. Pretty much every example it is trained on has a signature.

Not putting signatures in art unless otherwise requested should become a default in AI instances.

I've seen photorealistic generations add (distorted but still somewhat recognisable) watermarks too, because that's what the training data had.

Everything is a derivative work, and always has been. AI is just making that salient fact so much more visible, and now everyone who believes in the delusion of Imaginary Property is scared at that truth revealing itself.

Incidentally, this is also what young humans learning to draw will do. They start by copying what they've seen.

I follow anti-LLM discourse quite a lot, and across the main bulletpoints: energy/carbon emissions, content worker harm, job displacement, deskilling, mental health effects, and copyright/plagiarism, the plagiarism one seems to have the most attention, and it's also the most solvable, if there were only more serious effort on ethically sourced models that can actually do the real science / math / code work that is what LLMs are best at. The whole world of LLMs to create videos/books/art/literature is where most of the offense is (the video/imagery side of it is where most of the energy/carbon emissions problems are too. and content worker harm).

I really wish there'd be a split among these disciplines (science/math/code vs. videos/art/literature) - one is vastly more problematic than the other.

AI companies have created the best pirating tool known to man. And the president is complicit with senate / house lawmakers in his party refusing to take action to reign them in.

They should obviously train it to avoid this like they train it not to output verbatim passages from books and whatnot.

Hey anybody remember what happened to Aaron Swartz?

This is why I laugh when the companies cry about model distillation "attacks".

Christ, what an asshole

This is a pretty solid argument against people who argue that LLMs are more than just (very massive) next token predictors.

If there was any thought or underlying thought going on here not putting a signature (at least a real one) would be the right move, despite it being less likely. It would realize, while generating the pixels that eventually became a signature, that it shouldn't do that.

Man, forget the debate about AI being dangerous. The real reason all these companies need to be shut down is copy right infringement. None of this is "fair use." Does fair use imply doing things that actively harm the creators of works? Because training models to be able to replicate works only makes their skills less valuable...

Honestly, you all kind of mind fuck me that you're not more pissed off about AI basically replicating a large portion of your skills. This effects so many professions now and it's only going to get worse. I would expect a far greater outcry from software engineers trying to organise to ban this shit. But its like none of you even care?

[flagged]

[dead]

[dead]

[flagged]

[flagged]

Let's see the prompts, without that this discussion is meaningless.

I have a screenshot from 2024 when I asked ChatGPT to write me a draft of a story about a young boy who lost his parents and got send to a wizardry school. I specifically asked for the story to be original and not based on anything that has been already published.

As you probably suspect, chat gave me a full synopsis of Harry Potter.

I kept asking if that's an original idea, and it kept swearing on it's mothers grave, that the story has never been published before.