logoalt Hacker News

XenophileJKOyesterday at 7:19 AM3 repliesview on HN

I'm genuinely curious what you work on that is so "novel" that an LLM doesn't work well on?

I feel like so little is TRUELY novel. Almost everything is built on older concepts and to some degree expertise can be applied or repurposed.


Replies

EagnaIonatyesterday at 7:47 AM

Anything relatively new in a technology LLMs struggle with, especially if the documentation is lacking.

Godot for example in ChatGPT.

It may no longer still be the case, but the documentation for GoDot was lacking and often samples written by others didn't have a version number associated with it. So samples it would suggest would never work, and even when you told it the version number it failed to generate workable code.

The other stuff I've noticed is custom systems. One I work with is a variation of Java, but LLMs were treating it as javascript. I had to create a LoRA just to get the model from not trying to write javascript answer. Even then it could never work, because it had never been trained on real world examples.

geonyesterday at 9:11 AM

It doesn't have to be very novel at all. Anything but the most basic TODO-list app.

aydynyesterday at 8:25 AM

Literally anything in the science domain. Adding features to your software app is indeed usually not novel.

show 1 reply