logoalt Hacker News

hparadizyesterday at 8:43 PM0 repliesview on HN

With media ingestion this is called "eager" processing. Historically for things like pulling thumbnails for images / video and pre-generating common sizes for things. This follows the same pattern and makes all the sense in the world. My only concern is that due to the non deterministic nature of LLMs new models will reveal new information about your data.

For example you might identify a car in an image but the context is the car running a red light. A new model might pick that up while an old one doesn't. These context adjustments might sometimes require you to rerun your LLM processing or potentially have a one to many relationship for multiple runs so you can take the best of or combine results.

Actual usage will also reveal most commonly used assets and you can target the ones that are most trafficked and save a ton on processing that way.