alt Hacker NewsHow does this have value? Is any and every sentence in an e-mail considered 'fact' and thus to be fed into the AI?
90% of e-mails and Teams communications are inane. Polite banter, "thanks for taking care of that, I appreciate it" "please route the forms to Janet this week because Bill is on vacation" "unit will be un available until the parts come in" . I can't see the intrinsic fact value of this kind of communication without screening it. And after screening, the gold nuggets would be minimal.
Replies
> How does this have value? Is any and every sentence in an e-mail considered 'fact' and thus to be fed into the AI?
I have a hunch what this is for. AI companies want to make bigger inroads into nontechnical work settings. But LLM progress outside of fields where verifiable rewards for RL post-training can be synthetically generated (coding, math) has been pretty flat. Buying years of operational data from a company like an airline could be used to reconstruct long-horizon task trajectories in areas like customer service or marketing.
> Is any and every sentence in an e-mail considered 'fact' and thus to be fed into the AI?
If it comes with the context, yes. More data the better. Someone considered it served some purpose at some point. Thus it contains, no matter how tiny, a sliver of information.
A major selling point of AI chat bots is for customer service automation. A clean dataset like this is a huge find. I'm not sure what you mean by "facts" or "gold nuggets". Training data doesn't need to be factual.
It will have a different writing style from the average blog post. Maybe they just want to train an AI that sounds less like an AI. Or one that speaks in vapid management style.
This data shows exactly how a huge company of thousands of employees works and coordinates.
The perfect data to train an agent swarm on how to run a company.
Maybe it's innefficient and inane, but it's how you start.
The first LLMs, GPT-1, 2, were trained on complete garbage, the average document from the common crawl is random non-sense, yet they worked, and now we can use LLMs to filter the data for the next training run.
LLMs aren't a database. They're an attempt at brute-forcing an artificial mind. The who and what aren't really interesting there, it'll forget most of such details anyway. What matters is the patterns visible in the text at various scales. How people write. Why they write. To whom they write, in response to what. How does e-mails about mistakes correlate with PDFs they're referring to. How people work with ticketing systems - like how, actually, a ticket plays out. The jargon, the acronyms, the vibes, the causal links. It's all in there, and it's another slice through the set of things humans do, to be combined with other slices already in the training data, and enriching the whole.
(Something something we will add your distinctiveness to our own, you will be assimilated, ...)
(Hell, the fact that it's all from one org would make it a great dataset to have in the open for sociological studies. I bet that today, aided by LLMs to sift through it, you could use it to map how information flows through a large org - how incident on the floor travels through time and layers of management until it reaches the C-suite, what of it survives, how it gets reacted to, how the reactions flow down...)