alt Hacker NewsThere is a KISS way to do this with our current models:
use a time string that updates:
every second (that it is not responding) erase a byte string at the end, then send a new byte string (that represents time from last response): bot either outputs a null terminating token, or if the number gets high enough, instruct/train it to take initiative then.
That only really gets you a limited sense, certainly better than nothing though.
The approach I would like to see would be more expensive token wise, but I don't think that can be avoided for fully interactive real time AI.
Add another dimension to position encoding to allow for multiple simultaneous streams. Run a model on each stream with an additional "I must speak" output value. A simple adjudicator model (maybe just softmax would work) decides which token(s) are emitted, each tagged by the channel that emitted them, which in-turn goes to the additional position encoding dimension. Then reinforcement learn them all at once with attention being allowed to look at all streams.
Then the model is always emitting tokens but while waiting it might just be emitting dum-dee-dum twiddle thumbs, on a channel that doesn't go out to the user.
It needs a way to focus attention though because you need a much larger potential context.
Is there any work on attention being limited to looking at a subset of embeddings defined by a parameterisable function because if a model could emit special tokens to change those parameters, it would potentially be able to scan it's own memory. Could be tricky to train but a training process that reduced the number of locations attended to over time might force it to compensate for the loss in scope by focusing.