URAvatar: Universal Relightable Gaussian Codec Avatars

Seems like this would (eventually) be big for VR applications. Especially if the avatar could be animated using sensors installed on the headset so that the expressions match the headset user. Reminds me of the metaverse demo with Zuckerberg and Lex Friedman

Interesting that under the "URAvatar from Phone Scan" section, the first example shows a lady with blush/flush, which only appears in the center video when viewed straight on - the other angles remove this

Given the complete lack of any actual details about performance I would hazard a guess that this approach is likely barely realtime, requiring top hardware, and/or delivering an unimpressive fps. I would love to get more details though.

Those demo videos look great! Does anyone know how this compares to the state of the art in generating realistic, relightable models of things more broadly? For example, for video game assets?

I'm aware of traditional techniques like photogrammetry - which is neat, but the lighting always looks a bit off to me.

With the computational efficiency of Gaussian splatters, this could be ground-breaking for photorealistic avatars, possible driven by LLMs and generative audio.

Wow that looks pretty much solved! Is there code?

This is great work, although I note that the longer you look at them, and the more examples you look at in the page, the wow factor drops off a bit. The first example is exceptional, but when you get down to the video of "More from Phone Scan" and look at any individual avatar, you find yourself deep in the uncanny valley very quickly

ABEL

Who will use this? Handing over your photo information so someone can impersonate you in video call to trick your family or friends?