alt Hacker NewsGaussian Splatting – A$AP Rocky "Helicopter" music video
Comments
Hi,
I'm David Rhodes, Co-founder of CG Nomads, developer of GSOPs (Gaussian Splatting Operators) for SideFX Houdini. GSOPs was used in combination with OTOY OctaneRender to produce this music video.
If you're interested in the technology and its capabilities, learn more at https://www.cgnomads.com/ or AMA.
Try GSOPs yourself: https://github.com/cgnomads/GSOPs (example content included).
Never did I think I would ever see anything close to related to A$AP on HN. I love this place.
Super cool to read but can someone eli5 what Gaussian splatting is (and/or radiance fields?) specifically to how the article talks about it finally being "mature enough"? What's changed that this is now possible?
Really amazing video. Unfortunately this article is like 60% over my head. Regardless, I actually love reading jargon-filled statements like this that are totally normal to the initiated but are completely inscrutable to outsiders.
"That data was then brought into Houdini, where the post production team used CG Nomads GSOPs for manipulation and sequencing, and OTOY’s OctaneRender for final rendering. Thanks to this combination, the production team was also able to relight the splats."Hello! I’m Chris Rutledge, the post EP / cg supervisor at Grin Machine. Happy to answer any questions. Glad people are enjoying this video, was so fun to get to play with this technique and help break it into some mainstream production
To be honest it looks like it was rendered in an old version of Unreal Engine. That may be an intentional choice - I wonder how realistic guassian splatting can look? Can you redo lights, shadows, remove or move parts of the scene, while preserving the original fidelity and realism?
The way TV/movie production is going (record 100s of hours of footage from multiple angles and edit it all in post) I wonder if this is the end state. Gaussian splatting for the humans and green screens for the rest?
Tangential, but I've been exploring gaussian splatting as a photographic/artistic medium for a while, and love the expressionistic quality of the model output when deprived of data.
Direct link to the music video: https://www.youtube.com/watch?v=g1-46Nu3HxQ
Be sure to watch the video itself* - it’s really a great piece of work. The energy is frenetic and it’s got this beautiful balance of surrealism from the effects and groundedness from the human performances.
* (Mute it if you don’t like the music, just like the rest of us will if you complain about the music)
It’s interesting to see Gaussian splatting show up in a mainstream music video this quickly. A year ago it was mostly a research demo, and now artists are using it as part of their visual toolkit. What I find most notable is how well splats handle chaotic motion like helicopter shots — it’s one of the few 3D reconstruction methods that doesn’t completely fall apart with fast movement. Feels like we’re going to see a lot more of this in creative work before it shows up in anything “serious”.
Too bad, but I managed to watch about 30 seconds of the video before getting motion sickness.
Seems like a really cool technology, though.
I wonder if anyone else got the same response, or it's just me.
The end result is really interesting. As others have pointed out, it looks sort of like it was rendered by an early 2000s game engine. There’s a cohesiveness to the art direction that you just can’t get from green screens and the like. In service of some of the worst music made by human brains, but still really cool tech.
The rap seems rather derivative, lyrics contain mandatory token "pussy".
I was wondering how these kind of scenes were done in the movie Enter The Void back in 2009. Maybe they used different technique but looks very similar: https://vimeo.com/60239415
Dang, it's been cool watching gaussian splats go from tech demo to real workflow.
I can't really respect the artist though, after the assault on a random bystander in Stockholm in 2019 — for which he was convicted. He got off too easy.
I would have refused to work on this.
You could also pull a Michel Gondry and do it with practical effects. https://www.youtube.com/watch?v=s5FyfQDO5g0&list=RDs5FyfQDO5...
They really said it’s capturing everything when A$AP Rocky’s Gaussian splatted mouth in that video be looking worse than AI generated video lol
Both of my worlds are colliding with this article. I love reading about how deeply technical products/artifacts get used in art.
This reminds me about how Soulja Boy just used a cracked copy of Fruity Loops and a cheap microphone and recorded all his songs that made him millions.[1] Edit: Ok this was a big team of VFX producers who did this. Still, prices are coming down dramatically in general, but yeah that idea is a bit of an underfit to this case.
Can somebody explain to me what was actually scanned? Only the actors doing movements like push ups, or whole scenes / rooms?
In another setting, it looks like ass, but lo-fi, glitchy shit is perfectly compatible with hip-hop aesthetic. Good track though.
A$AP Rocky’s music videos has been always great.
It would be super cool if these articles or websites actually explained anywhere what volumetric capture and radiance fields actually are!
A$ap Rocky's music videos have some really good examples of how AI can be used creatively and not just to generate slop. My favorite is Taylor Swif, it's a super fun video to watch.
The splatting seems to be video only but I could be wrong.
It's only a matter of time until the #1 hit figures out how to make this work
"The team also used Blender heavily for layout and previs, converting splat sequences into lightweight proxy caches for scene planning."
The texture of Gaussian Splatting always looks off to me. It looks like the entire scene has been textured or has a bad, uniform film grain filter to me. Everything looks a little off in an unpleasing way -- things that should be sharp are aren't, and things that should be blurry are not. It's uncanny valley and not in a good way. I don't get what all the rage is about it and it always looks like really poor B-roll to me.
Good shit
How did Rhianna look him in the eyes and say "yes babe, good album, release it, this is what the people wanted after 7 years, it is pleasing to listen to and enjoyable"?
Wow, back to music videos. It’s been years. This is a great one.
so basically despite the higher resource requirements like 10TB of data for 30 minutes of footage, the compositing is so much faster and more flexible and those resources can be deleted or moved to long term storage in the cloud very quickly and the project can move on
fascinating
I wouldn't have normally read this and watched the video, but my Claude sessions were already executing a plan
the tl;dr is that all the actors were scanned in a 3D point cloud system and then "NeRF"'d which means to extrapolate any missing data about their transposed 3D model
this was then more easily placed into the video than trying to compose and place 2D actors layer by layer
Maybe I'm getting old, but I can't watch for more than about 5 seconds without getting a headache.
> One recurring reaction to the video has been confusion. Viewers assume the imagery is AI-generated. According to Evercoast, that couldn’t be further from the truth. Every stunt, every swing, every fall was physically performed and captured in real space. What makes it feel synthetic is the freedom volumetric capture affords.
No, it’s simply the framerate.
Pretty sure most of this could be filmed with a camera drone and preprogrammed flight path...
Did the Gaussian splatting actually make it any cheaper? Especially considering that it needed 50+ fixed camera angles to splat properly, and extensive post-processing work both computationally and human labour, a camera drone just seems easier.
I want to shoutout Nial Ashley (aka Llainwire) for doing this in 2023 as a solo act and doing the visuals himself as well - https://www.youtube.com/watch?v=M1ZXg5wVoUU
A shame that kid was slept on. Allegedly (according to discord) he abandoned this because so many artists reached out to have him do this style of mv, instead of wanting to collaborate on music.