alt Hacker NewsHad the pleasure of working with Jason Garrett-Glaser (better known as DarkShikari) circa 2011 on a patch for x264 to better utilize more cores (slice-threaded lookahead). At the time he was working with Sony to optimize their PlayStation game streaming service, and this had the potential to lower encoding latency (critical to game streaming) even on same-core systems by spreading the frame over more cores to do more parallel work, and doing more intelligent scheduling and convergence after that tile-based work was reconverged into a frame. At the time we were using servers with 48 physical cores to do transcoding, a bit of a rarity in a single NUMA space back then and there were no instances to test this on so I flamegraph traced a few encodes at very high thread parallel counts and send back the instrumented profiles (from a custom compiled version) showing the core work being done and he used that to optimize the motion estimation code which worked on the first shot to perfectly level out cpu usage and increase encoding speed across ~24 cores (was previously topping out around 8 and running one core hot iirc). I got a look into how exotic and complex this project was, particularly in terms how low level and advanced things like motion estimation, quantizing and perceptual compression were... it was and still is sort of magic how many smart techniques are baked into something that we barely pay attention to. They were, even at that time, working on x265.
His blog, Diary of an x264 developer, was a great read too for anyone wanting to get a vantage point on how crazy that codebase was.
[1]https://github.com/mstorsjo/x264/blob/master/doc/threads.txt
(edit: was Gaikai that was acquired by Sony in 2012 and is the basis for PlayStation Now)
Replies
Very cool. Took a similar approach to optimizing openutv by looking at flamegraphs. UTV is a fork of openrv which is an older release of the commercial RV. Like many things it uses ffmpeg and also native readers and writers for certain formats. Allowed me to get it to 8k@60fps buttery smooth playback with hardware acceleration. If you are still in the video/imaging space would love your feedback on UTV. openutv.com
Using multiple CPUs for encoding goes way back. The early (1994) C-Cube MPEG-1 and MPEG-2 real-time encoders for DirecTV were built with 8 chips running in parallel (out of necessity since they were only 60 MHz processors) to encode 704x480. Each chip encoded a slice of the image and an auxiliary bus was used to send reconstruction video between chips so that motion vectors could cross chip image boundaries.
Here's a picture of the VME bus development board. It was a two board sandwich to support 10 chips in parallel (only one board pictured).
https://www.w6rz.net/scorpio.jpg
The first real-time HDTV encoders for terrestrial digital television (ATSC) were built by Harris using an IBM encoder chip. It took 6 chips to encode 1920x1080. The IBM chips didn't have an auxiliary bus, so you could see motion vectors limited to chip image boundaries (in this case, 6 tiles arranged like so).