@blakeblackshear, My facial recognition component, relies on a caffe model run by opencv for face detection and in spite of running this pixel conversion from YUV to BGR, the full 3MP conversion is 2-3x more efficient than using FFMpeg directly. I really am not sure why.
I just evaluated the difference in wattage consumption per real-time stream of the video capture only using the GPU on my setup.
OpenCV: 3W//3MP 20fps stream
Ffmpeg: 10W/1MP 20fps stream, 25W/3MP.
The additional load is on the CPU. The additional GPU decoder load is about the same.
video+processing:
Video processing Face detection+face recognition, 3MP 20fps using openCV image processing+ caffe DNN model+Dlib encoding+ SciKit linear classifier: 8W/stream
Watsor person detection, 1MP(downscaled). 20fps: 20W/stream. At full 3MP resolution the consumption goes to ~50W/stream.