Reland of 30d29618, except actually tested this time.
The VideoToolbox encoder gives I-frames and P-frames a priority of 1,
but the RTMP output code expects I-frames to have priority 3 and
P-frames to have priority 2.
30d29618 changed the priority of all frames that aren't I-frames, but
that included B-frames as well as P-frames. B-frames are given a
priority of 0 by VideoToolbox and changing that priority causes
artifacts for reasons I don't understand. So ignore the B-frames by
ignoring slice packets with priority 0.
Code submissions have continually suffered from formatting
inconsistencies that constantly have to be addressed. Using
clang-format simplifies this by making code formatting more consistent,
and allows automation of the code formatting so that maintainers can
focus more on the code itself instead of code formatting.
In the vt_h264_video_info function, the format of the video to be encoded is
always being set to VIDEO_FORMAT_NV12, dispite the presence of code to set
the video format to VIDEO_FORMAT_I420 or VIDEO_FORMAT_I444. This commit fixes
that function to respect the video format choice of the user.
In addition, whilst testing this fix initially, I also discovered that the
4:4:4 colour format is not supported by the VideoToolbox H264 encoder.
Looking at the VideoToolbox code in ffmpeg as a reference, the ffmpeg code
errors out if a color format other than NV12 or I420 is set. Therefore, this
commit also logs a warning about I444 not being supported, and uses the NV12
default.
The fullrange variable is used to set appropriate video format information in
vt_h264_video_info. Set fullrange first so the video format data is correct in
all cases.
Adds a VideoToolbox based H264 encoder for OSX, which most notably
allows the use of hardware encoding (Quicksync).
NOTES:
- Hardware encoding is handled by Apple itself internally. The plugin
itself has little control over many details due to the way that Apple
designed the VideoToolbox interface. Generally however, quicksync is
used if available on the CPU, and quicksync is almost always available
due to the fact that macs are exclusively Intel.
- The VideoToolbox does not seem to implement CBR, so it won't be
available. These encoders are generally not recommended for
streaming.