obs-ffmpeg: Fix deadlock with nvenc lookahead

Lookahead requires examining frame data over a large number of frames,
so when pkv added the change to fully reset the encoder when the bitrate
changes, nvenc will invalidate all buffers and basically starts over
from a completely clean slate.

It's possible to make lookahead work when changing the bitrate, but due
to how lookahead seems to works internally in nvenc, it will cause
continually increasing latency every time the bitrate is updated, which
is unideal.

Additionally, when lookahead is enabled, deadlocks can occur when
changing the bitrate in a thread other than the graphics thread.
Currently we allow it to be reset outside of the graphics thread.  From
limited investigating, it would appear this deadlock occurs because
nvenc is locking and releasing old textures.

So instead of dealing with all these potential issues, disable the
ability to adjust bitrate when the user has lookahead enabled on nvenc.
It's not really worth implementing dynamic bitrate support when
lookahead is enabled if the latency is just going to continually
increase for every bitrate adjustment anyway.
This commit is contained in:
jp9000 2019-10-04 04:25:05 -07:00
parent 50c9e99b43
commit eabebd1774

View File

@ -449,6 +449,8 @@ static bool init_encoder(struct nvenc_data *enc, obs_data_t *settings)
if (lookahead && nv_get_cap(enc, NV_ENC_CAPS_SUPPORT_LOOKAHEAD)) { if (lookahead && nv_get_cap(enc, NV_ENC_CAPS_SUPPORT_LOOKAHEAD)) {
config->rcParams.lookaheadDepth = 8; config->rcParams.lookaheadDepth = 8;
config->rcParams.enableLookahead = 1; config->rcParams.enableLookahead = 1;
} else {
lookahead = false;
} }
/* psycho aq */ /* psycho aq */
@ -461,7 +463,8 @@ static bool init_encoder(struct nvenc_data *enc, obs_data_t *settings)
/* rate control */ /* rate control */
enc->can_change_bitrate = enc->can_change_bitrate =
nv_get_cap(enc, NV_ENC_CAPS_SUPPORT_DYN_BITRATE_CHANGE); nv_get_cap(enc, NV_ENC_CAPS_SUPPORT_DYN_BITRATE_CHANGE) &&
!lookahead;
config->rcParams.rateControlMode = twopass ? NV_ENC_PARAMS_RC_VBR_HQ config->rcParams.rateControlMode = twopass ? NV_ENC_PARAMS_RC_VBR_HQ
: NV_ENC_PARAMS_RC_VBR; : NV_ENC_PARAMS_RC_VBR;