Allows a frontend the ability to set the maximum audio buffering
latency, and specify whether that audio buffering is either fixed (to
the maximum audio buffering latency), or dynamically increasing from 0.
This will be useful if the user wishes to output audio to devices or
through a virtual audio device at a guaranteed minimal latency.
When audio buffering is maxed, certain sources will always repeat the
same debug logging message of "render audio source [name] has gone
backwards", which shouldn't apply if the audio timestamp is 0 rather
than a valid audio value.
This is in anticipation to adding low audio latency mode, will cause
audio buffering to be fixed rather than dynamically increasing. Having
fixed audio buffering means that audio latency always stays the same
rather than dynamically increasing.
df4eb82 fixed a bug that caused source audio timestamps to perpetually
lag. However, there is a deeper issue where after we reach max
buffering, lagging sources make OBS's entire audio pipeline fall over.
These may be corrected by later code, but still cause global audio
glitches at best. Persistent problems, as prior to df4eb82, cause audio
to fail entirely.
The root cause is that OBS's audio mixing tree cannot deal with
timestamps prior to the current audio tick. Intermediate mixing stages
assume that the lowest incoming timestamp is the base of the current
tick, and mix accordingly. This propagates lagged timestamps up the
tree, where at the top level mix_audio will drop the source entirely -
which at this point is a transition covering all inputs, thus glitching
audio globally. Where extra buffering can cover the slip, the entire mix
gets retried and the error corrected, but when the global buffer
duration is maxed out, it makes it to the output.
The solution is to catch laggy sources immediately after rendering, and
drop audio to bring them back in sync, or mark them pending if not
enough audio is available. This ensures later mixing stages are not fed
with out of sync timestamps.
This improves the ignore_audio code to only drop as much audio as
needed to bring the source back in sync, and moves its call to
immediately after source audio rendering.
Regression introduced by dc4e20500: while the stop detection is pending,
it should still return false so the rest of the discard code can run.
Otherwise, the source audio will remain in the buffer, lagging the
source and triggering audio buffering increases until max audio
buffering is reached.
As os_gettime_ns() gets large the current scaling methods, mostly by casting
to uint64_t, may lead to numerical overflows. Sweep the code and use
util_mul_div64() where applicable.
Signed-off-by: Hans Petter Selasky <hps@selasky.org>
If an audio source does not provide enough data at a steady pace, the
timestamp update does not happen, and buffering increases until it
maxes out. To counteract this, update the timestamp anyway.
Another issue for decoupled audio sources is that timing is not
adjusted for divergence from system time. Making this adjustment is
better for timing stability.
5+ hours of stable audio without any buffering on my GV-USB2 where it
used to add 21ms every 5 mintues or so.
Fixes https://obsproject.com/mantis/view.php?id=1269
Code submissions have continually suffered from formatting
inconsistencies that constantly have to be addressed. Using
clang-format simplifies this by making code formatting more consistent,
and allows automation of the code formatting so that maintainers can
focus more on the code itself instead of code formatting.
Uses obs_source_get_ref on the sources enumerated in the tick_sources
function in obs-video.c to ensure a reference has been incremented
before calling that source's video_tick, and replaces an
obs_source_addref with obs_source_get_ref in the push_audio_tree
function in obs-audio.c to ensure that it cannot increment a source that
has already decremented its reference to 0.
The default buffering time for audio was always 1 second before the
audio subsystem was changed, and it was always more than sufficient for
max audio buffering time
This code causes audio data in general to be reset (and subsequently
deleted). It should just be marked as pending and ignored until the
data is ready. The discard_if_stopped function will serve the same
purpose if the source's audio has actually stopped.
This variable is used to detect whether audio has stopped -- if audio
stops, it detects that no new data is coming in, and resets the audio
position so that it eliminates the chance of causing the audio buffering
to go haywire if audio starts up again. However, this variable was not
being reset every time the value changes, which it should.
If the circular audio buffer of the source has data remaining that's
less than the audio frame tick count (1024 frames), it would just leave
that audio data on the source without discarding it. However, this
could cause audio buffering to increase unnecessarily under certain
circumstances (when the next audio timestamp is within the timestamp
jump window), so it would append data to that circular buffer despite
the audio stopping that long ago, causing audio buffering to have to
increase to compensate.
Instead, just discard pending audio if it hasn't been written to. In
other words, if the audio has stopped and there's insufficient audio
left to continue processing.
Fixes an issue where audio data would not be popped if they were not
activated/presenting. This would cause the audio subsystem to
needlessly buffer when they were reactivated again. Rendering all audio
sources (excuding composite/filter sources) helps ensure that audio data
is always popped and not left to pile up.
The new audio subsystem fixes two issues:
- First Primary issue it fixes is the ability for parent sources to
intercept the audio of child sources, and do custom processing on
them. The main reason for this was the ability to do custom
cross-fading in transitions, but it's also useful for things such as
side-chain effects, applying audio effects to entire scenes, applying
scene-specific audio filters on sub-sources, and other such
possibilities.
- The secondary issue that needed fixing was audio buffering.
Previously, audio buffering was always a fixed buffer size, so it
would always have exactly a certain number of milliseconds of audio
buffering (and thus output delay). Instead, it now dynamically
increases audio buffering only as necessary, minimizing output delay,
and removing the need for users to have to worry about an audio
buffering setting.
The new design makes it so that audio from the leaves of the scene graph
flow to the root nodes, and can be intercepted by parent sources. Each
audio source handles its own buffering, and each audio tick a specific
number of audio frames are popped from the front of the circular buffer
on each audio source. Composite sources (such as scenes) can access the
audio for child sources and do custom processing or mixing on that
audio. Composite sources use the audio_render callback of sources to do
synchronous or deferred audio processing per audio tick. Things like
scenes now mix audio from their sub-sources.
Norihiro Kamae
tt2468
jp9000
jpark37
Hector Martin
Hans Petter Selasky
James Park
pkviet
Ryan Foster