(Note: This commit breaks libobs compilation. Skip if bisecting)
This variable is somewhat redundant. Volume is already known/accessible
to front-ends.
(Note: This commit breaks libobs compilation. Skip if bisecting)
Uses a callback and allows the caller to mix audio. Additionally,
allows the callback to return audio later, allowing it to buffer as much
as it needs.
This improves logging for when audio data insertion is way out of bounds
or is getting cut off in the front due to a bad negative sync offset.
Instead of throwing out a log message for every time this happens with
each piece of data, it now states when the out of bounds or cutoff has
started and stopped only.
This fixes a case where an insertion of audio data would pass
valid_timestamp_range yet the insert position would cause a negative
integer position and thus an unsigned integer overflow.
API changed:
--------------------------
void obs_output_set_audio_encoder(
obs_output_t *output,
obs_encoder_t *encoder);
obs_encoder_t *obs_output_get_audio_encoder(
const obs_output_t *output);
obs_encoder_t *obs_audio_encoder_create(
const char *id,
const char *name,
obs_data_t *settings);
Changed to:
--------------------------
/* 'idx' specifies the track index of the output */
void obs_output_set_audio_encoder(
obs_output_t *output,
obs_encoder_t *encoder,
size_t idx);
/* 'idx' specifies the track index of the output */
obs_encoder_t *obs_output_get_audio_encoder(
const obs_output_t *output,
size_t idx);
/* 'mixer_idx' specifies the mixer index to capture audio from */
obs_encoder_t *obs_audio_encoder_create(
const char *id,
const char *name,
obs_data_t *settings,
size_t mixer_idx);
Overview
--------------------------
This feature allows multiple audio mixers to be used at a time. This
capability was able to be added with surprisingly very little extra
overhead. Audio will not be mixed unless it's assigned to a specific
mixer, and mixers will not mix unless they have an active mix
connection.
Mostly this will be useful for being able to separate out specific audio
for recording versus streaming, but will also be useful for certain
streaming services that support multiple audio streams via RTMP.
I didn't want to use a variable amount of mixers due to the desire to
reduce heap allocations, so currently I set the limit to 4 simultaneous
mixers; this number can be increased later if needed, but honestly I
feel like it's just the right number to use.
Sources:
Sources can now specify which audio mixers their audio is mixed to; this
can be a single mixer or multiple mixers at a time. The
obs_source_set_audio_mixers function sets the audio mixer which an audio
source applies to. For example, 0xF would mean that the source applies
to all four mixers.
Audio Encoders:
Audio encoders now must specify which specific audio mixer they use when
they encode audio data.
Outputs:
Outputs that use encoders can now support multiple audio tracks at once
if they have the OBS_OUTPUT_MULTI_TRACK capability flag set. This is
mostly only useful for certain types of RTMP transmissions, though may
be useful for file formats that support multiple audio tracks as well
later on.
This Fixes a minor flaw with the API where data had to always be mutable
to be usable by the API.
Functions that do not modify the fundamental underlying data of a
structure should be marked as constant, both for safety and to signify
that the parameter is input only and will not be modified by the
function using it.
Typedef pointers are unsafe. If you do:
typedef struct bla *bla_t;
then you cannot use it as a constant, such as: const bla_t, because
that constant will be to the pointer itself rather than to the
underlying data. I admit this was a fundamental mistake that must
be corrected.
All typedefs that were pointer types will now have their pointers
removed from the type itself, and the pointers will be used when they
are actually used as variables/parameters/returns instead.
This does not break ABI though, which is pretty nice.
Audio that goes below the minimum expecting timing (current time -
buffering time) is automatically removed. However, delayed audio is not
removed regardless of its delay. This puts a hard cap of 6 seconds from
current time that the maximum delay audio can have. This will also
prevent the circular buffer from dynamically growing too large.
Doing timestamp smoothing in obs-source.c is good because timestamps can
typically operate on a different timebase, however, obs-source.c can
also change that time base dynamically (such as with async video and
unexpected timestamp jumps), so in order to ensure that audio is
seamless in the output as well, perform timestamp smoothing in
audio-io.c as well just as an extra precautionary measure.
- obs-outputs module: Add preliminary code to send out data, and add
an FLV muxer. This time we don't really need to build the packets
ourselves, we can just use the FLV muxer and send it directly to
RTMP_Write and it should automatically parse the entire stream for us
without us having to do much manual code at all. We'll see how it
goes.
- libobs: Add AVC NAL packet parsing code
- libobs/media-io: Add quick helper functions for audio/video to get
the width/height/fps/samplerate/etc rather than having to query the
info structures each time.
- libobs (obs-output.c): Change 'connect' signal to 'start' and 'stop'
signals. 'start' now specifies an error code rather than whether it
simply failed, that way the client can actually know *why* a failure
occurred. Added those error codes to obs-defs.h.
- libobs: Add a few functions to duplicate/free encoder packets
- Implement OBS encoder interface. It was previously incomplete, but
now is reaching some level of completion, though probably should
still be considered preliminary.
I had originally implemented it so that encoders only have a 'reset'
function to reset their parameters, but I felt that having both a
'start' and 'stop' function would be useful.
Encoders are now assigned to a specific video/audio media output each
rather than implicitely assigned to the main obs video/audio
contexts. This allows separate encoder contexts that aren't
necessarily assigned to the main video/audio context (which is useful
for things such as recording specific sources). Will probably have
to do this for regular obs outputs as well.
When creating an encoder, you must now explicitely state whether that
encoder is an audio or video encoder.
Audio and video can optionally be automatically converted depending
on what the encoder specifies.
When something 'attaches' to an encoder, the first attachment starts
the encoder, and the encoder automatically attaches to the media
output context associated with it. Subsequent attachments won't have
the same effect, they will just start receiving the same encoder data
when the next keyframe plays (along with SEI if any). When detaching
from the encoder, the last detachment will fully stop the encoder and
detach the encoder from the media output context associated with the
encoder.
SEI must actually be exported separately; because new encoder
attachments may not always be at the beginning of the stream, the
first keyframe they get must have that SEI data in it. If the
encoder has SEI data, it needs only add one small function to simply
query that SEI data, and then that data will be handled automatically
by libobs for all subsequent encoder attachments.
- Implement x264 encoder plugin, move x264 files to separate plugin to
separate necessary dependencies.
- Change video/audio frame output structures to not use const
qualifiers to prevent issues with non-const function usage elsewhere.
This was an issue when writing the x264 encoder, as the x264 encoder
expects non-const frame data.
Change stagesurf_map to return a non-const data type to prevent this
as well.
- Change full range parameter of video scaler to be an enum rather than
boolean
...The reason why audio didn't work was because I overwrote the bitrate
values.
As for semaphores, mac doesn't support unnamed semaphores without using
mach semaphores. So, I just implemented a semaphore wrapper for each
OS.
- Fix a bug where the initial audio data insertion would cause all
audio data to unintentionally clear (mixed up < and > operators, damn
human error)
- Fixed a potential interdependant lock scenario with channel mutex
locks and graphics mutex locks. The main video thread could lock the
graphics mutex and then while in the graphics mutex could lock the
channels mutex. Meanwhile in another thread, the channel mutex could
get locked, and then the graphics mutex would get locked, causing a
deadlock.
The best way to deal with this is to not let mutexes lock within
other mutexes, but sometimes it's difficult to avoid such as in the
main video thread.
- Audio devices should now be functional, and the devices in the audio
settings can now be changed as desired.
LOG_ERROR should be used in places where though recoverable (or at least
something that can be handled safely), was unexpected, and may affect
the user/application.
LOG_WARNING should be used in places where it's not entirely unexpected,
is recoverable, and doesn't really affect the user/application.
- Add CoreAudio device input capture for mac audio capturing. The code
should cover just about everything for capturing mac input device
audio. Because of the way mac audio is designed, users may have no
choice but to obtain the open source soundflower software to capture
their mac's desktop audio. It may be necessary for us to distribute
it with the program as well.
- Hide event backend
- Use win32 events for windows
- Allow timed waits for events
- Fix a few warnings
Implement a few audio options in to the user interface as well as a few
inline audio functions in audio-io.h.
Make it so ffmpeg plugin automatically converts to the desired format.
Use regular interleaved float internally for audio instead of planar
float.
This allows the changing of bideo settings without having to completely
reset all graphics data. Will recreate internal output/conversion
buffers and such and reset the main preview.
Make it so obs_data settings input in to *_update are applied to the
existing settings rather than fully replace the existing settings. That
way you can update with only certain specific settings, leaving other
settings untouched. Of course if you're already using the original
settings pointer in the first place then you've already done that, so
it'll just ignore it because you've already applied them.
Add a scaler interface (defaults to swscale), and if a separate output
wants to use a different scale or format than the default output format,
allow a scaler instance to be created automatically for that output,
which will then receive the new scaled output.
If there are for example more than one audio outputs and they have
different sample rates or channels and such, this will allow automatic
conversion of that audio to the request formats/channels/rates (but only
if requested).
There were a *lot* of warnings, managed to remove most of them.
Also, put warning flags before C_FLAGS and CXX_FLAGS, rather than after,
as -Wall -Wextra was overwriting flags that came before it.
- Fill in the rest of the FFmpeg test output code for testing so it
actually properly outputs data.
- Improve the main video subsystem to be a bit more optimal and
automatically output I420 or NV12 if needed.
- Fix audio subsystem insertation and byte calculation. Now it will
seamlessly insert new audio data in to the audio stream based upon
its timestamp value. (Be extremely cautious when using floating
point calculations for important things like this, and always round
your values and check your values)
- Use 32 byte alignment in case of future optimizations and export a
function to get the current alignment.
- Make os_sleepto_ns return true if slept, false if the time has
already been passed before the call.
- Fix sinewave output so that it actually properly calculates a middle
C sinewave.
- Change the use of row_bytes to linesize (also makes it a bit more
consistent with FFmpeg's naming as well)
- Add planar audio support. FFmpeg and libav use planar audio for many
encoders, so it was somewhat necessary to add support in libobs
itself.
- Improve/adjust FFmpeg test output plugin. The exports were somewhat
messed up (making me rethink how exports should be done). Not yet
functional; it handles video properly, but it still does not handle
audio properly.
- Improve planar video code. The planar video code was not properly
accounting for row sizes for each plane. Specifying row sizes for
each plane has now been added. This will also make it more compatible
with FFmpeg/libav.
- Fixed a bug where callbacks wouldn't create properly in audio-io and
video-io code.
- Implement 'blogva' function to allow for va_list usage with libobs
logging.
- Added some code for FFmpeg output that I'm still playing around with.
Right now I'm just trying to get it to output to file and try to
understand the FFmpeg/libav APIs. Hopefully in the future this plugin
can be used for any sort of output to FFmpeg.
- Fixed a cast warning in audio-io.c with size_t -> uint32_t
- Renamed the 'video_info' and 'audio_info' structures to
'video_conver_info' and 'audio_convert_info' to better represent their
actual purpose, and to avoid confusion with 'audio_output_info' and
'video_output_info' structures.
- Removed a few macros from obs-def.h that were at one point going to be
used but no longer going to be used (at least for now)
Completely revamped the entire media i/o data and handlers. The
original idea was to have a system that would have connecting media
inputs and outputs, but at a certain point I realized that this was an
unnecessary complexity for what we wanted to do. (Also, it reminded me
of directshow filters, and I HATE directshow with a passion, and
wouldn't wish it upon my greatest enemy)
Now, audio/video outputs are connected to directly, with better callback
handlers, and will eventually have the ability to automatically handle
conversions such as 4:4:4 to 4:2:0 when connecting to an input that uses
them. Doing this will allow the video/audio i/o handlers to also
prevent duplicate conversion, as well as make it easier/simple to use.
My true goal for this is to make output and encoder plugins as simple to
create as possible. I want to be able to be able to create an output
plugin with almost no real hassle of having to worry about image
conversions, media inputs/outputs, etc. A plugin developer shouldn't
have to handle that sort of stuff when he/she doesn't really need to.
Plugins will be able to simply create a callback via obs_video() and/or
obs_audio(), and they will automatically receive the audio/video data in
the formats requested via a simple callback, without needing to do
almost anything else at all.
- Add preliminary (yet to be tested) handling of timestamp invalidation
issues that can happen with specific devices, where timestamps can
reset or go backward/forward in time with no rhyme or reason. Spent
the entire day just trying to figure out the best way to handle this.
If both audio and video are present, it will increment a reference
counter if video timestamps invalidate, and decrement the reference
counter when the audio timestamps invalidate. When the reference
counter is not 0, it will not send audio as the audio will have
invalid timing. What this does is it ensures audio data will never go
out of bounds in relation to the video, and waits for both audio and
video timestamps to "jump" together before resuming audio.
- Moved async video frame timing adjustment code into
obs_source_getframe instead so it's automatically handled whenever
called.
- Removed the 'audio wait buffer' as it was an unnecessary complexity
that could have had problems in the future. Instead, audio will not
be added until video starts for sources that have both async
audio/video. Audio could have buffered for too long of a time anyway,
who knows what devices are going to do.
- Fixed a minor conversion warning in audio-io.c
jp9000
Palana
GoaLitiuM
Danni