Add API for streaming services. The services API simplifies the
creation of custom service features and user interface.
Custom streaming services later on will be able to do things such as:
- Be able to use service-specific APIs via modules, allowing a more
direct means of communicating with the service and requesting or
setting service-specific information
- Get URL/stream key via other means of authentication such as OAuth,
or be able to build custom URLs for services that require that sort
of thing.
- Query information (such as viewer count, chat, follower
notifications, and other information)
- Set channel information (such as current game, current channel title,
activating commercials)
Also, I reduce some repeated code that was used for all libobs objects.
This includes the name of the object, the private data, settings, as
well as the signal and procedure handlers.
I also switched to using linked lists for the global object lists,
rather than using an array of pointers (you could say it was..
pointless.) ..Anyway, the linked list info is also stored in the shared
context data structure.
Before, async video sources would flicker because they were only being
drawn when they were updated. So when updated, they'd draw that frame,
then it would stop drawing it until it updated again. This fixes that
issue and they should now draw properly.
Also, fix a few other minor bugs and issues relating to async video,
and make it so that non-async video filters can be properly applied to
them.
For the purposes of testing, change the 'test-random' source to an async
video source that updates every quarter of a second with a new random
face.
Also fix a bug where non-async video sources wouldn't have filter
effects applied properly.
- Implement the RTMP output module. This time around, we just use a
simple FLV muxer, then just write to the stream with RTMP_Write.
Easy and effective.
- Fix the FLV muxer, the muxer now outputs proper FLV packets.
- Output API:
* When using encoders, automatically interleave encoded packets
before sending it to the output.
* Pair encoders and have them automatically wait for the other to
start to ensure sync.
* Change 'obs_output_signal_start_fail' to 'obs_output_signal_stop'
because it was a bit confusing, and doing this makes a lot more
sense for outputs that need to stop suddenly (disconnections/etc).
- Encoder API:
* Remove some unnecessary encoder functions from the actual API and
make them internal. Most of the encoder functions are handled
automatically by outputs anyway, so there's no real need to expose
them and end up inadvertently confusing plugin writers.
* Have audio encoders wait for the video encoder to get a frame, then
start at the exact data point that the first video frame starts to
ensure the most accrate sync of video/audio possible.
* Add a required 'frame_size' callback for audio encoders that
returns the expected number of frames desired to encode with. This
way, the libobs encoder API can handle the circular buffering
internally automatically for the encoder modules, so encoder
writers don't have to do it themselves.
- Fix a few bugs in the serializer interface. It was passing the wrong
variable for the data in a few cases.
- If a source has video, make obs_source_update defer the actual update
callback until the tick function is called to prevent threading
issues.
Improve the properties API so that it can actually respond somewhat to
user input. Maybe later this might be further improved or replaced with
something script-based.
When creating a property, you can now add a callback to that property
that notifies when the property has been changed in the user interface.
Return true if you want the properties to be refreshed, or false if not.
Though now that I think about it I doubt there would ever be a case
where you would have this callback and *not* refresh the properties.
Regardless, this allows functions to change the values of properties or
settings, or enable/disable/hide other property controls from view
dynamically.
- Add a properties window for sources so that you can now actually edit
the settings for sources. Also, display the source by itself in the
window (Note: not working on mac, and possibly not working on linux).
When changing the settings for a source, it will call
obs_source_update on that source when you have modified any values
automatically.
- Add a properties 'widget', eventually I want to turn this in to a
regular nice properties view like you'd see in the designer, but
right now it just uses a form layout in a QScrollArea with regular
controls to display the properties. It's clunky but works for the
time being.
- Make it so that swap chains and the main graphics subsystem will
automatically use at least one backbuffer if none was specified
- Fix bug where displays weren't added to the main display array
- Make it so that you can get the properties of a source via the actual
pointer of a source/encoder/output in addition to being able to look
up properties via identifier.
- When registering source types, check for required functions (wasn't
doing it before). getheight/getwidth should not be optional if it's
a video source as well.
- Add an RAII OBSObj wrapper to obs.hpp for non-reference-counted
libobs pointers
- Add an RAII OBSSignal wrapper to obs.hpp for libobs signals to
automatically disconnect them on destruction
- Move the "scale and center" calculation in window-basic-main.cpp to
its own function and in its own source file
- Add an 'update' callback to WASAPI audio sources
Also, rename atomic functions to be consistent with the rest of the
platform/threading functions, and move atomic functions to threading*
files rather than platform* files
Ensure that a source has a valid name. Duplicates aren't a big deal
internally, but sources without a name are probably something that
should be avoided. Made is so that if a source is programmatically
created without a name, it's assigned an index based name.
In the main basic-mode window, made it check to make sure the name was
valid as well.
- Implement a means of obtaining default settings for an
input/output/encoder. obs_source_defaults for example will return
the default settings for a particular source type.
- Because C++ doesn't have designated initializers, use functions in
the WASAPI plugin to register the sources instead.
- Split input and output audio captures so that they're different
sources. This allows easier handling and enumeration of audio
devices without having to do some sort of string processing.
This way the user interface code can handle this a bit more easily,
and so that it doesn't confuse users either. This should be done for
all audio capture sources for all operating systems. You don't have
to duplicate any code, you just need to create input/output wrapper
functions to designate the audio as input or output before creation.
- Make it detect soundflower and wavtap devices as mac "output" devices
(even though they're actually input) for the mac output capture, and
make it so that users can select a default output capture and
automatically use soundflower or wavtap.
I'm not entirely happy about having to do this, but because mac is
designed this way, this is really the only way to handle it that
makes it easier for users and UI code to deal with.
Note that soundflower and wavtap are still also designated as input
devices, so will still show up in input device enumeration.
- Remove pragma messages because they were kind polluting the other
compiler messages and just getting in the way. In the future we can
just do a grep for TODO to find them.
- Redo list property again, this time using a safer internal array,
rather than requiring sketchy array inputs. Having functions handle
everything behind the scenes is much safer.
- Remove the reference counter debug log code, as it was included
unintentionally in a commit.
- Signals and dynamic callbacks now require declarations to be made
before being used. What this does is allows us to get information
about the functions dynamically which can be relayed to the user and
plugins for future extended usage (this should have big implications
later for scripting in particular, hopefully).
- Reduced the number of types calldata uses from "everything I could
think of" to simply integer, float, bool, pointer/object, string.
Integer data is now stored as long long. Floats are now stored as
doubles (check em).
- Use a more consistent naming scheme for lexer error/warning macros.
- Fixed a rather nasty bug where switching to an existing scene would
cause it to increment sourceSceneRefs, which would mean that it would
never end up never properly removing the source when the user clicks
removed (stayed in limbo, obs_source_remove never got called)
LOG_ERROR should be used in places where though recoverable (or at least
something that can be handled safely), was unexpected, and may affect
the user/application.
LOG_WARNING should be used in places where it's not entirely unexpected,
is recoverable, and doesn't really affect the user/application.
Split off activate to activate and show callbacks, and split off
deactivate to deactivate and hide callbacks. Sources didn't previously
have a means to know whether it was actually being displayed in the main
view or just happened to be visible somewhere. Now, for things like
transition sources, they have a means of knowing when they have actually
been "activated" so they can initiate their sequence.
A source is now only considered "active" when it's being displayed by
the main view. When a source is shown in the main view, the activate
callback/signal is triggered. When it's no longer being displayed by
the main view, deactivate callback/signal is triggered.
When a source is just generally visible to see by any view, the show
callback/signal is triggered. If it's no longer visible by any views,
then the hide callback/signal is triggered.
Presentation volume will now only be active when a source is active in
the main view rather than also in auxilary views.
Also fix a potential bug where parents wouldn't properly increment or
decrement all the activation references of a child source when a child
was added or removed.
This allows the changing of bideo settings without having to completely
reset all graphics data. Will recreate internal output/conversion
buffers and such and reset the main preview.
Make it so obs_data settings input in to *_update are applied to the
existing settings rather than fully replace the existing settings. That
way you can update with only certain specific settings, leaving other
settings untouched. Of course if you're already using the original
settings pointer in the first place then you've already done that, so
it'll just ignore it because you've already applied them.
- Remove obs_source::type because it became redundant now that the
type is always stored in the obs_source::info variable.
- Apply presentation volumes of 1.0 and 0.0 to sources when they
activate/deactivate, respectively. It also applies that presentation
volume to all sub-sources, with exception of transition sources.
Transition sources must apply presentation volume manually to their
sub-sources with the new transition functions below.
- Add a "transition_volume" variable to obs_source structure, and add
three functions for handling volume for transitions:
* obs_transition_begin_frame
* obs_source_set_transition_vol
* obs_transition_end_frame
Because the to/from targets of a transition source might both contain
some of the same sources, handling the transitioning of volumes for
that specific situation becomes an issue.
So for transitions, instead of modifying the presentation volumes
directly for both sets of sources, we do this:
- First, call obs_transition_begin_frame at the beginning of each
transition frame, which will reset transition volumes for all
sub-sources to 0. Presentation volumes remain unchanged.
- Call obs_source_set_transition_vol on each sub-source, which will
then add the volume to the transition volume for each source in
that source's tree. Presentation volumes still remain unchanged.
- Then you call obs_trandition_end_frame when complete, which will
then finally set the presentation volumes to the transition
volumes.
For example, let's say that there's one source that's within both the
"transitioning from" sources and "transition to" sources. It would
add both the fade in and fade out volumes to that source, and then
when the frame is complete, it would set the presentation volume to
the sum of those two values, rather than set the presentation volume
for that same source twice which would cause weird volume jittering
and also set the wrong values.
Now sources will be properly activated and deactivated when they are in
use or not in use.
Had to figure out a way to handle child sources, and children of
children, just ended up implementing simple functions that parents use
to signal adding/removal to help with hierarchial activation and
deactivation of child sources.
To prevent the source activate/deactivate callbacks from being called
more than once, added an activation reference counter. The first
increment will call the activate callback, and the last decrement will
call the deactivate callback.
Added "source-activate" and "source-deactivate" signals to the main obs
signal handler, and "activate" and "deactivate" to individual source
signal handlers.
Also, fixed the main window so it properly selects a source when the
current active scene has been changed.
Added a "master" volume for the entire audio subsystem.
Also, added a "presentation" volume for both the master volume and for
each invidiaul source. The presentation volume is used to control
things like transitioning volumes, preventing sources from outputting
any audio when they're inactive, as well as some other uses in the
future.
If audio was under, it originally did a full reset of the audio timing.
However, resetting the audio timing when this happens is kind of a bad
thing. It's better just to clamp the value to the expected timestamp to
ensure seamless audio output.
Also, implement audio timestamp smoothing to ensure audio tries to be as
seamless as possible.
Add a scaler interface (defaults to swscale), and if a separate output
wants to use a different scale or format than the default output format,
allow a scaler instance to be created automatically for that output,
which will then receive the new scaled output.
There were a *lot* of warnings, managed to remove most of them.
Also, put warning flags before C_FLAGS and CXX_FLAGS, rather than after,
as -Wall -Wextra was overwriting flags that came before it.
The API used to be designed in such a way to where it would expect
exports for each individual source/output/encoder/etc. You would export
functions for each and it would automatically load those functions based
on a specific naming scheme from the module.
The idea behind this was that I wanted to limit the usage of structures
in the API so only functions could be used. It was an interesting idea
in theory, but this idea turned out to be flawed in a number of ways:
1.) Requiring exports to create sources/outputs/encoders/etc meant that
you could not create them by any other means, which meant that
things like faruton's .net plugin would become difficult.
2.) Export function declarations could not be checked, therefore if you
created a function with the wrong parameters and parameter types,
the compiler wouldn't know how to check for that.
3.) Required overly complex load functions in libobs just to handle it.
It makes much more sense to just have a load function that you call
manually. Complexity is the bane of all good programs.
4.) It required that you have functions of specific names, which looked
and felt somewhat unsightly.
So, to fix these issues, I replaced it with a more commonly used API
scheme, seen commonly in places like kernels and typical C libraries
with abstraction. You simply create a structure that contains the
callback definitions, and you pass it to a function to register that
definition (such as obs_register_source), which you call in the
obs_module_load of the module.
It will also automatically check the structure size and ensure that it
only loads the required values if the structure happened to add new
values in an API change.
The "main" source file for each module must include obs-module.h, and
must use OBS_DECLARE_MODULE() within that source file.
Also, started writing some doxygen documentation in to the main library
headers. Will add more detailed documentation as I go.
- Fill in the rest of the FFmpeg test output code for testing so it
actually properly outputs data.
- Improve the main video subsystem to be a bit more optimal and
automatically output I420 or NV12 if needed.
- Fix audio subsystem insertation and byte calculation. Now it will
seamlessly insert new audio data in to the audio stream based upon
its timestamp value. (Be extremely cautious when using floating
point calculations for important things like this, and always round
your values and check your values)
- Use 32 byte alignment in case of future optimizations and export a
function to get the current alignment.
- Make os_sleepto_ns return true if slept, false if the time has
already been passed before the call.
- Fix sinewave output so that it actually properly calculates a middle
C sinewave.
- Change the use of row_bytes to linesize (also makes it a bit more
consistent with FFmpeg's naming as well)
- Add planar audio support. FFmpeg and libav use planar audio for many
encoders, so it was somewhat necessary to add support in libobs
itself.
- Improve/adjust FFmpeg test output plugin. The exports were somewhat
messed up (making me rethink how exports should be done). Not yet
functional; it handles video properly, but it still does not handle
audio properly.
- Improve planar video code. The planar video code was not properly
accounting for row sizes for each plane. Specifying row sizes for
each plane has now been added. This will also make it more compatible
with FFmpeg/libav.
- Fixed a bug where callbacks wouldn't create properly in audio-io and
video-io code.
- Implement 'blogva' function to allow for va_list usage with libobs
logging.
- Implement texture scaling/conversion/downloading for the main view so
we can finally start getting data to output.
Also, redesign how it works a bit, it will now properly wait one full
frame for each step in the process: rendering the main texture,
scaling the main texture to an output texture, staging/downloading the
ouput texture, and then outputting that staged data. This way, the
GPU will have more than enough time to fully complete each step.
- Fix a bug with OpenGL plugin's texture staging function. Was using
glBindBuffer instead of what should have been used: glBindTexture.
- Change the naming scheme of the variables in default.effect. It's now
named with the idea of just "color matrix" in mind instead of "yuv
matrix", and instead of DrawRGBToYUV, it's now just DrawMatrix.
- Implemented better C++ classes for handling scenes/sources/items in
obs.hpp, allowing them to automatically increment and decrement the
references of each, as well as assign them to QVariants.
- Because QVariants are now using the C++ classes, remove the pointer
QVariant wrapper.
- Use the new C++ classes with the QVariant user data of list box items,
both for the sake of thread safety and to ensure that the data
referenced is not freed until removed. NOTE: still might need some
testing.
- Implemented a source-remove signal from libobs, and start using that
signal instead of the source-destroy signal for signalling item
removal.
- Add property list callbacks to sources/outputs/encoders so that if
necessary user interface can be automatically generated or perhaps a
property list widget can be used for them.
- Change some of the property API names. obs_property_list_t felt a bit
awkward when actually using it, so I just renamed it to
obs_properties_t.
- Removed the getdata/setdata nad getparam/setparam functions from
sources/services, they will be superseded by the dynamic procedure
call API.
Add the ability to be able to call and use toolkit-specific or
program-specific user interface in modules.
User interface code can be either bundled with the module, or 'split'
out in to separate libraries (recommended).
There are three reasons why splitting is recommended:
1.) It allows plugins to be able to create custom user interface for
each toolkit if desired.
2.) Often, UI will be programmed in one language (the language of the
toolkit), and core logic may be programmed in another. This
allows plugins to keep the languages separated if necessary.
3.) It prevents direct linkage of UI toolkits libraries with core
module logic.
Splitting is not required, though is recommended if you want your plugin
to be more flexible with other user interface toolkits or programs.
Will implement a generic properties lookup next, which will be used for
automatic UI handling so that plugin UI isn't necessarily required.
- Add 'set_default' functions to obs-data.*. These functions ensure
that a paramter exists and that the parameter is of a specific type.
If not, it will create or overwrite the value with the default setting
instead.
These functions are meant to be explicitly called before using any of
the 'get' functions. The reason why it was designed this way is to
encourage defaults to be set in a single place/function.
For example, ideal usage is to create one function for your data,
"set_my_defaults(obs_data_t data)", set all the default values within
that function, and then call that function on create/update, that way
all defaults are centralized to a single place.
- Ensure that data passed to sources/encoders/outputs/etc is always
valid, and not a null value.
- While I'm remembering, fix a few defaults of the main program config
file data.
Add a fairly easy to use settings interface that can be passed to
plugins, and replaced the old character string system that was being
used before. The new data interface allows for an easier method of
getting/altering settings for plugins, and is built to be serializable
to/from JSON.
Also, removed another wxFormBuilder file that was no longer in use.
I'm doing this because I might create another data structure called
obs_data for a different purpose. That and obs_program_data feels a bit
less vague for what it does.
- Added some code for FFmpeg output that I'm still playing around with.
Right now I'm just trying to get it to output to file and try to
understand the FFmpeg/libav APIs. Hopefully in the future this plugin
can be used for any sort of output to FFmpeg.
- Fixed a cast warning in audio-io.c with size_t -> uint32_t
- Renamed the 'video_info' and 'audio_info' structures to
'video_conver_info' and 'audio_convert_info' to better represent their
actual purpose, and to avoid confusion with 'audio_output_info' and
'video_output_info' structures.
- Removed a few macros from obs-def.h that were at one point going to be
used but no longer going to be used (at least for now)
Just a minor fix mostly because I noticed that I kept accidentally
forgetting to add checks to the code properly. This is one of those
cases where macros come in useful, as macros can automate the process
and help prevent these mistakes from happening by accident.
Completely revamped the entire media i/o data and handlers. The
original idea was to have a system that would have connecting media
inputs and outputs, but at a certain point I realized that this was an
unnecessary complexity for what we wanted to do. (Also, it reminded me
of directshow filters, and I HATE directshow with a passion, and
wouldn't wish it upon my greatest enemy)
Now, audio/video outputs are connected to directly, with better callback
handlers, and will eventually have the ability to automatically handle
conversions such as 4:4:4 to 4:2:0 when connecting to an input that uses
them. Doing this will allow the video/audio i/o handlers to also
prevent duplicate conversion, as well as make it easier/simple to use.
My true goal for this is to make output and encoder plugins as simple to
create as possible. I want to be able to be able to create an output
plugin with almost no real hassle of having to worry about image
conversions, media inputs/outputs, etc. A plugin developer shouldn't
have to handle that sort of stuff when he/she doesn't really need to.
Plugins will be able to simply create a callback via obs_video() and/or
obs_audio(), and they will automatically receive the audio/video data in
the formats requested via a simple callback, without needing to do
almost anything else at all.
When the first async video frame is used it would not set audio timing,
moved that code into obs_source_getframe. Also, might consider renaming
obs_source_getframe. "Query frame" instead perhaps? Will sleep on it,
might not even bother.
- Add preliminary (yet to be tested) handling of timestamp invalidation
issues that can happen with specific devices, where timestamps can
reset or go backward/forward in time with no rhyme or reason. Spent
the entire day just trying to figure out the best way to handle this.
If both audio and video are present, it will increment a reference
counter if video timestamps invalidate, and decrement the reference
counter when the audio timestamps invalidate. When the reference
counter is not 0, it will not send audio as the audio will have
invalid timing. What this does is it ensures audio data will never go
out of bounds in relation to the video, and waits for both audio and
video timestamps to "jump" together before resuming audio.
- Moved async video frame timing adjustment code into
obs_source_getframe instead so it's automatically handled whenever
called.
- Removed the 'audio wait buffer' as it was an unnecessary complexity
that could have had problems in the future. Instead, audio will not
be added until video starts for sources that have both async
audio/video. Audio could have buffered for too long of a time anyway,
who knows what devices are going to do.
- Fixed a minor conversion warning in audio-io.c
- Often, timestamps will go "back" in time with certain.. terrible
devices that no one should use. When this occurs, timing is now
reset so that the new audio comes in directly after the old audio
seamlessly.
- Audio data was just being popped to the "front" of the mix buffer, so
instead it now properly pops into the correct position in the mix
buffer (proper mixing still needs to be implemented)
- Added a test audio sinewave test source that should just play a sine
wave of the middle C note. Using unsigned 8 bit mono to test
ffmpeg's audio resampler, seems to work pretty good.
- Fixed a boolean trap in threading.h for the event_init function, it
now uses enum event_type, which can be EVENT_TYPE_MANUAL or
EVENT_TYPE_AUTO, to specify whether the event is automatically reset
or not.
- Changed display names of test sources to something a little less
vague.
- Removed te whole "if timestamp is 0 just use current system time"
when outputting source audio, if you want to use system time you
should just use system time yourself. Using 0 as some sort of
"indicator" like that just makes things confusing, and prevents you
from legitimately using 0 as a timestamp for your audio data.
- Mixing still isn't implemented, but the audio system should be able
to start up, and mix at least once audio line for the time being.
Will have to write some test audio sources to verify things are
working properly, and build the rest of the output functionality.
- Added a volume variable to the obs_source structure and implemented
functions for manipulating source volume.
- Added a volume variable to the audio_data structure so that the
volume will be applied when mixing.
Scenes will now signal via their source when an item has been added
or removed from them.
"add" - Item added to the scene.
Parameters: "scene": Scene that the item was added to.
"item": Item that was added.
"remove" - Item removed from the scene.
Parameters: "scene": Scene that the item was removed from.
"item": Item that was removed.
jp9000
Palana