API removed:
--------------------
gs_effect_t *obs_get_default_effect(void);
gs_effect_t *obs_get_default_rect_effect(void);
gs_effect_t *obs_get_opaque_effect(void);
gs_effect_t *obs_get_solid_effect(void);
gs_effect_t *obs_get_bicubic_effect(void);
gs_effect_t *obs_get_lanczos_effect(void);
gs_effect_t *obs_get_bilinear_lowres_effect(void);
API added:
--------------------
gs_effect_t *obs_get_base_effect(enum obs_base_effect effect);
Summary:
--------------------
Combines multiple near-identical functions into a single function with
an enum parameter.
API changed from:
obs_source_info::get_name(void)
obs_output_info::get_name(void)
obs_encoder_info::get_name(void)
obs_service_info::get_name(void)
API changed to:
obs_source_info::get_name(void *type_data)
obs_output_info::get_name(void *type_data)
obs_encoder_info::get_name(void *type_data)
obs_service_info::get_name(void *type_data)
This allows the type data to be used when getting the name of the
object (useful for plugin wrappers primarily).
NOTE: Though a parameter was added, this is backward-compatible with
older plugins due to calling convention. The new parameter will simply
be ignored by older plugins, and the stack (if used) will be cleaned up
by the caller.
When an audio filter is applied to a video source that also has
accompanying audio, it would cause the video from the source to stop
rendering.
The original code this was from was to prevent audio-only sources from
rendering video, but I neglected to make sure that this would not apply
to filters, and thus when an audio filter is on a source with video, the
code would kill the video.
Due to the fact that async timestamps themselves can be susceptible to
minor jitter from certain types of inputs, increase the allowable jitter
compensation value to ensure that the rendered frame timing from async
video sources is always as close as possible to the compositor.
When the framerate of the source is the same as the framerate as the
compositor, this (combined with the fact that clamped video timing now
being used with async video frames) helps ensure that buffered async
video sources will sync up their rendering to the compositor as
accurately as possible despite jitter from the source's timestamps.
If there is no jitter in the source's timestamps then it'll always sync
up perfectly with the compositor, thanks to clamped video timing.
When playing back buffered async frames, this reduces the probability
that new frames will be missed/skipped due to jitter in the system
timestamps.
If a buffered async source is playing at the same framerate as the
compositor and there is no jitter in the async source's timestamps, then
the async source will play back perfectly in sync with the compositor
thanks to this change, ensuring that there's no skipped or missed frames
in video playback.
When buffering is enabled for an async video source, sometimes minor
drift in timestamps or unexpected delays to frames can cause frames to
slowly buffer more and more in memory, in some cases eventually causing
the system to run out of memory.
The circumstances in which this can happen seems to depend on both the
computer and the devices in use. So far, the only known circumstances
in which this happens are with heavily buffered devices, such as
hauppauge, where decoding can sometimes take too long and cause
continual frame playback delay, and thus continual buffering until
memory runs out. I've never been able to replicate it on any of my
machines however, even after hours of testing.
This patch is a precautionary measure that puts a hard limit on the
number of async frames that can be currently queued to prevent any case
where memory might continually build for whatever reason. If it goes
over the limit, it clears the cache to reset the buffering.
I had a user with this problem test this patch with success and positive
feedback, and the intervals between buffering resets were long to where
it wasn't even noticeable while streaming/recording.
Ideally when decoding frames (such as from those devices), frame
dropping should be used to ensure playback doesn't incur extra delay,
although this sort of hard limit on the frame cache should still be
implemented regardless just as a safety precaution. For DirectShow
encoded devices I should just switch to faruton's libff for decoding and
enable the frame dropping options. It would probably explain why no
one's ever reported it for the media source, and pretty much only from
DirectShow device usage.
Fixes a crash that could happen if any of the mutexes are used in the
create callback, or before the obs_source_init function is called.
I'm not sure how this function order slipped because it seems fairly
obvious that these mutexes should be created before the create callback.
Had this crash happen to me when creating a WASAPI output source, the
create callback of the WASAPI source creates a thread which outputs
audio, and that thread managed to call obs_source_output_audio before
the obs_source_init function was called, which in turn caused it to try
to use a null mutex.
When caching a new frame, keep a reference to the frame while copying to
ensure that the frame is not potentially destroyed for whatever reason
while that data is being copied.
The obs_source::async_reset_texture variable can cause a data race
between threads to occur because it could be set to true in one thread
then changed back to false in another thread. This could cause the
async texture to not update its size when it's supposed to, which can
cause a crash or corruption when copying data from a frame of a
differing size.
The solution to this is to:
- Delete the async_reset_texture variable, and make the
set_async_texture_size function change the texture size if the
async_width, async_height, or async_format variables differ from the
frame's width/height/format. Those variables are then only ever set
in the libobs graphics thread.
- Make the cache_video function use separate variables from other
functions to detect a change in size (due to the fact that the texture
size should only be resized in the libobs graphics thread). These
variables are async_cache_width, async_cache_height, and
async_cache_format, which are only be set in the thread that calls
obs_source_output_video.
How to replicate the data race:
- On OSX, use window capture on a textedit window, then continually
resize the textedit window.
Due to a bad 'if' expression, when a filter that is not last in the
chain is disabled or being bypassed, it ends up still calling the
filter's video processing function unintentionally.
This fix makes sure that it only calls the appropriate render functions
if the next filter target is the source, otherwise it will just call
obs_source_video_render to process the next filter in the chain.
How to replicate the bug:
1. Create two crop filters on the same source
2. Give each crop filter a different distinct value
3. Disable both crop filters
4. The image would still be cropped
This fixes an issue where cache frames would not free at all after
having been allocated with no upper limit on the cached frame size. If
cached frames go unused for a specific period of time, they are
deallocated and removed from the cache.
This is preferable to having an upper cache limit due to the potential
for async delay filtering.
Under certain circumstances the cache could be prone to growing too
large unintentionally. Setting a hard maximum limit should prevent
memory from growing if we suddenly get a lot of frames.
Async frames are only swapping when rendering, or when not visible.
This is a flawed design due to the fact that there are certain
circumstances where the source is neither visible nor currently
rendering.
This is what caused a memory leak when scene items were marked as
invisible, because if a source has an async child source and decides not
to render that source for whatever reason, the child source would not
process the async frames at all, and the cache would just grow.
To fix this, simply moving the async frame cycle to tick fixes the issue
due to the fact that tick is always called regardless of circumstance.
Filtering the video before it's output to the texture means that it
happens after all the processing on the timestamps and such of the video
data. This way, the video filter does not have to worry about what's
currently buffered, and it won't affect timing.
When OBS is shutting down, if for some reason the filter is destroyed
before the parent source is destroyed, it would try to remove itself
from the source, but it would decrement the reference and try to destroy
itself again while already in the process of destroying itself.
So, the solution was simply to make sure that if it's removing itself
from the source that it doesn't decrement its own reference.
The "last" filter that's rendered is technically at filter index 0, so
enumeration needs to be from the last index in the list to the first
index in the list.
When rendering a filter to a texture, the target is empty and unused, so
there's no reason for blending to be on when rendering the filter to a
render target.
obs_source_process_filter tried to do everything in a single function,
but the problem is that effect parameters would not properly be
accounted for due to the way it internally draws, therefore it was
necessary to split the functions in to two, you first call
obs_source_process_filter_begin, then you set your effect parameters,
then you finally call obs_source_process_filter_end. This ensures that
when the filter is drawn, that the effect parameters are set.
When the filter chain finally reaches the source and the last filter in
the chain is set to not render directly (meaning it has to render to
texture), it would not render the source with any effect due to the fact
that it expects a filter to be present.
Adds a source callback function that is used when a filter is removed
from a source. This ensures that any data that might be associated with
that source can be cleaned up if necessary.
There are a few more conditions which need to be checked to ensure
whether we can actually bypass or not; in this particular case we are
using the YUV texture shaders, plus the image can also be flipped, so we
can't really use the bypass optimization in those situations.
These functions are primarily for use with filters, filters need to be
able to get the width/height of a target source without it necessarily
getting the post-filtered dimensions.
Previously I had it set so that it would set the async_width and
async_height variables to 0,0 if a null frame occurs, but the problem
with this is that it was inadvertently cause the frame cache to clear
when it starts receiving textures again because it detects it as a size
change.
So instead of changing async_width and async_height to 0 when a null
frame occurs, change it so that if the source is set to an inactive
state, make obs_source_get_width/_get_height return 0 for their
dimensions instead.
When a frame is processed by a filter, it comes directly from the
source's video frame cache. However, if a filter is using or processing
those frames for whatever reason, there would be no guarantee that the
frames would persist during processing, and frames could eventually be
deallocated unexpected, for example when the resolution or format
changes.
So the solution is to implement simple reference counting for the frames
so that the frames will exist until they have been released by any
source or filter that's using them.
Fix a bug where when a source first starts up its async cache, it
unintentionally resets its cache, which means that the first few frames
would be lost.
The wrong function was being used to recurse through the filter chain in
obs_source_process_filter, obs_source_get_[width/height] would get the
post-effect dimensions rather than the pre-effect dimensions.
If the audio data had the same format/samplerate as the obs audio
subsystem, it would fail to simply destroy the resampler and set it to
NULL, and then any audio data going through would use the resampler that
was being used before that, causing audio to become garbage.
This bug only started appearing when I recently changed the libobs
internal audio subsystem format to non-interleaved floating point, which
is a common format, and thus caused this bug to actually occur more
often.
I when a source has both async audio/video capability, it would ignore
audio until the video has started. There's really no need to do this,
when the video starts it'll just fix up the timing automatically.
This should fix the case where things like the media source would not be
able to play audio-only files.
Core API functions changed:
-----------------------------
EXPORT bool obs_reset_audio(struct audio_output_info *aoi);
EXPORT bool obs_get_audio_info(struct audio_output_info *aoi);
To:
-----------------------------
EXPORT bool obs_reset_audio(const struct obs_audio_info *oai);
EXPORT bool obs_get_audio_info(struct obs_audio_info *oai);
Core structure added:
-----------------------------
struct obs_audio_info {
uint32_t samples_per_sec;
enum speaker_layout speakers;
uint64_t buffer_ms;
};
Non-interleaved (planar) floating point output is standard with audio
filtering, so to prevent audio filters from having to worry about
different audio format implementations and for the sake consistency
between user interfaces, make it so that audio is always set to
non-interleaved floating point output.
This code was originally meant to skip some checks as an optimization,
but it did not account for async video sources and would call
obs_source_default_render (which is only for synchronous video sources),
this fixes the issue by just calling obs_source_video_render.
The obs_source_get_width and obs_source_get_height functions need to
account for filters that may change their width/height such as a crop
filter or something similar.
As a side effect of this commit, because these functions need to lock
the filter mutex, these functions can no longer be used with a
const-qualified obs_source_t pointer.
This function is somewhat big and is a function called conditionally,
therefore having it be inline is somewhat of a waste if it's not always
being called.
Graphics data has to be freed inside of an active graphics context,
otherwise the resources will not be freed; in the future, we should
probably make sure that the graphics subsystem automatically
asserts/warns about this scenario.
I was iterating through the obs_source::filters array to remove filters,
but every time I removed a filter, it would remove itself from the
array, therefore it would end up skipping items in the array, therefore
leaving filters unreleased. This just removes the first filter until
all filters have been removed.
For the show/hide and activate/deactivate callbacks, schedule these
callbacks to only be called from within the video thread rather than in
a separate thread. This ensures that any potential graphics activity
that occurs within them is kept in the same thread.
obs_source_inc_showing and obs_source_dec_showing are used to indicate
that a source is showing or no longer being shown in a particular area
of the program outside from the main render view.
One could use an obs_view, but I felt like it was unnecessary because
using an obs_view just to display a single source feels somewhat
superfluous.
API changed:
--------------------------
void obs_output_set_audio_encoder(
obs_output_t *output,
obs_encoder_t *encoder);
obs_encoder_t *obs_output_get_audio_encoder(
const obs_output_t *output);
obs_encoder_t *obs_audio_encoder_create(
const char *id,
const char *name,
obs_data_t *settings);
Changed to:
--------------------------
/* 'idx' specifies the track index of the output */
void obs_output_set_audio_encoder(
obs_output_t *output,
obs_encoder_t *encoder,
size_t idx);
/* 'idx' specifies the track index of the output */
obs_encoder_t *obs_output_get_audio_encoder(
const obs_output_t *output,
size_t idx);
/* 'mixer_idx' specifies the mixer index to capture audio from */
obs_encoder_t *obs_audio_encoder_create(
const char *id,
const char *name,
obs_data_t *settings,
size_t mixer_idx);
Overview
--------------------------
This feature allows multiple audio mixers to be used at a time. This
capability was able to be added with surprisingly very little extra
overhead. Audio will not be mixed unless it's assigned to a specific
mixer, and mixers will not mix unless they have an active mix
connection.
Mostly this will be useful for being able to separate out specific audio
for recording versus streaming, but will also be useful for certain
streaming services that support multiple audio streams via RTMP.
I didn't want to use a variable amount of mixers due to the desire to
reduce heap allocations, so currently I set the limit to 4 simultaneous
mixers; this number can be increased later if needed, but honestly I
feel like it's just the right number to use.
Sources:
Sources can now specify which audio mixers their audio is mixed to; this
can be a single mixer or multiple mixers at a time. The
obs_source_set_audio_mixers function sets the audio mixer which an audio
source applies to. For example, 0xF would mean that the source applies
to all four mixers.
Audio Encoders:
Audio encoders now must specify which specific audio mixer they use when
they encode audio data.
Outputs:
Outputs that use encoders can now support multiple audio tracks at once
if they have the OBS_OUTPUT_MULTI_TRACK capability flag set. This is
mostly only useful for certain types of RTMP transmissions, though may
be useful for file formats that support multiple audio tracks as well
later on.
The temporary unoptimized code we were using before just completely
allocated a new copy of each frame every single time a new async frame
was output by the source plugin. This just creates a cache of frames as
needed for the current format/width/height to minimize the allocation
and deallocation. If new frames come in that are of a different
format/width/height, it'll just clear the cache. This is a fairly
important optimization.
all the async video related stuff usually started with async_*, and
there were two that didn't. So I just renamed them so they have the
same naming convention
If an async video source stops video for whatever reason, it would get
stuck on the last frame that was played. This was particularly awkward
when I wanted to give the user the ability to deactivate a source such
as a webcam because it would get stuck on the last frame.
I unintentionally made it use obs_source::sample_info instead of using
the actual target channel count, which is designated by the OBS output
sampler info. obs_source::sample_info is actually used to indicate the
currently set sampler information for incoming samples. So if a source
is outputting 5.1 channel 48khz audio, and OBS is running at stereo
44.1khz, then the obs_source::sample_info value would be set to
5.1/48khz, not the other way around. It indicates what the source
itself is running at, not what OBS is running at.
I suppose the variable needs a better name because even I used it
incorrectly despite actually having been the one who wrote it.
The copy_audio_data function really shouldn't be inlined because it's
being called twice. It's somewhat unnecessary, I think I left it inline
by accident.
This changes the way source volume handles transitioning between being
active and inactive states.
The previous way that transitioning handled volume was that it set the
presentation volume of the source and all of its sub-sources to 0.0 if
the source was inactive, and 1.0 if active. Transition sources would
then also set the presentation volume for sub-sources to whatever their
transitioning volume was. However, the problem with this is that the
design didn't take in to account if the source or its sub-sources were
active anywhere else, so because of that it would break if that ever
happened, and I didn't realize that when I was designing it.
So instead, this completely overhauls the design of handling
transitioning volume. Each frame, it'll go through all sources and
check whether they're active or inactive and set the base volume
accordingly. If transitions are currently active, it will actually walk
the active source tree and check whether the source is in a
transitioning state somewhere.
- If the source is a sub-source of a transition, and it's not active
outside of the transition, then the transition will control the
volume of the source.
- If the source is a sub-source of a transition, but it's also active
outside of the transition, it'll defer to whichever is louder.
This also adds a new callback to the obs_source_info structure for
transition sources, get_transition_volume, which is called to get the
transitioning volume of a sub-source.
The reason to keep a reference counter for transitions is due to an
optimization I'm planning on when calculating transition volumes. I'm
planning on walking the source tree to be able to calculate the current
base volume of a source, but *only* if there are transitions active,
because the only time that the volume can be anything other than 1.0
or 0.0 is when there are active transitions, which may change the base
volume of a source.
When the presentation volume is set for a source, it's set for all of
its children and their children. The original intention for doing this
was to be able to use it for transitioning, but honestly it's just bad
design, and I feel there are better ways to handle transitioning volume.
Changed the design from using obs_source::enum_refs to just simply
preventing infinite source recursion in general, rather than allowing it
through the enum_refs variable. obs_source_add_child has been changed
so that it now returns a boolean, and if the function fails, it means
that the child cannot be added due to that potential recursion.
Remove the calculation of volume levels and the corresponding signal
from obs_source since this is now handled in the volume meter.
Code that is interested in the volume levels can either use the
volmeter provided from obs_audio_controls or use the audio_data signal
to gain access to the raw audio data.
This adds a new signal to (audio) sources which is emitted whenever new
audio data is received from the source. This enables other code that is
interested in the raw audio data to directly access it when it becomes
available.
This replaces the old code for the audio meter that was using
calculations in two different places with the new audio meter api.
The source signal will now emit simple levels instead of dB values,
in order to avoid dB conversion and calculation in the source.
The GUI on the other hand now expects simple position values from
the volume meter api with no knowledge about dB calculus either.
That way all code that handles those conversions is in one place,
with the option to easily add new mappings that can be used
everywhere.
Because a vec3 structure can contain a __m128 variable and not the
expected three floats x, y, and z, you must use vec3_set when
setting a value for a vec3 structure to ensure that it uses the proper
intrinsics internally if necessary.
If you look at the previous commits, you'll see I had added
obs_source_draw before. For custom drawn sources in particular, each
time obs_source_draw was called, it would restart the effect and its
passes for each draw call, which was not optimal. It should really use
the effect functions for that. I'll have to add a function to simplify
effect usage.
I also realized that including the color matrix parameters in
obs_source_draw made the function kind of messy to use; instead,
separating the color matrix stuff out to
obs_source_draw_set_color_matrix feels a lot more clean.
On top of that, having the ability to set the position would be nice to
have as well, rather than having to mess with the matrix stuff each
time, so I also added that for the sake of convenience.
obs_source_draw will draw a texture sprite, optionally of a specific
size and/or at a specific position, as well as optionally inverted. The
texture used will be set to the 'image' parameter of whatever effect is
currently active.
obs_source_draw_set_color_matrix will set the color matrix value if the
drawing requires color matrices. It will set the 'color_matrix',
'color_range_min', and 'color_range_max' parameters of whatever effect
is currently active.
Overall, these feel much more clean to use than the previous iteration.
This function simplifies drawing textures for sources in order to help
reduce boilerplate code. If a source is a custom drawn source, it will
automatically set up the effect to draw the sprite. If it's not a
custom drawn source, it will simply draw the sprite as per normal. If
the source uses a specific color matrix, it will also handle that as
well.
This moves the 'flags' variable from the obs_source_frame structure to
the obs_source structure, and allows user flags to be set for a specific
source. Having it set on the obs_source_frame structure didn't make
much sense.
OBS_SOURCE_UNBUFFERED makes it so that the source does not buffer its
async video output in order to try to play it on time. In other words,
frames are played as soon as possible after being received.
Useful when you want a source to play back as quickly as possible
(webcams, certain types of capture devices)
This reverts commit c3f4b0f018.
The obs_source_frame should not need to take flags to do this. This
shouldn't be a setting associated with the frame, but rather a setting
associated with the source itself. This was the wrong approach to
solving this particular problem.
If audio timestamps are within the operating system timing threshold,
always use those values directly as a timestamp, and do not apply the
regular jump checks and timing adjustments that normally occur.
This potentially fixes an issue with plugins that use OS timestamps
directly as timestamp values for their audio samples, and bypasses the
timing conversions to system time for the audio line and uses it
directly as the timestamp value. It prevents those calculations from
potentially affecting the audio timestamp value when OS timestamps are
used.
For example, if the first set of audio samples from the audio source
came in delayed, while the subsequent samples were not delayed, those
first samples could have potentially inadvertently triggered the timing
adjustments, which would affect all subsequent audio samples.
This combines the 'direct' timestamp variance threshold with the maximum
timestamp jump threshold (or rather just removes the max timestamp jump
threshold and uses the timestamp variance threshold for both timestamp
jumps and detecting timestamps).
The reason why this was done was because a timestamp jump could occur at
a higher threshold than the threshold used for detecting OS timestamps
within a certain threshold. If timestamps got between those two
thresholds it kind of became a weird situation where timestamps could be
sort of 'stuck' back or forward in time more than intended. Better to
be consistent and use the same threshold for both values.
Add 'flags' member variable to obs_source_frame structure.
The OBS_VIDEO_UNBUFFERED flags causes the video to play back as soon as
it's received (in the next frame playback), causing it to disregard the
timestamp value for the sake of video playback (however, note that the
video timestamp is still used for audio synchronization if audio is
present on the source as well).
This is partly a convenience feature, and partly a necessity for certain
plugins (such as the linux v4l plugin) where timestamp information for
the video frames can sometimes be unreliable.
Due to a small error in the timestamp smoothing code the timestamp of
audio packages that were too early was always set to the next expected
timestamp, even if the difference was bigger than the smoothing threshold.
This would cause obs to simply append all audio data to the buffer even if
the real timestamp was way smaller than the next that was expected.
This should reduce corruption problems with for example the pulseaudio
plugin, which resends data under certain conditions.
obs_source_update_properties should be called by sources when property
values change, e.g. a capture device source would use this when it
detects a new capture device (in case its properties contain a list of
available capture devices or similar)
This Fixes a minor flaw with the API where data had to always be mutable
to be usable by the API.
Functions that do not modify the fundamental underlying data of a
structure should be marked as constant, both for safety and to signify
that the parameter is input only and will not be modified by the
function using it.
Typedef pointers are unsafe. If you do:
typedef struct bla *bla_t;
then you cannot use it as a constant, such as: const bla_t, because
that constant will be to the pointer itself rather than to the
underlying data. I admit this was a fundamental mistake that must
be corrected.
All typedefs that were pointer types will now have their pointers
removed from the type itself, and the pointers will be used when they
are actually used as variables/parameters/returns instead.
This does not break ABI though, which is pretty nice.
This helps ensure that an asynchronous video source is played as close
to its framerate as possible, reduces the risk of duplication as
much as possible, and helps to ensure that playback is as smooth as
possible.
This prevents multiple needless calls to obs_source_get_frame and other
functions. If the texture has already been processed, then just render
it as-is in any subsequent calls to obs_source_video_render.
This is actually unnecessary now that there's a hard limit on the
maximum offset in which audio can be inserted.
This also assumes too much about the audio; it assumes audio is always
on, where as with some devices (such as the elgato) audio is not on
until the stream starts, and when the video has already incremented the
counter.
If the audio didn't start at the 0 timestamp, it would misinterpret it
as a timestamp jump because obs_source::next_audio_ts_min is set to 0 on
creation. Timestamp starting values should be allowed to start at any
arbitrary value.
Instead of having functions like obs_signal_handler() that can fail to
properly specify their actual intent in the name (does it signal a
handler, or does it return a signal handler?), always prefix functions
that are meant to get information with 'get' to make its functionality
more explicit.
Previous names: New names:
-----------------------------------------------------------
obs_audio obs_get_audio
obs_video obs_get_video
obs_signalhandler obs_get_signal_handler
obs_prochandler obs_get_proc_handler
obs_source_signalhandler obs_source_get_signal_handler
obs_source_prochandler obs_source_get_proc_handler
obs_output_signalhandler obs_output_get_signal_handler
obs_output_prochandler obs_output_get_proc_handler
obs_service_signalhandler obs_service_get_signal_handler
obs_service_prochandler obs_service_get_proc_handler
Changed:
- obs_source_gettype
To:
- enum obs_source_type obs_source_get_type(obs_source_t source);
- const char *obs_source_get_id(obs_source_t source);
This function was inconsistent for a number of reasons. First, it
returns both the ID and the type of source (input/transition/filter),
which is inconsistent with the name of "get type". Secondly, the
'squishy' naming convention which has just turned out to be bad
practice and causes inconsistencies. So it's now replaced with two
functions that just return the type and the ID.
Prefix with obs_ for the sake of consistency
Renamed enums:
- order_movement (now obs_order_movement)
Affected functions:
- obs_source_filter_setorder
- obs_sceneitem_setorder
Renamed functions:
- obs_source_getframe (rename to obs_source_get_frame)
- obs_source_releaseframe (rename to obs_source_release_frame)
For the sake of consistency and helping to get rid of the "squishy
function name" issue
With the recent change to module handling by BtbN, I felt that having
this information might be useful in case someone is actually using make
install to set up their libraries.
NOTE: In texture_setimage, I had to move variables to the top of the
scope because microsoft's C compiler will give the legacy C90 error of:
'illegal use of this type as an expression'.
To sum it up, microsoft's C compiler is still utter garbage.
Similar to the shader functions, the effect parameter functions take
the effect as a parameter. However, the effect parameter is pretty
pointless, because the effect parameter.. parameter stores the effect
pointer interally.
The locale parameter was a mistake, because it puts extra needless
burden upon the module developer to have to handle this variable for
each and every single callback function. The parameter is being removed
in favor of a single centralized module callback function that
specifically updates locale information for a module only when needed.
This replaces the older code which simply queried the max volume level
value for any given audio.
I'm still not 100% sure on if this is how I want to approach the
problem, particularly, whether this should be done in obs_source or in
audio_line, but it can always be moved later if needed.
This uses the calculations by the awesome Bill Hamilton that OBS1 used
for its volume levels. It calculates the current max (level),
magnitude, and current peak. This data then can be used to create
awesome volume meter controls later on.
NOTE: Will probably need optimization, does one float at a time right
now.
Also, change some of the naming conventions. I actually need to change
a lot of the naming conventions in general so that all words are
separated by underscores. Kind of a bad practice there on my part.
When a source's private data is being created by a module, it wasn't
able to call most source functions because most functions rely on the
obs_source_info part of the context to be set. This fixes that issue.
It was strange that this wasn't already the case because the other
context types already did the same thing.
This uses the reverse planar YUV 4:2:0 conversion shader to output a YUV
texture without having to convert it via CPU. Again, this will reduce
video upload bandwidth usage to 37.5% of the original rate. I suspect
this will be particularly useful for when an FFmpeg or libav input
plugin for playing videos is made.
NOTE: There's an issue with certain texture sizes right now I haven't
been able to identify, if the full size of texture data divided by the
base texture width is an uneven number, the V chroma plane seems like it
can potentially shift, though I only had this happen with 160x90
resolution C920. Almost all resolutions tend to be even. Needs further
testing with more devices that support planar YUV 4:2:0 output.
First, if the private data of the source fails to be created, then do
not destroy the source. If the source is destroyed, all the user's data
associated with that source is lost, which could end up being a
potential problem. Instead, let it linger as a 'dead' source until the
user chooses to fix the problem (though this should never really happen,
the source module functions should be programmed to handle this
scenario)
Secondly, rename new_frame_ready to ready_async_frame, and fix a
potential memory leak with it.
obs_source_output_video can cause cached frames to be freed twice if
called with a partially destroyed source, among other undesirable
effects; freeing the source private data right after the destroy signal
has been processed ensures proper behavior
- Add volume control
These volume controls are basically nothing more than sliders. They
look terrible and hopefully will be as temporary as they are
terrible.
- Allow saving of specific non-user sources via obs_load_source and
obs_save_source functions.
- Save data of desktop/mic audio sources (sync data, volume data, etc),
and load the data on startup.
- Make it so that a scene is created by default if first time using the
application. On certain operating systems where supported, a default
capture will be created. Desktop capture on mac, particularly. Not
sure what to do about windows because monitor capture on windows 7 is
completely terrible and is bad to start users off with.
If a source with async video wasn't currently active, it would endlessly
buffer the video data, which would cause memory to grow endlessly until
available memory was extinguished.
This really needs to be replaced with a proper caching mechanism at some
point.
This saves scenes/sources from json on exit, and properly loads it back
up when starting up the program again, as well as the currently active
scene.
I had to add a 'load' and 'save' callback to the source interface
structure because I realizes that certain sources (such as scenes)
operate different with their saved data; scenes for example would have
to keep track of their settings information constantly, and that was
somewhat unacceptable to make it functional.
The optional 'load' callback will be called only after having loaded
setttings specifically from file/imported data, and the 'save' function
will be called only specifically when data actually needs to be saved.
I also had to adjust the obs_scene code so that it's a regular input
source type now, and I also modified it so that it doesn't have some
strange custom creation code anymore. The obs_scene_create function is
now simply just a wrapper for obs_source_create. You could even create
a scene with obs_source_create manually as well.
Add API for streaming services. The services API simplifies the
creation of custom service features and user interface.
Custom streaming services later on will be able to do things such as:
- Be able to use service-specific APIs via modules, allowing a more
direct means of communicating with the service and requesting or
setting service-specific information
- Get URL/stream key via other means of authentication such as OAuth,
or be able to build custom URLs for services that require that sort
of thing.
- Query information (such as viewer count, chat, follower
notifications, and other information)
- Set channel information (such as current game, current channel title,
activating commercials)
Also, I reduce some repeated code that was used for all libobs objects.
This includes the name of the object, the private data, settings, as
well as the signal and procedure handlers.
I also switched to using linked lists for the global object lists,
rather than using an array of pointers (you could say it was..
pointless.) ..Anyway, the linked list info is also stored in the shared
context data structure.
Before, async video sources would flicker because they were only being
drawn when they were updated. So when updated, they'd draw that frame,
then it would stop drawing it until it updated again. This fixes that
issue and they should now draw properly.
Also, fix a few other minor bugs and issues relating to async video,
and make it so that non-async video filters can be properly applied to
them.
For the purposes of testing, change the 'test-random' source to an async
video source that updates every quarter of a second with a new random
face.
Also fix a bug where non-async video sources wouldn't have filter
effects applied properly.
- Implement the RTMP output module. This time around, we just use a
simple FLV muxer, then just write to the stream with RTMP_Write.
Easy and effective.
- Fix the FLV muxer, the muxer now outputs proper FLV packets.
- Output API:
* When using encoders, automatically interleave encoded packets
before sending it to the output.
* Pair encoders and have them automatically wait for the other to
start to ensure sync.
* Change 'obs_output_signal_start_fail' to 'obs_output_signal_stop'
because it was a bit confusing, and doing this makes a lot more
sense for outputs that need to stop suddenly (disconnections/etc).
- Encoder API:
* Remove some unnecessary encoder functions from the actual API and
make them internal. Most of the encoder functions are handled
automatically by outputs anyway, so there's no real need to expose
them and end up inadvertently confusing plugin writers.
* Have audio encoders wait for the video encoder to get a frame, then
start at the exact data point that the first video frame starts to
ensure the most accrate sync of video/audio possible.
* Add a required 'frame_size' callback for audio encoders that
returns the expected number of frames desired to encode with. This
way, the libobs encoder API can handle the circular buffering
internally automatically for the encoder modules, so encoder
writers don't have to do it themselves.
- Fix a few bugs in the serializer interface. It was passing the wrong
variable for the data in a few cases.
- If a source has video, make obs_source_update defer the actual update
callback until the tick function is called to prevent threading
issues.
Improve the properties API so that it can actually respond somewhat to
user input. Maybe later this might be further improved or replaced with
something script-based.
When creating a property, you can now add a callback to that property
that notifies when the property has been changed in the user interface.
Return true if you want the properties to be refreshed, or false if not.
Though now that I think about it I doubt there would ever be a case
where you would have this callback and *not* refresh the properties.
Regardless, this allows functions to change the values of properties or
settings, or enable/disable/hide other property controls from view
dynamically.
- Add a properties window for sources so that you can now actually edit
the settings for sources. Also, display the source by itself in the
window (Note: not working on mac, and possibly not working on linux).
When changing the settings for a source, it will call
obs_source_update on that source when you have modified any values
automatically.
- Add a properties 'widget', eventually I want to turn this in to a
regular nice properties view like you'd see in the designer, but
right now it just uses a form layout in a QScrollArea with regular
controls to display the properties. It's clunky but works for the
time being.
- Make it so that swap chains and the main graphics subsystem will
automatically use at least one backbuffer if none was specified
- Fix bug where displays weren't added to the main display array
- Make it so that you can get the properties of a source via the actual
pointer of a source/encoder/output in addition to being able to look
up properties via identifier.
- When registering source types, check for required functions (wasn't
doing it before). getheight/getwidth should not be optional if it's
a video source as well.
- Add an RAII OBSObj wrapper to obs.hpp for non-reference-counted
libobs pointers
- Add an RAII OBSSignal wrapper to obs.hpp for libobs signals to
automatically disconnect them on destruction
- Move the "scale and center" calculation in window-basic-main.cpp to
its own function and in its own source file
- Add an 'update' callback to WASAPI audio sources
Also, rename atomic functions to be consistent with the rest of the
platform/threading functions, and move atomic functions to threading*
files rather than platform* files
Ensure that a source has a valid name. Duplicates aren't a big deal
internally, but sources without a name are probably something that
should be avoided. Made is so that if a source is programmatically
created without a name, it's assigned an index based name.
In the main basic-mode window, made it check to make sure the name was
valid as well.
- Implement a means of obtaining default settings for an
input/output/encoder. obs_source_defaults for example will return
the default settings for a particular source type.
- Because C++ doesn't have designated initializers, use functions in
the WASAPI plugin to register the sources instead.
- Split input and output audio captures so that they're different
sources. This allows easier handling and enumeration of audio
devices without having to do some sort of string processing.
This way the user interface code can handle this a bit more easily,
and so that it doesn't confuse users either. This should be done for
all audio capture sources for all operating systems. You don't have
to duplicate any code, you just need to create input/output wrapper
functions to designate the audio as input or output before creation.
- Make it detect soundflower and wavtap devices as mac "output" devices
(even though they're actually input) for the mac output capture, and
make it so that users can select a default output capture and
automatically use soundflower or wavtap.
I'm not entirely happy about having to do this, but because mac is
designed this way, this is really the only way to handle it that
makes it easier for users and UI code to deal with.
Note that soundflower and wavtap are still also designated as input
devices, so will still show up in input device enumeration.
- Remove pragma messages because they were kind polluting the other
compiler messages and just getting in the way. In the future we can
just do a grep for TODO to find them.
- Redo list property again, this time using a safer internal array,
rather than requiring sketchy array inputs. Having functions handle
everything behind the scenes is much safer.
- Remove the reference counter debug log code, as it was included
unintentionally in a commit.
- Signals and dynamic callbacks now require declarations to be made
before being used. What this does is allows us to get information
about the functions dynamically which can be relayed to the user and
plugins for future extended usage (this should have big implications
later for scripting in particular, hopefully).
- Reduced the number of types calldata uses from "everything I could
think of" to simply integer, float, bool, pointer/object, string.
Integer data is now stored as long long. Floats are now stored as
doubles (check em).
- Use a more consistent naming scheme for lexer error/warning macros.
- Fixed a rather nasty bug where switching to an existing scene would
cause it to increment sourceSceneRefs, which would mean that it would
never end up never properly removing the source when the user clicks
removed (stayed in limbo, obs_source_remove never got called)
LOG_ERROR should be used in places where though recoverable (or at least
something that can be handled safely), was unexpected, and may affect
the user/application.
LOG_WARNING should be used in places where it's not entirely unexpected,
is recoverable, and doesn't really affect the user/application.
Split off activate to activate and show callbacks, and split off
deactivate to deactivate and hide callbacks. Sources didn't previously
have a means to know whether it was actually being displayed in the main
view or just happened to be visible somewhere. Now, for things like
transition sources, they have a means of knowing when they have actually
been "activated" so they can initiate their sequence.
A source is now only considered "active" when it's being displayed by
the main view. When a source is shown in the main view, the activate
callback/signal is triggered. When it's no longer being displayed by
the main view, deactivate callback/signal is triggered.
When a source is just generally visible to see by any view, the show
callback/signal is triggered. If it's no longer visible by any views,
then the hide callback/signal is triggered.
Presentation volume will now only be active when a source is active in
the main view rather than also in auxilary views.
Also fix a potential bug where parents wouldn't properly increment or
decrement all the activation references of a child source when a child
was added or removed.
This allows the changing of bideo settings without having to completely
reset all graphics data. Will recreate internal output/conversion
buffers and such and reset the main preview.
Make it so obs_data settings input in to *_update are applied to the
existing settings rather than fully replace the existing settings. That
way you can update with only certain specific settings, leaving other
settings untouched. Of course if you're already using the original
settings pointer in the first place then you've already done that, so
it'll just ignore it because you've already applied them.
- Remove obs_source::type because it became redundant now that the
type is always stored in the obs_source::info variable.
- Apply presentation volumes of 1.0 and 0.0 to sources when they
activate/deactivate, respectively. It also applies that presentation
volume to all sub-sources, with exception of transition sources.
Transition sources must apply presentation volume manually to their
sub-sources with the new transition functions below.
- Add a "transition_volume" variable to obs_source structure, and add
three functions for handling volume for transitions:
* obs_transition_begin_frame
* obs_source_set_transition_vol
* obs_transition_end_frame
Because the to/from targets of a transition source might both contain
some of the same sources, handling the transitioning of volumes for
that specific situation becomes an issue.
So for transitions, instead of modifying the presentation volumes
directly for both sets of sources, we do this:
- First, call obs_transition_begin_frame at the beginning of each
transition frame, which will reset transition volumes for all
sub-sources to 0. Presentation volumes remain unchanged.
- Call obs_source_set_transition_vol on each sub-source, which will
then add the volume to the transition volume for each source in
that source's tree. Presentation volumes still remain unchanged.
- Then you call obs_trandition_end_frame when complete, which will
then finally set the presentation volumes to the transition
volumes.
For example, let's say that there's one source that's within both the
"transitioning from" sources and "transition to" sources. It would
add both the fade in and fade out volumes to that source, and then
when the frame is complete, it would set the presentation volume to
the sum of those two values, rather than set the presentation volume
for that same source twice which would cause weird volume jittering
and also set the wrong values.
Now sources will be properly activated and deactivated when they are in
use or not in use.
Had to figure out a way to handle child sources, and children of
children, just ended up implementing simple functions that parents use
to signal adding/removal to help with hierarchial activation and
deactivation of child sources.
To prevent the source activate/deactivate callbacks from being called
more than once, added an activation reference counter. The first
increment will call the activate callback, and the last decrement will
call the deactivate callback.
Added "source-activate" and "source-deactivate" signals to the main obs
signal handler, and "activate" and "deactivate" to individual source
signal handlers.
Also, fixed the main window so it properly selects a source when the
current active scene has been changed.
Added a "master" volume for the entire audio subsystem.
Also, added a "presentation" volume for both the master volume and for
each invidiaul source. The presentation volume is used to control
things like transitioning volumes, preventing sources from outputting
any audio when they're inactive, as well as some other uses in the
future.
If audio was under, it originally did a full reset of the audio timing.
However, resetting the audio timing when this happens is kind of a bad
thing. It's better just to clamp the value to the expected timestamp to
ensure seamless audio output.
Also, implement audio timestamp smoothing to ensure audio tries to be as
seamless as possible.
Add a scaler interface (defaults to swscale), and if a separate output
wants to use a different scale or format than the default output format,
allow a scaler instance to be created automatically for that output,
which will then receive the new scaled output.
There were a *lot* of warnings, managed to remove most of them.
Also, put warning flags before C_FLAGS and CXX_FLAGS, rather than after,
as -Wall -Wextra was overwriting flags that came before it.
The API used to be designed in such a way to where it would expect
exports for each individual source/output/encoder/etc. You would export
functions for each and it would automatically load those functions based
on a specific naming scheme from the module.
The idea behind this was that I wanted to limit the usage of structures
in the API so only functions could be used. It was an interesting idea
in theory, but this idea turned out to be flawed in a number of ways:
1.) Requiring exports to create sources/outputs/encoders/etc meant that
you could not create them by any other means, which meant that
things like faruton's .net plugin would become difficult.
2.) Export function declarations could not be checked, therefore if you
created a function with the wrong parameters and parameter types,
the compiler wouldn't know how to check for that.
3.) Required overly complex load functions in libobs just to handle it.
It makes much more sense to just have a load function that you call
manually. Complexity is the bane of all good programs.
4.) It required that you have functions of specific names, which looked
and felt somewhat unsightly.
So, to fix these issues, I replaced it with a more commonly used API
scheme, seen commonly in places like kernels and typical C libraries
with abstraction. You simply create a structure that contains the
callback definitions, and you pass it to a function to register that
definition (such as obs_register_source), which you call in the
obs_module_load of the module.
It will also automatically check the structure size and ensure that it
only loads the required values if the structure happened to add new
values in an API change.
The "main" source file for each module must include obs-module.h, and
must use OBS_DECLARE_MODULE() within that source file.
Also, started writing some doxygen documentation in to the main library
headers. Will add more detailed documentation as I go.
- Fill in the rest of the FFmpeg test output code for testing so it
actually properly outputs data.
- Improve the main video subsystem to be a bit more optimal and
automatically output I420 or NV12 if needed.
- Fix audio subsystem insertation and byte calculation. Now it will
seamlessly insert new audio data in to the audio stream based upon
its timestamp value. (Be extremely cautious when using floating
point calculations for important things like this, and always round
your values and check your values)
- Use 32 byte alignment in case of future optimizations and export a
function to get the current alignment.
- Make os_sleepto_ns return true if slept, false if the time has
already been passed before the call.
- Fix sinewave output so that it actually properly calculates a middle
C sinewave.
- Change the use of row_bytes to linesize (also makes it a bit more
consistent with FFmpeg's naming as well)
- Add planar audio support. FFmpeg and libav use planar audio for many
encoders, so it was somewhat necessary to add support in libobs
itself.
- Improve/adjust FFmpeg test output plugin. The exports were somewhat
messed up (making me rethink how exports should be done). Not yet
functional; it handles video properly, but it still does not handle
audio properly.
- Improve planar video code. The planar video code was not properly
accounting for row sizes for each plane. Specifying row sizes for
each plane has now been added. This will also make it more compatible
with FFmpeg/libav.
- Fixed a bug where callbacks wouldn't create properly in audio-io and
video-io code.
- Implement 'blogva' function to allow for va_list usage with libobs
logging.
- Implement texture scaling/conversion/downloading for the main view so
we can finally start getting data to output.
Also, redesign how it works a bit, it will now properly wait one full
frame for each step in the process: rendering the main texture,
scaling the main texture to an output texture, staging/downloading the
ouput texture, and then outputting that staged data. This way, the
GPU will have more than enough time to fully complete each step.
- Fix a bug with OpenGL plugin's texture staging function. Was using
glBindBuffer instead of what should have been used: glBindTexture.
- Change the naming scheme of the variables in default.effect. It's now
named with the idea of just "color matrix" in mind instead of "yuv
matrix", and instead of DrawRGBToYUV, it's now just DrawMatrix.
- Implemented better C++ classes for handling scenes/sources/items in
obs.hpp, allowing them to automatically increment and decrement the
references of each, as well as assign them to QVariants.
- Because QVariants are now using the C++ classes, remove the pointer
QVariant wrapper.
- Use the new C++ classes with the QVariant user data of list box items,
both for the sake of thread safety and to ensure that the data
referenced is not freed until removed. NOTE: still might need some
testing.
- Implemented a source-remove signal from libobs, and start using that
signal instead of the source-destroy signal for signalling item
removal.
jp9000
Richard Stanway
kc5nra
Palana
fryshorts
HomeWorld
Danni