...The reason why audio didn't work was because I overwrote the bitrate
values.
As for semaphores, mac doesn't support unnamed semaphores without using
mach semaphores. So, I just implemented a semaphore wrapper for each
OS.
- Add some temporary streaming code using FFmpeg. FFmpeg itself is not
very ideal for streaming; lack of direct control of the sockets and
no framedrop handling means that FFmpeg is definitely not something
you want to use without wrapper code. I'd prefer writing my own
network framework in this particular case just because you give away
so much control of the network interface. Wasted an entire day
trying to go through FFmpeg issues.
There's just no way FFmpeg should be used for real streaming (at
least without being patched or submitting some sort of patch, but I'm
sort of feeling "meh" on that idea)
I had to end up writing multiple threads just to handle both
connecting and writing, because av_interleaved_write_frame blocks
every call, stalling the main encoder thread, and thus also stalling
draw signals.
- Add some temporary user interface for streaming settings. This is
just temporary for the time being. It's in the outputs section of
the basic-mode settings
- Make it so that dynamic arrays do not free all their data when the
size just happens to be reduced to 0. This prevents constant
reallocation when an array keeps going from 1 item to 0 items. Also,
it was bad to become dependent upon that functionality. You must now
always explicitly call "free" on it to ensure the data is free, and
that's how it should be. Implicit functionality can lead to
confusion and maintainability issues.
- Fix a bug where the initial audio data insertion would cause all
audio data to unintentionally clear (mixed up < and > operators, damn
human error)
- Fixed a potential interdependant lock scenario with channel mutex
locks and graphics mutex locks. The main video thread could lock the
graphics mutex and then while in the graphics mutex could lock the
channels mutex. Meanwhile in another thread, the channel mutex could
get locked, and then the graphics mutex would get locked, causing a
deadlock.
The best way to deal with this is to not let mutexes lock within
other mutexes, but sometimes it's difficult to avoid such as in the
main video thread.
- Audio devices should now be functional, and the devices in the audio
settings can now be changed as desired.
- Implement a means of obtaining default settings for an
input/output/encoder. obs_source_defaults for example will return
the default settings for a particular source type.
- Because C++ doesn't have designated initializers, use functions in
the WASAPI plugin to register the sources instead.
- Implement windows monitor capture (code is so much cleaner than in
OBS1). Will implement duplication capture later
- Add GDI texture support to d3d11 graphics library
- Fix precision issue with sleep timing, you have to call
timeBeginPeriod otherwise windows sleep will be totally erratic.
- Add WASAPI audio capture for windows, input and output
- Check for null pointer in os_dlopen
- Add exception-safe 'WinHandle' and 'CoTaskMemPtr' helper classes that
will automatically call CloseHandle on handles and call CoTaskMemFree
on certain types of memory returned from windows functions
- Changed the wide <-> MBS/UTF8 conversion functions so that you use
buffers (like these functions are *supposed* to behave), and changed
the ones that allocate to a different naming scheme to be safe
- Split input and output audio captures so that they're different
sources. This allows easier handling and enumeration of audio
devices without having to do some sort of string processing.
This way the user interface code can handle this a bit more easily,
and so that it doesn't confuse users either. This should be done for
all audio capture sources for all operating systems. You don't have
to duplicate any code, you just need to create input/output wrapper
functions to designate the audio as input or output before creation.
- Make it detect soundflower and wavtap devices as mac "output" devices
(even though they're actually input) for the mac output capture, and
make it so that users can select a default output capture and
automatically use soundflower or wavtap.
I'm not entirely happy about having to do this, but because mac is
designed this way, this is really the only way to handle it that
makes it easier for users and UI code to deal with.
Note that soundflower and wavtap are still also designated as input
devices, so will still show up in input device enumeration.
- Remove pragma messages because they were kind polluting the other
compiler messages and just getting in the way. In the future we can
just do a grep for TODO to find them.
- Redo list property again, this time using a safer internal array,
rather than requiring sketchy array inputs. Having functions handle
everything behind the scenes is much safer.
- Remove the reference counter debug log code, as it was included
unintentionally in a commit.
If the default device changes, set the reconnect interval to 200
milliseconds so it pretty much immediately tries to reinitialize the
audio with the newly selected default device. Otherwise, use 2000
millisecond intervals, and assume disconnection.
Also, reduced FFmpeg logging to just regular FFmpeg information rather
than everything FFmpeg logs.
LOG_ERROR should be used in places where though recoverable (or at least
something that can be handled safely), was unexpected, and may affect
the user/application.
LOG_WARNING should be used in places where it's not entirely unexpected,
is recoverable, and doesn't really affect the user/application.
I can't believe I wasn't doing this. This is why file output was
getting corrupted. Audio and video send in data from separate threads.
I should be embarassed for not having considered that.
Key lesson: Increase threading paranoia levels. Apparently my
threading paranoid levels are lackluster.
Actually, if initializing failed at all, it would never properly
uninitialize because the 'initialized' variable was never set until the
very end. Instead, set the "initialized" flag from the beginning to
ensure initialization.
- Add CoreAudio device input capture for mac audio capturing. The code
should cover just about everything for capturing mac input device
audio. Because of the way mac audio is designed, users may have no
choice but to obtain the open source soundflower software to capture
their mac's desktop audio. It may be necessary for us to distribute
it with the program as well.
- Hide event backend
- Use win32 events for windows
- Allow timed waits for events
- Fix a few warnings
FFmpeg test output wasn't make any attempt to sync data before. Should
be much more accurate now.
Also, added a restart message to audio settings if base audio settings
are changed.
Implement a few audio options in to the user interface as well as a few
inline audio functions in audio-io.h.
Make it so ffmpeg plugin automatically converts to the desired format.
Use regular interleaved float internally for audio instead of planar
float.
Add a scaler interface (defaults to swscale), and if a separate output
wants to use a different scale or format than the default output format,
allow a scaler instance to be created automatically for that output,
which will then receive the new scaled output.
If there are for example more than one audio outputs and they have
different sample rates or channels and such, this will allow automatic
conversion of that audio to the request formats/channels/rates (but only
if requested).
- Changed glMapBuffer to glMapBufferRange to allow invalidation. Using
just glMapBuffer alone was causing some unacceptable stalls.
- Changed dynamic buffers from GL_DYNAMIC_WRITE to GL_STREAM_WRITE
because I had misunderstood the OpenGL specification
- Added _OPENGL and _D3D11 builtin preprocessor macros to effects to
allow special processing if needed
- Added fmod support to shaders (NOTE: D3D and GL do not function
identically with negative numbers when using this. Positive numbers
however function identically)
- Created a planar conversion shader that converts from packed YUV to
planar 420 right on the GPU without any CPU processing. Reduces
required GPU download size to approximately 37.5% of its normal rate
as well. GPU usage down by 10 entire percentage points despite the
extra required pass.
There were a *lot* of warnings, managed to remove most of them.
Also, put warning flags before C_FLAGS and CXX_FLAGS, rather than after,
as -Wall -Wextra was overwriting flags that came before it.
The API used to be designed in such a way to where it would expect
exports for each individual source/output/encoder/etc. You would export
functions for each and it would automatically load those functions based
on a specific naming scheme from the module.
The idea behind this was that I wanted to limit the usage of structures
in the API so only functions could be used. It was an interesting idea
in theory, but this idea turned out to be flawed in a number of ways:
1.) Requiring exports to create sources/outputs/encoders/etc meant that
you could not create them by any other means, which meant that
things like faruton's .net plugin would become difficult.
2.) Export function declarations could not be checked, therefore if you
created a function with the wrong parameters and parameter types,
the compiler wouldn't know how to check for that.
3.) Required overly complex load functions in libobs just to handle it.
It makes much more sense to just have a load function that you call
manually. Complexity is the bane of all good programs.
4.) It required that you have functions of specific names, which looked
and felt somewhat unsightly.
So, to fix these issues, I replaced it with a more commonly used API
scheme, seen commonly in places like kernels and typical C libraries
with abstraction. You simply create a structure that contains the
callback definitions, and you pass it to a function to register that
definition (such as obs_register_source), which you call in the
obs_module_load of the module.
It will also automatically check the structure size and ensure that it
only loads the required values if the structure happened to add new
values in an API change.
The "main" source file for each module must include obs-module.h, and
must use OBS_DECLARE_MODULE() within that source file.
Also, started writing some doxygen documentation in to the main library
headers. Will add more detailed documentation as I go.
- Fill in the rest of the FFmpeg test output code for testing so it
actually properly outputs data.
- Improve the main video subsystem to be a bit more optimal and
automatically output I420 or NV12 if needed.
- Fix audio subsystem insertation and byte calculation. Now it will
seamlessly insert new audio data in to the audio stream based upon
its timestamp value. (Be extremely cautious when using floating
point calculations for important things like this, and always round
your values and check your values)
- Use 32 byte alignment in case of future optimizations and export a
function to get the current alignment.
- Make os_sleepto_ns return true if slept, false if the time has
already been passed before the call.
- Fix sinewave output so that it actually properly calculates a middle
C sinewave.
- Change the use of row_bytes to linesize (also makes it a bit more
consistent with FFmpeg's naming as well)
- Add planar audio support. FFmpeg and libav use planar audio for many
encoders, so it was somewhat necessary to add support in libobs
itself.
- Improve/adjust FFmpeg test output plugin. The exports were somewhat
messed up (making me rethink how exports should be done). Not yet
functional; it handles video properly, but it still does not handle
audio properly.
- Improve planar video code. The planar video code was not properly
accounting for row sizes for each plane. Specifying row sizes for
each plane has now been added. This will also make it more compatible
with FFmpeg/libav.
- Fixed a bug where callbacks wouldn't create properly in audio-io and
video-io code.
- Implement 'blogva' function to allow for va_list usage with libobs
logging.
Add a fairly easy to use settings interface that can be passed to
plugins, and replaced the old character string system that was being
used before. The new data interface allows for an easier method of
getting/altering settings for plugins, and is built to be serializable
to/from JSON.
Also, removed another wxFormBuilder file that was no longer in use.
- Added some code for FFmpeg output that I'm still playing around with.
Right now I'm just trying to get it to output to file and try to
understand the FFmpeg/libav APIs. Hopefully in the future this plugin
can be used for any sort of output to FFmpeg.
- Fixed a cast warning in audio-io.c with size_t -> uint32_t
- Renamed the 'video_info' and 'audio_info' structures to
'video_conver_info' and 'audio_convert_info' to better represent their
actual purpose, and to avoid confusion with 'audio_output_info' and
'video_output_info' structures.
- Removed a few macros from obs-def.h that were at one point going to be
used but no longer going to be used (at least for now)
- First, I redid the output interface for libobs. I feel like it's
going in a pretty good direction in terms of design.
Right now, the design is so that outputs and encoders are separate.
One or more outputs can connect to a specific encoder to receive its
data, or the output can connect directly to raw data from libobs
output itself, if the output doesn't want to use a designated encoder.
Data is received via callbacks set when you connect to the encoder or
raw output. Multiple outputs can receive the data from a single
encoder context if need be (such as for streaming to multiple channels
at once, and/or recording with the same data).
When an encoder is first connected to, it will connect to raw output,
and start encoding. Additional connections will receive that same
data being encoded as well after that. When the last encoder has
disconnected, it will stop encoding. If for some reason the encoder
needs to stop, it will use the callback with NULL to signal that
encoding has stopped. Some of these things may be subject to change
in the future, though it feels pretty good with this design so far.
Will have to see how well it works out in practice versus theory.
- Second, Started adding preliminary RTMP/x264 output plugin code.
To speed things up, I might just make a direct raw->FFmpeg output to
create a quick output plugin that we can start using for testing all
the subsystems.
Completely revamped the entire media i/o data and handlers. The
original idea was to have a system that would have connecting media
inputs and outputs, but at a certain point I realized that this was an
unnecessary complexity for what we wanted to do. (Also, it reminded me
of directshow filters, and I HATE directshow with a passion, and
wouldn't wish it upon my greatest enemy)
Now, audio/video outputs are connected to directly, with better callback
handlers, and will eventually have the ability to automatically handle
conversions such as 4:4:4 to 4:2:0 when connecting to an input that uses
them. Doing this will allow the video/audio i/o handlers to also
prevent duplicate conversion, as well as make it easier/simple to use.
My true goal for this is to make output and encoder plugins as simple to
create as possible. I want to be able to be able to create an output
plugin with almost no real hassle of having to worry about image
conversions, media inputs/outputs, etc. A plugin developer shouldn't
have to handle that sort of stuff when he/she doesn't really need to.
Plugins will be able to simply create a callback via obs_video() and/or
obs_audio(), and they will automatically receive the audio/video data in
the formats requested via a simple callback, without needing to do
almost anything else at all.
jp9000
Palana
BtbN