Until a proper fix is found to support devices that use stepwise
or continuous values for framesize and framerate this adds fixed
values that can be selected.
For devices that support discrete values those are still queried
and used.
Autoselection is only implemented for frame rate as it doesn't make
sense to autoselect the video format (Any fills the role of library
supplied autoselection) or the resolution (it may make sense to
implement resolution matching based on the current transformation
settings in the future though)
Use warn/info/debug helper macro functions to output
warnings/information/debug log data that's preformatted to include the
module and the current output name.
NOTE: In texture_setimage, I had to move variables to the top of the
scope because microsoft's C compiler will give the legacy C90 error of:
'illegal use of this type as an expression'.
To sum it up, microsoft's C compiler is still utter garbage.
Instead of using strings for setting names, use macros. Macros prevent
mis-spellings, prevent usage of the wrong name, and if you get it wrong,
it will simply fail to compile, helping to ensure that the right setting
value is being used.
Similar to the shader functions, the effect parameter functions take
the effect as a parameter. However, the effect parameter is pretty
pointless, because the effect parameter.. parameter stores the effect
pointer interally.
Since adding a source now opens the config dialog for that source
the convenience gain from having a device automatically start capturing
is outweighed by the inconvenience from having captured images
unintentionally recorded/streamed
The locale parameter was a mistake, because it puts extra needless
burden upon the module developer to have to handle this variable for
each and every single callback function. The parameter is being removed
in favor of a single centralized module callback function that
specifically updates locale information for a module only when needed.
Having the value stored here is somewhat pointless, so this is one step
in fixing the locale handling. Locale should be handled by the modules
themselves with their own loaded locale lookup information.
Due to the recent change with matrices, the way space is represented by
matrices is a bit different. The matrix stack represents the view
matrix, rather than the model matrix, so the position is more
representative of the camera (view) itself, rather than that of the
objects (model).
Common services that require specific encoder settings (specified in the
'recommended' section of the json file) will now have those encoder
settings applied before the encoders start up.
This fixes the issue with services like twitch that weren't getting
their required 2-second keyframe maximum interval set.
Apparently, despite the fact that Apple added
kAudioHardwarePropertyTranslateUIDToDevice in 10.8, it's not actually
usable in 10.8, only 10.9. So, instead of being able to use it like a
normal, sane person, we have to enumerate all devices manually and find
the AudioDeviceID ourselves. A slight annoyance and a mark against
apple's competence, but audio devices should now be working again on
10.8 at least, so whatever.
The biggest problem with DirectShow is that available configuration
capabilities can change if you so much as look at it the wrong way.
Previously, configuring devices often didn't configure the device
settings correctly, you would choose one setting and then another
setting wouldn't be compatible with that settings.
Let's take the terrible microsoft lifecam series for example. First,
you'd be at 640x480 happily webcam'ing away, which is using the YUY2
format. Then you decide "hey, this webcam resolution is a bit low. I
would love to have it a bit high resolution so it's a bit more crisp and
clear." You'd select 1280x720, and then suddenly the only format
supported is MJPEG output. However, the interface has to update that
fact, it can't list YUY2 if MJPEG is the only one available for this
resolution. This doesn't just apply to formats either, this applies to
framerates and such as well. Some framerates will only be supported by
certain resolutions which can in turn only be supported by certain
formats.
This causes user interface for configuration to be really be a nightmare
to manage if you want these features to be available to the user. It's
extremely annoying because you have to update all the configuration UI
after something has changed, double check the configuration values, and
if the values aren't supported, update those configuration values.
This covers the basics of devices. Mostly functional but not at 100%
yet. Uses 'libdshowcapture' library to capture directshow video/audio.
Both libdshowcapture and the plugin still need some work. Should at
least capture basic webcams and capture cards for the time being.
Character conversion functions did not previously ask for a maximum
buffer size for their 'dst' parameter, it's unsafe to assume some given
destination buffer may have enough size to accommodate a conversion.
This doesn't add FLV file output to the user interface yet, but we'll
get around to that eventually. This just adds an FLV output type.
Also, removed ftello/fseeko because off_t is a really annoying data
type, and I'd rather have a firm int64_t for large sizes, so I named it
to os_fseeki64 and os_ftelli64 instead, and changed the file size
function to return an int64_t.
This patch adds support for multi-screen setups by using xinerama
information if available to determine the screen size and position.
In case xinerama is not available or not active it uses the information
provided for the x screen.
To clean up the code some generic helper functions were moved to a
separate source file.
- Add volume control
These volume controls are basically nothing more than sliders. They
look terrible and hopefully will be as temporary as they are
terrible.
- Allow saving of specific non-user sources via obs_load_source and
obs_save_source functions.
- Save data of desktop/mic audio sources (sync data, volume data, etc),
and load the data on startup.
- Make it so that a scene is created by default if first time using the
application. On certain operating systems where supported, a default
capture will be created. Desktop capture on mac, particularly. Not
sure what to do about windows because monitor capture on windows 7 is
completely terrible and is bad to start users off with.
If a source with async video wasn't currently active, it would endlessly
buffer the video data, which would cause memory to grow endlessly until
available memory was extinguished.
This really needs to be replaced with a proper caching mechanism at some
point.
With this option enabled the plugin will generate timestamps
based on the system time instead of using the ones provided by
pulse audio. This might fix problems with audio desync and may
become the default/only option in the future.
The defaults functions will now return the default device for the
input/output as provided by pulseaudio.
The default output device is the monitor of the default sink.
The wrapping library uses a global mainloop and context which
allows operations to share the connection. The global mainloop
is created and destroyed based on internal reference counting.
The capture code won't spawn a new thread for each input anymore
but instead just create the recording stream and rely on the
threaded mainloop to execute the read callback when data is available.
- Updated the services API so that it links up with an output and
the output gets data from that service rather than via settings.
This allows the service context to have control over how an output is
used, and makes it so that the URL/key/etc isn't necessarily some
static setting.
Also, if the service is attached to an output, it will stick around
until the output is destroyed.
- The settings interface has been updated so that it can allow the
usage of service plugins. What this means is that now you can create
a service plugin that can control aspects of the stream, and it
allows each service to create their own user interface if they create
a service plugin module.
- Testing out saving of current service information. Saves/loads from
JSON in to obs_data_t, seems to be working quite nicely, and the
service object information is saved/preserved on exit, and loaded
again on startup.
- I agonized over the settings user interface for days, and eventually
I just decided that the only way that users weren't going to be
fumbling over options was to split up the settings in to simple/basic
output, pre-configured, and then advanced for advanced use (such as
multiple outputs or services, which I'll implement later).
This was particularly painful to really design right, I wanted more
features and wanted to include everything in one interface but
ultimately just realized from experience that users are just not
technically knowledgable about it and will end up fumbling with the
settings rather than getting things done.
Basically, what this means is that casual users only have to enter in
about 3 things to configure their stream: Stream key, audio bitrate,
and video bitrate. I am really happy with this interface for those
types of users, but it definitely won't be sufficient for advanced
usage or for custom outputs, so that stuff will have to be separated.
- Improved the JSON usage for the 'common streaming services' context,
I realized that JSON arrays are there to ensure sorting, while
forgetting that general items are optimized for hashing. So
basically I'm just using arrays now to sort items in it.
This plugin is just a generic service plugin for basic RTMP streaming
service stuff.
This just has a 'common' service that has a list of common/simple
streaming services that don't have their own custom service modules, and
then a 'custom' service that allows you to enter in the stream URL and
key manually, without a service/server list.
Also, copy the jansson VS projects file (don't modify the old one) so
that it's located in the vs/2013 directory, so that other libraries can
properly link with it without having to enter in extra information just
to include jansson
A little bit of history about frame dropping:
I did a large number of experiments with frame dropping in old versions
of OBS1, and it's not an easy thing to deal with. I tried just about
everything from standard i-frame delay, to large buffers, to dumping
packets, to super-unnecessarily-complex things that just ended up
causing more problems than they was worth.
When I did my experiments, I found that the most ideal frame drop system
(in terms of reducing the amount of total data that needed to be
dropped) was in the 0.4xx days where I had a 3 second frame-drop buffer
where I could calculate the actual buffer size in bytes, and then
intellgently choose packets in that buffer to trim it down to a specific
size while minimizing the number of p-frames and i-frames dropped, and
preventing the actual impact of dropped frames on the stream. The
downside of it was that it required too much extra latency, and far too
many people complained about it, so it was removed in favor of the
current system.
The current system I just refer to just as 'packet dumping', which when
combined with low keyframe intervals (like most services use these
days), is the next-best method from my experience. Just dump the buffer
when you reach a threshold of buffering (which I prefer to measure with
time rather than in size), then wait for a new i-frame. Simple,
effective, and reduces the risk of consecutive buffering, while still
having fairly low impact on the stream output due to the low keyframe
interval of services.
By the way, audio will not (and should not ever) be dropped, lest you
end up with syncing issues (among other nasty things) specific to server
implementation.
- Fix an issue that could occur when using more than one video encoder.
Audio/video would not sync up correctly because they were expected to
be paired with a particular encoder. This simply adds a little
helper variable to encoder packets that specifies the system time in
microseconds. We then use that system time to sync
- Fix an issue with x264 with fractional FPS rates (29.97 and 59.94
particularly) where it would create ridiculously large stream
outputs. The problem was that you shouldn't set the timebase_*
variables in the x264 params manually, let x264 handle the default
values for it and leave them at 0.
- Make x264 use CFR output, because there's no reason to ever use VFR
in this case.
- Implement the RTMP output module. This time around, we just use a
simple FLV muxer, then just write to the stream with RTMP_Write.
Easy and effective.
- Fix the FLV muxer, the muxer now outputs proper FLV packets.
- Output API:
* When using encoders, automatically interleave encoded packets
before sending it to the output.
* Pair encoders and have them automatically wait for the other to
start to ensure sync.
* Change 'obs_output_signal_start_fail' to 'obs_output_signal_stop'
because it was a bit confusing, and doing this makes a lot more
sense for outputs that need to stop suddenly (disconnections/etc).
- Encoder API:
* Remove some unnecessary encoder functions from the actual API and
make them internal. Most of the encoder functions are handled
automatically by outputs anyway, so there's no real need to expose
them and end up inadvertently confusing plugin writers.
* Have audio encoders wait for the video encoder to get a frame, then
start at the exact data point that the first video frame starts to
ensure the most accrate sync of video/audio possible.
* Add a required 'frame_size' callback for audio encoders that
returns the expected number of frames desired to encode with. This
way, the libobs encoder API can handle the circular buffering
internally automatically for the encoder modules, so encoder
writers don't have to do it themselves.
- Fix a few bugs in the serializer interface. It was passing the wrong
variable for the data in a few cases.
- If a source has video, make obs_source_update defer the actual update
callback until the tick function is called to prevent threading
issues.
Now that we have the priorties window in we can finally be able to
select windows for capture source such as window capture. Only took
about an hour or two to write.
Also, fixed some depednency issues on winmm.lib with obs-outputs
- Add interleaving of video/audio packets for outputs that are encoded
and expect both video and audio data, sorting the packets and sending
them to the output when both video and audio is received.
- Combine create and initialize callbacks for the encoder API callback
interface.
Improve the properties API so that it can actually respond somewhat to
user input. Maybe later this might be further improved or replaced with
something script-based.
When creating a property, you can now add a callback to that property
that notifies when the property has been changed in the user interface.
Return true if you want the properties to be refreshed, or false if not.
Though now that I think about it I doubt there would ever be a case
where you would have this callback and *not* refresh the properties.
Regardless, this allows functions to change the values of properties or
settings, or enable/disable/hide other property controls from view
dynamically.
- Add start/stop code to obs-output module
- Use a circular buffer for the buffered encoder packets instead of a
dynamic array
- Add pthreads.lib as a dependency to obs-output module on windows in
visual studio project files
- Fix an windows export bug for avc parsing functions on windows.
Also, rename those functions to be more consistent with each other.
- Make outputs use a single function for encoded data rather than
multiple functions
- Add the ability to make 'text' properties be passworded
- obs-outputs module: Add preliminary code to send out data, and add
an FLV muxer. This time we don't really need to build the packets
ourselves, we can just use the FLV muxer and send it directly to
RTMP_Write and it should automatically parse the entire stream for us
without us having to do much manual code at all. We'll see how it
goes.
- libobs: Add AVC NAL packet parsing code
- libobs/media-io: Add quick helper functions for audio/video to get
the width/height/fps/samplerate/etc rather than having to query the
info structures each time.
- libobs (obs-output.c): Change 'connect' signal to 'start' and 'stop'
signals. 'start' now specifies an error code rather than whether it
simply failed, that way the client can actually know *why* a failure
occurred. Added those error codes to obs-defs.h.
- libobs: Add a few functions to duplicate/free encoder packets
Note that this is a somewhat heavily modified custom version of librtmp.
I modified all the platform specific code that we were using for the
OBS1 to make it platform-independent.
I don't really like the code in this library, but it works well enough,
so I can't really fault anyone for it. It's just very.. unclean. Even
for a C library, quite unclean. Some parts are also a little less safe
than I'd prefer as well.
- Make it so that encoders can be assigned to outputs. If an encoder
is destroyed, it will automatically remove itself from that output.
I specifically didn't want to do reference counting because it leaves
too much potential for unchecked references and it just felt like it
would be more trouble than it's worth.
- Add a 'flags' value to the output definition structure. This lets
the output specify if it uses video/audio, and whether the output is
meant to be used with OBS encoders or not.
- Remove boilerplate code for outputs. This makes it easier to program
outputs. The boilerplate code involved before was mostly just
involving connecting to the audio/video data streams directly in each
output plugin.
Instead of doing that, simply add plugin callback functions for
receiving video/audio (either encoded or non-encoded, whichever it's
set to use), and then call obs_output_begin_data_capture and
obs_output_end_data_capture to automatically handle setting up
connections to raw or encoded video/audio streams for the plugin.
- Remove 'active' function from output callbacks, as it's no longer
really needed now that the libobs output context automatically knows
when the output is active or not.
- Make it so that an encoder cannot be destroyed until all data
connections to the encoder have been removed.
- Change the 'start' and 'stop' functions in the encoder interface to
just an 'initialize' callback, which initializes the encoder.
- Make it so that the encoder must be initialized first before the data
stream can be started. The reason why initialization was separated
from starting the encoder stream was because we need to be able to
check that the settings used with the encoder *can* be used first.
This problem was especially annoying if you had both video/audio
encoding. Before, you'd have to check the return value from
obs_encoder_start, and if that second encoder fails, then you
basically had to stop the first encoder again, making for
unnecessary boilerplate code whenever starting up two encoders.
- Add a properties window for sources so that you can now actually edit
the settings for sources. Also, display the source by itself in the
window (Note: not working on mac, and possibly not working on linux).
When changing the settings for a source, it will call
obs_source_update on that source when you have modified any values
automatically.
- Add a properties 'widget', eventually I want to turn this in to a
regular nice properties view like you'd see in the designer, but
right now it just uses a form layout in a QScrollArea with regular
controls to display the properties. It's clunky but works for the
time being.
- Make it so that swap chains and the main graphics subsystem will
automatically use at least one backbuffer if none was specified
- Fix bug where displays weren't added to the main display array
- Make it so that you can get the properties of a source via the actual
pointer of a source/encoder/output in addition to being able to look
up properties via identifier.
- When registering source types, check for required functions (wasn't
doing it before). getheight/getwidth should not be optional if it's
a video source as well.
- Add an RAII OBSObj wrapper to obs.hpp for non-reference-counted
libobs pointers
- Add an RAII OBSSignal wrapper to obs.hpp for libobs signals to
automatically disconnect them on destruction
- Move the "scale and center" calculation in window-basic-main.cpp to
its own function and in its own source file
- Add an 'update' callback to WASAPI audio sources
Also, rename atomic functions to be consistent with the rest of the
platform/threading functions, and move atomic functions to threading*
files rather than platform* files
- Implement OBS encoder interface. It was previously incomplete, but
now is reaching some level of completion, though probably should
still be considered preliminary.
I had originally implemented it so that encoders only have a 'reset'
function to reset their parameters, but I felt that having both a
'start' and 'stop' function would be useful.
Encoders are now assigned to a specific video/audio media output each
rather than implicitely assigned to the main obs video/audio
contexts. This allows separate encoder contexts that aren't
necessarily assigned to the main video/audio context (which is useful
for things such as recording specific sources). Will probably have
to do this for regular obs outputs as well.
When creating an encoder, you must now explicitely state whether that
encoder is an audio or video encoder.
Audio and video can optionally be automatically converted depending
on what the encoder specifies.
When something 'attaches' to an encoder, the first attachment starts
the encoder, and the encoder automatically attaches to the media
output context associated with it. Subsequent attachments won't have
the same effect, they will just start receiving the same encoder data
when the next keyframe plays (along with SEI if any). When detaching
from the encoder, the last detachment will fully stop the encoder and
detach the encoder from the media output context associated with the
encoder.
SEI must actually be exported separately; because new encoder
attachments may not always be at the beginning of the stream, the
first keyframe they get must have that SEI data in it. If the
encoder has SEI data, it needs only add one small function to simply
query that SEI data, and then that data will be handled automatically
by libobs for all subsequent encoder attachments.
- Implement x264 encoder plugin, move x264 files to separate plugin to
separate necessary dependencies.
- Change video/audio frame output structures to not use const
qualifiers to prevent issues with non-const function usage elsewhere.
This was an issue when writing the x264 encoder, as the x264 encoder
expects non-const frame data.
Change stagesurf_map to return a non-const data type to prevent this
as well.
- Change full range parameter of video scaler to be an enum rather than
boolean
Make sure it locks the write mutex before freeing the packets, and put
the detach code in the main thread loop rather than off in a separate
function for clarity
...The reason why audio didn't work was because I overwrote the bitrate
values.
As for semaphores, mac doesn't support unnamed semaphores without using
mach semaphores. So, I just implemented a semaphore wrapper for each
OS.
- Add some temporary streaming code using FFmpeg. FFmpeg itself is not
very ideal for streaming; lack of direct control of the sockets and
no framedrop handling means that FFmpeg is definitely not something
you want to use without wrapper code. I'd prefer writing my own
network framework in this particular case just because you give away
so much control of the network interface. Wasted an entire day
trying to go through FFmpeg issues.
There's just no way FFmpeg should be used for real streaming (at
least without being patched or submitting some sort of patch, but I'm
sort of feeling "meh" on that idea)
I had to end up writing multiple threads just to handle both
connecting and writing, because av_interleaved_write_frame blocks
every call, stalling the main encoder thread, and thus also stalling
draw signals.
- Add some temporary user interface for streaming settings. This is
just temporary for the time being. It's in the outputs section of
the basic-mode settings
- Make it so that dynamic arrays do not free all their data when the
size just happens to be reduced to 0. This prevents constant
reallocation when an array keeps going from 1 item to 0 items. Also,
it was bad to become dependent upon that functionality. You must now
always explicitly call "free" on it to ensure the data is free, and
that's how it should be. Implicit functionality can lead to
confusion and maintainability issues.
- Fix a bug where the initial audio data insertion would cause all
audio data to unintentionally clear (mixed up < and > operators, damn
human error)
- Fixed a potential interdependant lock scenario with channel mutex
locks and graphics mutex locks. The main video thread could lock the
graphics mutex and then while in the graphics mutex could lock the
channels mutex. Meanwhile in another thread, the channel mutex could
get locked, and then the graphics mutex would get locked, causing a
deadlock.
The best way to deal with this is to not let mutexes lock within
other mutexes, but sometimes it's difficult to avoid such as in the
main video thread.
- Audio devices should now be functional, and the devices in the audio
settings can now be changed as desired.
- Implement a means of obtaining default settings for an
input/output/encoder. obs_source_defaults for example will return
the default settings for a particular source type.
- Because C++ doesn't have designated initializers, use functions in
the WASAPI plugin to register the sources instead.
- Implement windows monitor capture (code is so much cleaner than in
OBS1). Will implement duplication capture later
- Add GDI texture support to d3d11 graphics library
- Fix precision issue with sleep timing, you have to call
timeBeginPeriod otherwise windows sleep will be totally erratic.
- Add WASAPI audio capture for windows, input and output
- Check for null pointer in os_dlopen
- Add exception-safe 'WinHandle' and 'CoTaskMemPtr' helper classes that
will automatically call CloseHandle on handles and call CoTaskMemFree
on certain types of memory returned from windows functions
- Changed the wide <-> MBS/UTF8 conversion functions so that you use
buffers (like these functions are *supposed* to behave), and changed
the ones that allocate to a different naming scheme to be safe
- Split input and output audio captures so that they're different
sources. This allows easier handling and enumeration of audio
devices without having to do some sort of string processing.
This way the user interface code can handle this a bit more easily,
and so that it doesn't confuse users either. This should be done for
all audio capture sources for all operating systems. You don't have
to duplicate any code, you just need to create input/output wrapper
functions to designate the audio as input or output before creation.
- Make it detect soundflower and wavtap devices as mac "output" devices
(even though they're actually input) for the mac output capture, and
make it so that users can select a default output capture and
automatically use soundflower or wavtap.
I'm not entirely happy about having to do this, but because mac is
designed this way, this is really the only way to handle it that
makes it easier for users and UI code to deal with.
Note that soundflower and wavtap are still also designated as input
devices, so will still show up in input device enumeration.
- Remove pragma messages because they were kind polluting the other
compiler messages and just getting in the way. In the future we can
just do a grep for TODO to find them.
- Redo list property again, this time using a safer internal array,
rather than requiring sketchy array inputs. Having functions handle
everything behind the scenes is much safer.
- Remove the reference counter debug log code, as it was included
unintentionally in a commit.
If the default device changes, set the reconnect interval to 200
milliseconds so it pretty much immediately tries to reinitialize the
audio with the newly selected default device. Otherwise, use 2000
millisecond intervals, and assume disconnection.
Also, reduced FFmpeg logging to just regular FFmpeg information rather
than everything FFmpeg logs.
LOG_ERROR should be used in places where though recoverable (or at least
something that can be handled safely), was unexpected, and may affect
the user/application.
LOG_WARNING should be used in places where it's not entirely unexpected,
is recoverable, and doesn't really affect the user/application.
I can't believe I wasn't doing this. This is why file output was
getting corrupted. Audio and video send in data from separate threads.
I should be embarassed for not having considered that.
Key lesson: Increase threading paranoia levels. Apparently my
threading paranoid levels are lackluster.
fryshorts
Palana
Anry
jp9000
BtbN
Christoph Hohmann
Azat Khasanshin