The gs_texture_create_from_dmabuf function is able to create textures
from multiplanar DMA-BUF data. However, the only in-tree consumer of
this function - the PipeWire capture - does not use that.
Make PipeWire create textures from multiplanar buffers.
The gs_draw_sprite_subregion() function is used when a cropping
rectangle is received from PipeWire. It is usually used by
compositors to implement window screencast - where a large and
mostly empty frame is sent, the window contents are only a small
part of it, and the crop rectangle tells us that.
Recently the wlroots implementation of portals started to use it
to implement cropping, and it exposed a bug in the PipeWire code
in OBS Studio. The gs_draw_sprite_subregion() function takes a pair
of integers representing position (x, y) and a pair of integers
representing size (width, height). The PipeWire code, however,
passes a second pair of positions (x2, y2) instead of sizes, and
it causes overrendering the crop area.
This bug wasn't hit yet because both GNOME and KDE implementations
always send (0, 0) as position, which practically never trigger
this condition.
Pass only width and height to gs_draw_sprite_subregion(), instead
of adding x and y to them.
Fixes https://github.com/obsproject/obs-studio/issues/4982
Intead of creating one pair of GDBusConnection + GDBusProxy objects
for each PipeWire capture, be it window or desktop, use the global
ones managed by portal.c.
Even if g_bus_get_sync() ends up reusing the same object under the
hood, it's still a net gain, since it has to perform some thread
synchronization routines that aren't necessary here. Creating the
proxy object was a worse offender, because despite being asynchronous,
it would still incur in a few socket messages + a cancellable fd +
thread synchronization.
Reuse these objects from portal.c. The biggest code change here is
that create_proxy() and on_proxy_created_cb() were merged into
init_obs_pipewire().
The cursor bitmap is centered on the hotspot, so not accounting
for it means PipeWire captures were positioning the cursor sprite
slightly off.
Properly account for the hotspot by subtracting it from the cursor
position.
Related: https://github.com/obsproject/obs-studio/issues/4766
The assertion exposed a bug in the KDE implementation of the Desktop
portal. However, it comes with the side effect of exiting OBS Studio
in this case, where we can actually make it work (even if buggy).
De-escalate the assertion to a warning, and then attempt to find the
correct stream to use.
Add a new Linux capture based on PipeWire [1] and the Desktop portal [2].
This new capture starts by asking the Desktop portal for a screencapture session.
There are quite a few D-Bus calls involved in this, but the key points are:
1. A connection to org.freedesktop.portal.ScreenCast is estabilished, and the
available cursor modes are updated.
2. CreateSession() is called. This is the first step of the negotiation.
3. SelectSources() is called. This is when a system dialog pops up asking the
user to either select a monitor (desktop capture) or a window (window capture).
4. Start() is called. This signals the compositor that it can setup a PipeWire
stream, and start sending buffers.
The reply to this fourth call gives OBS Studio the PipeWire fd, and the id of the
PipeWire node where the buffers are being sent to. This allows creating a consumer
PipeWire stream, and receive the buffers.
Metadata cursor is always preferred, but on the lack of it, we ask the stream for
an embedded cursor (i.e. the cursor is drawn at the buffer, and OBS Studio has no
control over it.)
Window capturing is implemented as a crop operation on the buffer. Compositors
can send big buffers, and a crop rectangle, and this is used to paint a subregion
of the buffer in the scene.
The new capture is only loaded when running on EGL, since it depends on EGL to
call gs_texture_create_from_dmabuf().
[1] https://pipewire.org/
[2] https://github.com/flatpak/xdg-desktop-portal/
