Don't allow unsupported Vulkan formats to fall back to B8G8R8A8.
Probably better to fail completely than do an illegal copy.
Also remove bad conversion for VK_FORMAT_A2R10G10B10_UNORM_PACK32.
Red and blue channels were reversed, and there's no DXGI equivalent.
Addresses #2796. We can do more later if justified.
Make sure HWND tracking is cleaned up when Vulkan surfaces are
destroyed. Also use unbounded linked list to fix games that leak
surfaces on Alt+Tab like Doom.
Also replace CRITICAL_SECTION with SRWLOCK, both for claimed speed
benefit, and to remove initialization code.
Sometimes the width/height would initially be at 0x0 when the swap chain
was alt-tabbing, causing the capture to fail full-stop when trying to
reacquire.
Game capture assumes the top-level ancestor HWND, not a child HWND.
This fixes bugs such as the VLC window not capturing anymore due to the
fact that the capture HWND of VLC is a child window of the main VLC
window and not the main VLC window itself.
Checks the hook version to ensure compatibility with hook DLL. It's
unlikely it'll ever be necessary to increment the hook version, but this
is just a precautionary thing that allows a hook DLL to make sure it's
rejected by an older OBS version if needed. Again however, very
unlikely that the major version will ever be incremented.
Prevents multiple separate hook DLLs from being loaded in to a target at
once. This is done just in the off-chance that someone might add
another hook to the Vulkan layer registry.
Allows automatically outputting the function name as part of the hook
logging. This really doesn't need to be a manual process. Makes code a
bit cleaner when used.
Before this change, after a game capture source would send a signal to
init or restart a graphics hook, it would respond to any and all hook
ready signals.
With multiple game capture sources in the same scene, a source could
receive the signal intended for another source, and show the wrong
texture.
This change adds the window handle to the name for shared data with the
hook, resulting in hooks for other sources being ignored.
**Commit message modified and clarified by Jim**
When hooking a program that has both DirectX and OpenGL contexts in use,
it is possible to cause a crash on shutdown due to capture_active()
returning true when an OpenGL context is deleted. Normally, when
capturing an OpenGL program, this would not happen because the 'active'
variable would not be set due to OpenGL capture not being initialized,
but if DirectX is captured while an OpenGL context is available, and
OpenGL could not load these required functions, then GL can crash due to
trying to use unavailable functions.
This case is extremely rare and doesn't happen under normal
circumstances; only if a program is using both DirectX and OpenGL within
the same program simultaneously, and *only* if OpenGL could not load the
required functions. This likely almost never happens under normal
programs, games, and hardware. This was apparently produced by hooking
a GL Qt program that used QWebEngine, which used multiple contexts at
once.
Code submissions have continually suffered from formatting
inconsistencies that constantly have to be addressed. Using
clang-format simplifies this by making code formatting more consistent,
and allows automation of the code formatting so that maintainers can
focus more on the code itself instead of code formatting.
Makes Visual C runtime libraries consistent across
Debug/MinSizeRel/Release/RelWithDebInfo, rather than just changing those
flags for RelWithDebInfo. Also adds /Zl for statically linked
libraries.
Closesobsproject/obs-studio#1421
Fixes an issue where align_pos could be smaller than
sizeof(struct shmem_data), potentially overwriting memory of the header.
References jp9000/obs-studio#1202
"Life is Feudal: Your Own" will use Direct3D to render the game, then
OpenGL to render its in-game menus, which causes a conflict with itself.
This specifically blacklists the game from capturing OpenGL to prevent
that from happening.
When hooking DXGI-based graphics programs in the hook, the first hook
point is IDXGISwapChain::Present. To be able to initiate a capture, a
pointer to the device context that created the swap chain is required,
which can be retrieved via IDXGISwapChain::GetDevice. Determining
whether the device context was D3D10 or D3D11 has always been somewhat
of an issue due to D3D10 and D3D11 being nearly identical, as well as
their interoperability/interchangeability. The GetDevice function would
first be called with the UUID of ID3D10Device, then if that failed,
the UUID of ID3D11Device.
However, with certain specific D3D11 games, GetDevice would for some
unknown reason succeed with the UUID of ID3D10Device, which would cause
capture to fail. (Conversely, attempting to call GetDevice with the
UUID of ID3D11Device on a device that's actually ID3D10Device would
always succeed, so reversing the order of the test was not an option).
There were originally three known D3D11 games that would erroneously
succeed when querying a D3D10 device interface: Call of Duty: Ghosts,
Just Cause 3, and theHunter: Call of the Wild. All other known D3D11
games would work correctly. Because it was only these three games, a
hack was originally implemented in the form of an executable exception
list that would force them to capture D3D11.
Unfortunately, Oculus games are now failing under the same circumstance
as well, so a simple hack will no longer work. To fix this, a more
reliable method of detecting which context it is had to be discovered:
simply check the feature level using ID3D11Device::GetFeatureLevel. If
the feature level is D3D_FEATURE_LEVEL_11_0 or D3D_FEATURE_LEVEL_11_1,
the device is definitely a D3D11 device. Otherwise, continue the tests
as they were before. Successfully tested with many D3D10 games, many
D3D11 games, and especially those three D3D11 games that previously were
detected as D3D10 erroneously.
Using and creating a window can use issues in game capture if multiple
game captures are active, so revert back to using a mutex, and just
ignore the keepalive check failure if injected inside a UWP program
(only check to see if GetLastError reports that it's not found -- if it
returns access denied or any other error, assume it's in a UWP program,
and ignore the keepalive check).
If the backbuffer count is larger than 3, it could still try to assign
backbuffers to pointers beyond the variable's array size when calling
swap->GetBuffer.
D3D12 capture does not take in to account when multiple backbuffers are
in used. With previous versions of Direct3D there was no need to do
this, but with D3D12 you must explicitly capture each specific
backbuffer currently in use.
