84 Commits

Author SHA1 Message Date
jp9000
7e78c17ace win-capture: Check hook version before capture init
Checks the hook version to ensure compatibility with hook DLL.  It's
unlikely it'll ever be necessary to increment the hook version, but this
is just a precautionary thing that allows a hook DLL to make sure it's
rejected by an older OBS version if needed.  Again however, very
unlikely that the major version will ever be incremented.
2020-02-29 04:53:29 -08:00
Matthieu Cunzi
74acbd7ed0 win-capture: Add Vulkan capture
Co-authored-by: jp9000 <obs.jim@gmail.com>
Co-authored-by: jpark37 <jpark37@users.noreply.github.com>
2020-02-29 04:53:28 -08:00
jp9000
a20f1168a7 win-capture/graphics-hook: Don't allow multiple hooks
Prevents multiple separate hook DLLs from being loaded in to a target at
once.  This is done just in the off-chance that someone might add
another hook to the Vulkan layer registry.
2020-02-29 00:15:54 -08:00
jpark37
39dc64f0f8 win-capture/graphics-hook: Refactor mutex check
OpenMutex doesn't return INVALID_HANDLE_VALUE, so there's no need to
check for that.  Also only call CloseHandle if the call was successful.
2020-02-29 00:15:54 -08:00
jp9000
3e3521b9e1 win-capture/graphics-hook: Fix OBS detection
Fixes where the graphics hook would try to detect if OBS is currently
active or not.
2020-02-28 23:48:11 -08:00
jp9000
ef6a6827f4 win-capture/graphics-hook: Add flog and flog_hr
Allows automatically outputting the function name as part of the hook
logging.  This really doesn't need to be a manual process.  Makes code a
bit cleaner when used.
2020-02-28 23:48:11 -08:00
VodBox
d2e115881d win-capture: Fix hook collisions with multiple game captures
Before this change, after a game capture source would send a signal to
init or restart a graphics hook, it would respond to any and all hook
ready signals.

With multiple game capture sources in the same scene, a source could
receive the signal intended for another source, and show the wrong
texture.

This change adds the window handle to the name for shared data with the
hook, resulting in hooks for other sources being ignored.
2020-02-18 21:20:48 +13:00
tujinshu
76e6a99124 win-capture: Fix rare crash when GL program exits
**Commit message modified and clarified by Jim**

When hooking a program that has both DirectX and OpenGL contexts in use,
it is possible to cause a crash on shutdown due to capture_active()
returning true when an OpenGL context is deleted.  Normally, when
capturing an OpenGL program, this would not happen because the 'active'
variable would not be set due to OpenGL capture not being initialized,
but if DirectX is captured while an OpenGL context is available, and
OpenGL could not load these required functions, then GL can crash due to
trying to use unavailable functions.

This case is extremely rare and doesn't happen under normal
circumstances; only if a program is using both DirectX and OpenGL within
the same program simultaneously, and *only* if OpenGL could not load the
required functions.  This likely almost never happens under normal
programs, games, and hardware.  This was apparently produced by hooking
a GL Qt program that used QWebEngine, which used multiple contexts at
once.
2019-11-05 13:44:15 -08:00
jpark37
3ddef4bf7d win-capture: Remove support for feature level 9.3
Be consistent with libobs-d3d11, which now uses 10_0 as the baseline.
2019-09-08 22:43:41 -07:00
jpark37
ef39715c31 win-capture: Remove unused variable from D3D12 capture 2019-09-08 22:36:25 -07:00
craftwar
e004ce8064 graphics-hook: Fix format specifier 2019-09-02 21:12:27 +08:00
jp9000
f53df7da64 clang-format: Apply formatting
Code submissions have continually suffered from formatting
inconsistencies that constantly have to be addressed.  Using
clang-format simplifies this by making code formatting more consistent,
and allows automation of the code formatting so that maintainers can
focus more on the code itself instead of code formatting.
2019-06-23 23:49:10 -07:00
geemion
f6581952bc win-capture/graphics-hook: Check if mutex abandoned
It's possible that the mutexes used with shared memory capture to return
WAIT_ABANDONED if OBS is shut down abnormally while the mutex is locked.
2019-04-14 03:44:48 -07:00
jp9000
b3ef46d986 cmake: Make static VC runtime libraries consistent
Makes Visual C runtime libraries consistent across
Debug/MinSizeRel/Release/RelWithDebInfo, rather than just changing those
flags for RelWithDebInfo.  Also adds /Zl for statically linked
libraries.

Closes obsproject/obs-studio#1421
2018-09-12 07:49:13 -07:00
jp9000
8a16fa8341 win-capture/graphics-hook: Fix memory offset calculation
Fixes an issue where align_pos could be smaller than
sizeof(struct shmem_data), potentially overwriting memory of the header.

References jp9000/obs-studio#1202
2018-02-24 21:48:50 -08:00
Bazhenoff
b9d044267d graphics-hook: blacklist OpenGL capture for "cm_client.exe"
Closes jp9000/obs-studio#1076
2017-11-16 17:57:11 -08:00
jp9000
e5d660e944 graphics-hook: Blacklist specific game from GL capture
"Life is Feudal: Your Own" will use Direct3D to render the game, then
OpenGL to render its in-game menus, which causes a conflict with itself.
This specifically blacklists the game from capturing OpenGL to prevent
that from happening.
2017-10-02 15:29:04 -07:00
jp9000
1ca9f8872e win-capture: Add IDXGISwapChain1::Present1 hook support
Allows capturing games/programs that may be using Present1 instead of
the regular Present call for rendering.
2017-06-19 15:45:00 -07:00
jp9000
c4036b18bb win-capture/graphics-hook: Fix D3D10/D3D11 detection
When hooking DXGI-based graphics programs in the hook, the first hook
point is IDXGISwapChain::Present.  To be able to initiate a capture, a
pointer to the device context that created the swap chain is required,
which can be retrieved via IDXGISwapChain::GetDevice.  Determining
whether the device context was D3D10 or D3D11 has always been somewhat
of an issue due to D3D10 and D3D11 being nearly identical, as well as
their interoperability/interchangeability.  The GetDevice function would
first be called with the UUID of ID3D10Device, then if that failed,
the UUID of ID3D11Device.

However, with certain specific D3D11 games, GetDevice would for some
unknown reason succeed with the UUID of ID3D10Device, which would cause
capture to fail.  (Conversely, attempting to call GetDevice with the
UUID of ID3D11Device on a device that's actually ID3D10Device would
always succeed, so reversing the order of the test was not an option).
There were originally three known D3D11 games that would erroneously
succeed when querying a D3D10 device interface:  Call of Duty: Ghosts,
Just Cause 3, and theHunter: Call of the Wild.  All other known D3D11
games would work correctly.  Because it was only these three games, a
hack was originally implemented in the form of an executable exception
list that would force them to capture D3D11.

Unfortunately, Oculus games are now failing under the same circumstance
as well, so a simple hack will no longer work.  To fix this, a more
reliable method of detecting which context it is had to be discovered:
simply check the feature level using ID3D11Device::GetFeatureLevel.  If
the feature level is D3D_FEATURE_LEVEL_11_0 or D3D_FEATURE_LEVEL_11_1,
the device is definitely a D3D11 device.  Otherwise, continue the tests
as they were before.  Successfully tested with many D3D10 games, many
D3D11 games, and especially those three D3D11 games that previously were
detected as D3D10 erroneously.
2017-06-15 03:48:43 -07:00
Richard Stanway
9e95b2eb6f
Various: Don't use boolean bitfields
Using bitfields causes less optimized code generation and the memory
savings are minimal as none of the objects are instantiated enough
times to be worth it.

See https://blogs.msdn.microsoft.com/oldnewthing/20081126-00/?p=20073
2017-05-10 23:28:46 +02:00
Ryan Foster
be98cee2a0 Fix various typos across multiple modules 2017-04-25 22:39:42 -04:00
Richard Stanway
d2e9e47bb4 graphics-hook: Account for sizeof(wchar_t) in len 2017-02-25 17:18:55 +01:00
Cam
8a6491c9bf win-capture: Add hook exception for theHunter: COTW
Adds an exception to skip d3d10 checks for theHunter: Call of the Wild,
which fixes incompatibility with game capture.

Closes jp9000/obs-studio#801
2017-02-25 06:42:36 -08:00
jp9000
21d70fa207 win-capture: Don't use FindWindow for game capture keepalive
Using and creating a window can use issues in game capture if multiple
game captures are active, so revert back to using a mutex, and just
ignore the keepalive check failure if injected inside a UWP program
(only check to see if GetLastError reports that it's not found -- if it
returns access denied or any other error, assume it's in a UWP program,
and ignore the keepalive check).
2017-01-15 09:15:45 -08:00
jp9000
21be33805d win-capture: Release backbuffers immediately upon init
There's no need to keep a reference to the backbuffers.  That and the
backbuffer references weren't being released anyway, so that fixes that
bug.
2016-12-23 08:07:26 -08:00
jp9000
74a5bdf993 win-capture: If backbuffer count is 1, disable dxgi 1.4 use
Executing the dxgi 1.4 code causes it to cycle through backbuffers,
backbuffers which may not exist if in discard mode.
2016-12-23 08:02:35 -08:00
jp9000
414ff5ba14 win-capture: Refactor DX12 backbuffer code
It was undesirable to pass arrays via function parameters, so a
structure should be used instead.

Also increases total allowable backbuffers to 8.
2016-12-23 08:02:30 -08:00
jp9000
a4143be052 win-capture: Fix possible access of array beyond size
If the backbuffer count is larger than 3, it could still try to assign
backbuffers to pointers beyond the variable's array size when calling
swap->GetBuffer.
2016-12-23 07:49:13 -08:00
jp9000
3b5a30ce97 win-capture: Capture all D3D12 backbuffers
D3D12 capture does not take in to account when multiple backbuffers are
in used.  With previous versions of Direct3D there was no need to do
this, but with D3D12 you must explicitly capture each specific
backbuffer currently in use.
2016-12-23 01:53:15 -08:00
jp9000
d4a99e062b win-capture: Output hook debug messages if addresses missing 2016-12-16 14:05:57 -08:00
jp9000
b5f216ef88 win-capture: Use wide strings for named objects
Prevents from having to mix ansi/wide string usage for opening UWP
programs.
2016-12-16 02:21:10 -08:00
jp9000
ab9bda52e0 win-capture: Create all named objects within hook
All named objects (including file mapped shared memory) need to be
created within the hook itself due to the fact that UWP programs cannot
access named objects outside of the UWP process.

Because shared memory needs to be created within the hook, the capture
loop cannot start until the shared memory has been filled with valid
data.  Creating an additional "initialize" event fixes this issue.

Additionally, changed the way that named kernel objects are
opened/created.  Before, there were functions that would first try to
open named objects and then implicitly create them if opening failed
(assuming that if the hook didn't create it first, game capture would),
now it's been changed so that you can only either explicitly open or
create.
2016-12-16 02:21:07 -08:00
jp9000
d19342442f win-capture: Use window for keepalive check
To check to make sure game capture is still active in the capture
program, it currently uses a named event, and then it checks to see if
that named event exists.  However with UWP programs, you can't open a
named event outside of the UWP process.  FindWindow on the other hand
does work, so instead of checking to see if a named kernel object
exists, create a window and check to see if that window exists.
2016-12-16 02:21:06 -08:00
jp9000
4ec1033741 win-capture: Do not require pipe/mutex within hook
Named pipes cannot be used within UWP programs, so make them optional.
2016-12-15 13:03:44 -08:00
jp9000
746061fb3a win-capture: Always use minimal access rights within hook
This prevents issues with opening handles to objects within UWP
programs, which have increased security limitations.
2016-12-15 13:03:42 -08:00
jp9000
e148087636 win-capture: Only duplicate to get cur thread handle
Instead of calling OpenThread, use DuplicateHandle with the minimal
access rights (SYNCHRONIZE) to be able to get the current thread handle
within a UWP program.
2016-12-12 08:10:06 -08:00
jp9000
ca607f8b35 cmake: Add _CRT_SECURE_NO_WARNINGS to all projects
This prevents issues with using standard C functions, where microsoft
would otherwise spit out pointless warnings to encourage using
microsoft-specific functions instead.
2016-12-08 03:27:36 -08:00
jp9000
2f305cb550 win-capture: Use static runtimes for hooks/helpers
(Note: This commit also modifies the ipc-util/seg-service modules)

When compiling the final project, always compile
ipc-util/get-graphics-offsets/graphics-hook/inject-helper/seg-service
with static MSVC runtimes to prevent the need of requiring the MSVC
runtimes for both architectures.
2016-11-23 06:03:00 -08:00
jp9000
0b27b51294 win-capture: Add D3D12 capture support
Currently only supports shared texture capture, and doesn't support
rescaling.
2016-11-03 12:13:42 -07:00
jp9000
f49065b1e6 win-capture: Use IUnknown for getting swap backbuffers
Using IDXGIResource doesn't work with D3D12.  D3D12 textures are not a
subclass of that class.
2016-11-03 09:28:32 -07:00
jp9000
6d33f7e091 win-capture: Add trick to ensure game cap. shared tex support
This new trick forgoes the use of patches and allows the ability to use
shared GPU resources despite being a Direct3D 9.0c context.
2016-08-12 18:44:54 -07:00
jp9000
571c3a6859 win-capture: Update D3D9 patches
Fixes performance issues with windows 10.0.14393.0
2016-08-06 15:27:33 -07:00
jp9000
183ee11e24 win-capture: Fix lookup order for d3d9 patch
It's supposed to look for patch segments in ascending order, from the
smallest offset to the largest offset.  Patch type/comparison is
identical to the one it's being swapped with, so only the offsets need
to be swapped.
2016-08-06 15:27:33 -07:00
jp9000
7feb21572f win-capture: Update D3D9 shared texture patches 2016-07-15 13:00:21 -07:00
jp9000
d7ed0f1976 Revert "win-capture: Track gl "swap" invocations to prevent duplicate work"
This reverts commit 4c505e7030f2f0b7dab9c51125ac45cf3c28adb1.

Reverting this for the time being due to issues with quakelive.  This
will be more thoroughly tested and hopefully added again.
2016-07-10 00:38:15 -07:00
Palana
4c505e7030 win-capture: Track gl "swap" invocations to prevent duplicate work
Tested using FTL (steam): SwapBuffers ultimately calls wgl_swap_buffers
causing an additional copy which just isn't necessary

This also causes game capture to sometimes capture overlays even when
not intended
2016-07-03 13:18:23 -07:00
jp9000
b012c7280a win-capture/graphics-hook: Fix vs2015 internal compiler error
Removing this union fixes the internal compile error that would occur on
visual studio 2015 update 2 and above when these variables were all in a
union.
2016-06-22 14:40:54 -07:00
jp9000
d32424e5ad win-capture: Reset d3d9 capture if device recreated
Fixes a bug where if a D3D9 program recreates its device the capture
would become invalid.  Certain games (especially blizzard games) will
completely recreate their Direct3D device if a critical D3D9 error
occurs.
2016-06-11 12:27:06 -07:00
jp9000
6ca3cd3504 win-capture: Fix VS2015 update 2 compiler error
Having volatile variables inside of a union causes a C1001 compiler
error in visual studio 2015 update 2.

Closes jp9000/obs-studio#544
2016-05-28 09:40:49 -07:00
jp9000
2591f24594 win-capture: Add game capture fix for darkest dungeon
Darkest dungeon uses an unusual technique for drawing its frames: a
fixed 1920x1080 frame buffer used in place of the backbuffer, which is
then stretched to fit the size of the screen (whether the screen is
bigger or smaller than the actual texture).

The custom frame would cause glReadBuffer to initially fail with an
error.  When this happens, their custom frame buffer is in use, so all
that needs to be done is simply reset the capture and force the current
output size to 1920x1080 while that custom frame is in use.

They presumably did this in order to ensure the game looks the same at
any resolution.  Instead of having to use power-of-two sprites and
mipmaps for every single game sprite and stretch/skew each of them
(which would risk the final output "not looking quite right" at
different resolutions), they simply use non-pow-2 sprites with no
mipmaps and render them all on to one texture of a fixed size and then
stretch that final output texture.  That ensures that the actual
composite of the game still looks the same at any resolution, while
reducing texture memory by not requiring each sprite to use a
power-of-two texture and mipmaps.
2016-01-26 11:49:28 -08:00