libobs' shader language is basically HLSL, and tex.Load uses an int3 for
2D textures, with texture mipmap index for the last component. This bug
bypassed testing because the front-end automatically switches to OpenGL
if D3D11 initialization fails, and when converted to GLSL, works fine
because texelFetch only requires two components. This also means
there's a bug in GLSL shader conversion code, because it's essentially
ignoring the third component when it shouldn't be.
Eventually, most things should be replaced with Load where applicable
(though in some cases sub-pixel sampling is desired).
This commit also fixes a bug where NV12 async sources wouldn't render
correctly.
When render targets are used, they output to the render target inverted
due to the way that opengl works. This fixes that issue by inverting
the projection matrix so that it renders the image upside down and
inverting the front face from counterclockwise to clockwise.
Just for a quick background: D3D's fmod intrinsic is very imprecise.
Naturally floating points aren't precise at all, and when the numbers
you're dealing with become very large, it can often be off by 0.1 or
more.
However, apparently 0.1 isn't enough of an offset to ensure a proper
value when using the fmod intrinsic and then flooring the value. 0.2
seems to fix the issue and make the image display properly.
jp9000
BtbN