libobs: Fix calculation copying aligned textures

Direct3D textures are usually aligned to a specific pitch, so their
internal width is often not equal to the expected output width; this
means that if we want to use it on our texture output, that we must
de-align the texture while copying the texture data.

However, I unintentionally messed up the calculation at some point with
RGBA textures, so the variable size I was supposed to be using was
supposed to be multiplied by 4 (for RGBA), while I was still expecting
single channel data.  So, if the texture width was something like 1332,
the source (directx) texture line size would be somewhere at or above
5328 (because it's RGBA), then destination is at 1332 (YUV luma plane),
and it would unintentionally treat 3996 (or 5328 - 1332) bytes as the
unused alignment data.  So this fixes that miscalculation.
This commit is contained in:
jp9000 2015-01-14 14:57:27 -08:00
parent 0e81b0a2ac
commit 05fc9c5b78

View File

@ -394,7 +394,7 @@ static void fix_gpu_converted_alignment(struct obs_core_video *video,
struct video_frame *output, const struct video_data *input)
{
uint32_t src_linesize = input->linesize[0];
uint32_t dst_linesize = output->linesize[0];
uint32_t dst_linesize = output->linesize[0] * 4;
uint32_t src_pos = 0;
for (size_t i = 0; i < 3; i++) {