# VertexShader - Parameter / Matrix Issues

**.lars**Aug 23, 2011 5:34 AM

I'm trying to implement a simple mask-shader for my 2D engine and I'm having multiple issues getting it to work.

Basically the shader takes two images and two matrices and tries to map back the vertex position of the sprites vertex back to the local mask space position and then calculate the local uv-coordinates for the mask, but I'm stuck with some basic calculations here.

Bug 1: Divisions in the material-vertex kernel don't work at all. That's why I have to put invertedMaskSize (which is 1.0 / 128.0) into the shader (but we already know that )

Bug 2: interpolated vars have to be float4, float2 is not possible (That's a pretty old bug as well)

I tried the following changes in the shader. The shader posted here, just tries to display the resulting uv-coordinates. No textures are used.

Case 1:

interpolatedMaskUV = float4((vertexPos.x + halfMaskSize.x) * invertedMaskSize.x, (vertexPoss.y + halfMaskSize.y) * invertedMaskSize.y, 0.0, 0.0);

The output is this: http://dev.nulldesign.de/plain_uvcoords.png Just like you expect! Perfect, let's proceed.

Case 2:

Change the halfMaskSize and invertedMaskSize to float2 and set set the parameters as two vectors of length two of course in AS. The output: http://dev.nulldesign.de/float2_uvcoords.png

Case 3:

Masking Test, matrix multiplication. First calculating the world space position of the vertex:

float4 worldSpacePos = float4(vertexPos.x, vertexPos.y, 0.0, 1.0) * objectToClipSpaceTransform;

Then mapping it back to the local space of the mask:

float4 localMaskSpacePos = worldSpacePos * maskObjectToClipSpaceTransform;

And calculating the uv-coords:

interpolatedMaskUV = float4((localMaskSpacePos.x + halfMaskSize.x) * invertedMaskSize.x, (localMaskSpacePos.y + halfMaskSize.y) * invertedMaskSize.y, 0.0, 0.0);

For testing, I set the maskObjectToClipSpaceTransform to the inverse of the objectToClipSpaceTransform. In theory and on paper, this should work.

But, I think, something gets out of order and maybe the maskObjectToClipSpaceTransform is screwed up in the shader, just like when I set the halfMaskSize and invertedMaskSize to float2. The result is this: http://dev.nulldesign.de/local_uvcoords.png and I have no idea how to fix this...

<languageVersion : 1.0;>

material kernel texture

<

namespace : "ND2D_Shader";

vendor : "nulldesign";

version : 1;

>

{

input vertex float2 uvCoord

<

id : "PB3D_UV";

>;

input vertex float2 vertexPos

<

id : "PB3D_POSITION";

>;

parameter float2 uvOffset;

parameter float4x4 objectToClipSpaceTransform;

parameter float4x4 maskObjectToClipSpaceTransform;

// if set to float2, strange things happen

parameter float4 halfMaskSize;

parameter float4 invertedMaskSize;

interpolated float4 interpolatedUV;

interpolated float4 interpolatedMaskUV;

void evaluateVertex()

{

// not used in the current test ...

interpolatedUV = float4(uvCoord.x + uvOffset.x, uvCoord.y + uvOffset.y, 0.0, 0.0);

float4 worldSpacePos = float4(vertexPos.x, vertexPos.y, 0.0, 1.0) * objectToClipSpaceTransform;

// doesn't work as expected

float4 localMaskSpacePos = worldSpacePos * maskObjectToClipSpaceTransform;

interpolatedMaskUV = float4((localMaskSpacePos.x + halfMaskSize.x) * invertedMaskSize.x,

(localMaskSpacePos.y + halfMaskSize.y) * invertedMaskSize.y,

0.0, 0.0);

}

input image4 textureImage;

input image4 textureMaskImage;

parameter float4 color;

output float4 result;

void evaluateFragment()

{

// just visualize the uv-coords

result = float4(interpolatedMaskUV.x, interpolatedMaskUV.y, 0.0, 1.0);

/*

float4 texel = sample(textureImage, float2(interpolatedUV.x, interpolatedUV.y), PB3D_2D | PB3D_MIPNEAREST | PB3D_CLAMP);

float4 texel2 = sample(textureMaskImage, float2(interpolatedMaskUV.x, interpolatedMaskUV.y), PB3D_2D | PB3D_MIPNEAREST | PB3D_CLAMP);

result = float4(texel.r * color.r,

texel.g * color.g,

texel.b * color.b,

texel.a * color.a * texel2.a);

*/

}

}

I know that we're working with a four month old version of pb3d and I hope that a new version will be out soon and maybe all these bugs I encountered are already solved, but if not.... here's another shader to fix