This content has been marked as final.
Show 3 replies
-
1. Re: Color Conversion using FP10 vs PBToolkit
Kevin Goldsmith Apr 1, 2009 3:23 PM (in response to jbfiedler)I'll see if I can get a chance to check this out in the player. The math in the player is done at 32 bpc, but you are right, it is expanded/compressed to 8bpc on the way in/out. I'd have to see how serious the color shifts were to see if that might be the culprit, it certainly wouldn't be at the same fidelity as in the PBT. There was also a bug in how we export min/max in PBJ that might be partly to blame. We'll be releasing a new toolkit with a fix for that issue very soon and you could try re-exporting the PBJ and seeing if the issue is fixed. -
2. Re: Color Conversion using FP10 vs PBToolkit
AIF Bob Apr 1, 2009 4:32 PM (in response to jbfiedler)Can you try replacing the min and max functions with their equivalents using if statements. So instead of
r = min( a, b );
you'd have
r = a;
if( b < a )
r = b;
and the similar construction for max.
I am also a little worried that you're doing exact comparisons with floats - e.g.
delta_Max == 0.0
depending on the exact way in which delta_Max is caluclated (and that exact way is different betwen the flash runtime and the toolkit runtime) you might end up with a result that is exactly 0.0 in one case, and just a fraction off in the other case. The usual fix for this is to use something like:
abs( delta_Max ) < epsilon
where epsilon is set to whatever small value you think is appropriate for your application -
3. Re: Color Conversion using FP10 vs PBToolkit
jbfiedler Apr 1, 2009 5:21 PM (in response to jbfiedler)Thank you both for your posts. The problem was in the min and max functions. Once I replaced them with Bobs suggestion the shaders worked wonderfully.


