VUYA_4444_32f_709 ranging to 10 bit int

[jamesatdrastic]

Hi

I am converting VUYA_4444_32f_709 to 10 bit integer for a codec in a Premiere exporter.  When I try various scale values (1024, 960/940) for luma and chroma, the luma and the chroma are both to large (eg. the vectorscope goes at the correct angles, but beyond the boxes, and the waveform overshoots).  Is there some documentation on the correct way to take a VUYA_4444_32f_709 buffer and convert it to a integer bit depth?  Also, I have been treating the component as 0.0->1.0, scaling by 1024.0.  This seems to work, but I would like know if that is the correct way to handle it.

Thanks, James

[Zac Lam]

Hi James,

32f formats are not clipped, and may contain values outside of [0.0, 1.0].  Could that be what you're seeing?

[jamesatdrastic]

Hi Zac

Unfortunately not.  It is not clipping, it is just to large.  Here is a normal v210 output from the exporter

and here it the 4444 YUVA when using a 1024.0 scale (e.g. assuming full range, where back would be 0.0625 and white would be 9.918)

By using scales of around 0.9 in luma and 0.82 in chroma, it gets quite close

But those number don't make much sense, other then the ratio of 960/940 is correct.

Cheers, James

[Zac Lam · Accepted solution]

Our floating point YUV format is basically YPbPr using the equations in Poynton’s Digital Video and HDTV algorithms book.

An 8-bit luma less than 16 would map to a negative floating point value, and 8-bit luma over 235 would map to values >1.0.

Chroma reference levels are mapped from –0.5 to 0.5 in floating point space. So in 8-bit, Cr of 16 would map to –0.5 and Cr of 240 maps to 0.5. Values outside this range exceed the –0.5 to 0.5 range.

// 601Y'PbPr color space represented in 32f:

//        8-bit UNIT | 32f

// ----------------+-------

//  Y          0        -16/219 = -0.07306

//  Y      1..15        (footroom / "super blacks")

//  Y         16        0.0 (black)

//  Y        128        (128-16)/219 = 0.50228

//  Y        235        1.0 (white)

//  Y   236..254        (headroom / "super whites")

//  Y        255        +239/219 = 1.09132

//

//  Pb/Pr      0        -128/224 = -0.57143

//  Pb/Pr     16        -112/224 = -0.5 (reference level - Cb: max yellow/min blue, Cr = max cyan/min red)

//  Pb/Pr    128        0.0

//  Pb/Pr    240        +112/224 = 0.5 (reference level - Cb: max blue/min yellow, Cr = max red/min cyan)

//  Pb/Pr    255        128/224 = 0.57143

//

//  Alpha      0        0.0

//  Alpha    1.0        1.0

[jamesatdrastic]

Thanks Zac!  That's exactly what I needed

J