id summary reporter owner description type status priority milestone component version resolution keywords cc
544 Non-normative aspects in scaling process (8.6.3) should be removed fbossen bbross "Equations (8-262), (8-264), and (8-270) should be removed as they relate to a specific implementation to avoid 32-bit overflow in some corner cases. Removing these equations has no normative impact.
levelLimit defined in equations (8-262) and (8-264) takes value 15 most of the time, and value 14 for a combination of log2TrSize=2 and large qP values such that BitDepth - qP/6 = 0.
The case where levelLimit is 15 is not relevant anymore following adoption of I0254.
In the case where levelLimit is 14, the clipping to the -16384..16383 range defined in (8-270) has no effect on the value computed in (8-271) since (8-271) scales the output of (8-270) by a factor of at least 40 and defines another clipping operation to the -32768..32767 range. The clipping operation of (8-270) should thus be removed from the specification as it has no normative effect.
For simplicity, it may also be preferable to define the scaling process using a single equation (without changing the normative behavior):
d_ij = Clip3( −32768, 32767,( ( c_ij * M[i][j] * levelScale[ qP%6 ] << ( qP/6 ) ) + ( 1 << ( shift + 3 ) ) ) >> (shift + 4) )
where M[i][j] = 16 if scaling_list_present_flag is equal to 0
It may be appropriate to add a note to clarify than for c_ij = -32768, M[i][j] = 255, qP = 51, and shift = 1, the correct value of d_ij is -32768 (and not 32767, which is the value one might get if not correctly implementing corner cases).
" defect closed minor D7 Text D7 (I1003) d1 fixed bbross wjhan jct-vc@…