We may have found a bug in the HM model. There is a mismatch between the written specification and HM reference C model.

Dec 2016 Spec: 8.4.4.2.7 Decoding process for palette mode

HM code version : HM-16.15+SCM-8.5 Function : TDecCu::xDecodePaletteTexture() in TDecCu.cpp The problem equation happens in both clauses of an if-else statement. The overflow happens in tmpVal, which is used to store the inverse quantized palette escape values in (8-77). tmpVal is then clipped according to (8-78) to generate the reconstructed samples. levelScale <= 72 (7-bit). In SCM, tmpVal is of type “Pel”. So, once we changed Pixel from Short to Int the overflow was fixed.

It is a high bit depth problem, most likely only happen when pixel depth is > 8-bit.

There is not enough precision in the internal stages of the Adaptive Color Transform. Solution : The intermediate ACT stages must use more than 16b to avoid overflow, when not using HIGH_BIT_DEPTH_SUPPORT.

Section 8.6.8.2 (H.265 12/2016 spec) – Residual samples rY[ x ][ y ], rCb[ x ][ y ] and rCr[ x ][ y ] are modified as follows: tmp = rY[ x ][ y ] − ( rCb[ x ][ y ] >> 1 ) (8-338) rY[ x ][ y ] = tmp + rCb[ x ][ y ] (8-339) rCb[ x ][ y ] = tmp − ( rCr[ x ][ y ] >> 1 ) (8-340) rCr[ x ][ y ] += rCb[ x ][ y ] (8-341)

When without HIGH_BIT_DEPTH_SUPPORT, HM C-model using only Short (16b; typedef Short Pel) to store pDst0/1/2.

HM-16.15+SCM-8.5 File : TComYuv.cpp, function Void TComYuv::convert() Pel* pDst0

Int t = yo - (cg>>1); pDst0[x] = cg + t; pDst1[x] = t - (co>>1); pDst2[x] = co + pDst1[x];

Thanks Sam.

In SCM8.5, after getInterMergeCandidates(), the merge candidate bi-pred restriction, i.e. xRestrictBipredMergeCand(), is done before the merge candidate rounding, i.e. roundMergeCandidates(). But in the Spec, the merge candidate list generation follows the below steps, where the bi-pred restriction should be done after the merge candidate rounding. This caused the inconsistence between SCM and Spec.

Step 1: Derive the merge candidates (step 1-8 in spec 8.5.3.2.2, and SCM function is getInterMergeCandidates()) Step 2: Round the merge candidates if needed (step 9 in spec 8.5.3.2.2, and SCM function is roundMergeCandidates()) Step 3: Disallow 8x4 and 4x8 bi-pred merge candidates(step 10 in spec 8.5.3.2.2, and the SCM function is xRestrictBipredMergeCand()) Step 4: Disallow bi-pred merge candidates for some 8x8 CUs (the below part in spec 8.5.3.2.1, and the SCM function is xRestrictBipredMergeCand())

When all of the following conditions are true, refIdxL1 is set equal to −1 and predFlagL1 is set equal to 0. – predFlagL0 is equal to 1. – predFlagL1 is equal to 1. – nPbSw is equal to 8. – nPbSh is equal to 8. – TwoVersionsOfCurrDecPicFlag is equal to 1. – noIntegerMvFlag is equal to 1. – identicalMvs is equal to 0.