id summary reporter owner description type status priority milestone component version resolution keywords cc 1076 Encoder generates a non-conforming bitstream by encoding a coefficient level of +32768 when sign hiding is enabled livium "I encountered a case where the encoder generated a bitstream containing a +32768 coefficient level when sign hiding is enabled. After some debugging, the code responsible is located inside the function xRateDistOptQuant() of TComTrQuant.cpp. Relevant code (lines 1988-2000): {{{ if(piQCoef[minPos] == 32767 || piQCoef[minPos] == -32768) { finalChange = -1; } if(plSrcCoeff[minPos]>=0) { piDstCoeff[minPos] += finalChange ; } else { piDstCoeff[minPos] -= finalChange ; } }}} Adding a printf after the first 'if' block: printf(""xRateDistOptQuant: minPos=%d plSrcCoeff[minPos]=%d piQCoef[minPos]=%d piDstCoeff[minPos]=%d finalChange=%d\n"", minPos, plSrcCoeff[minPos], piQCoef[minPos], piDstCoeff[minPos], finalChange); I got the following output (for a DC coefficient): xRateDistOptQuant: minPos=0 plSrcCoeff[minPos]=20059 piQCoef[minPos]=124277 piDstCoeff[minPos]=32767 finalChange=1 As a result, the second 'if' adjusted the coefficient level to +32768 which got placed into the bitstream. Two possible solutions: 1. Add saturation to [-32768, +32767] for piDstCoeff[minPos] after the second 'if' block. 2. It's very likely that the first 'if' is wrong and piDstCoeff should be checked instead of piQCoef. A similar piece of code can be found in lines 969-981 of the same file. " defect closed minor HM-10.1 HM HM-10.0 duplicate fbossen ksuehring davidf jct-vc@…