Thank you Peter.

This has highlighted an issue with the possible range of the residual due to cross-component-prediction, and a corresponding thread will be created on the main JCTVC email reflector.

With cross-component-prediction, the output of the inverse-transforms (inverse-DCT/DST, inverse-RDPCM, inverse-transform-skip, inverse-transquant-bypass etc) are combined: a fraction of the luma is added to the chroma.

The specification does not have a clipping stage at the output of the inverse transforms; however, in the software, a clipping stage has been applied at the end of the inverse-transform to protect the resulting data from overflowing the size of the 16-bit Pel residual array. This was previously valid for Ver.1 since the only processing-stages following were the addition of the residual to the prediction and then clipping to the internal bit depth (IBD).

With CCP present, the clipping has to be removed - it cannot be clipped at the end of the transform, because there is a possibility that a large luma value is combined with a large negative chroma value and turns into a small value.

It is likely that this only occurs for pathological / "designed-to-be-evil" bit-streams.

Here are some calculations (please check) showing the output bit depths of the various tools using non-extended-precision and various internal bit depths (IBD), and highlighting cases that require residuals to be more than 16-bit.

(* Not an input to cross-component-prediction.)

For now, the work-around to use the HIGH_BIT_DEPTH_SUPPORT=1 build (which uses 32-bit Pels) is valid, although this will double the run-time memory footprint.

Modified active ticket for RExt component to HM component.

At the 19th JCTVC meeting in Strasbourg, the JCTVC concluded that a
restriction should be placed on the bit-stream, as such overflow
conditions are not expected to be generated by a real encoder. This
restriction has been marked for inclusion into the HEVC Screen
Content Coding Draft Text.

The restriction would likely be placed in Section 8.6.6: Residual
modification process for transform blocks using cross-component
prediction, and would be something similar to:

The bitstream shall not cause the luma residual
values rY to be outside of the inclusive range
defined by (CoeffMinY to CoeffMaxY), nor the residual
samples r to be outside of the inclusive range
defined by (CoeffMinC to CoeffMaxC).

The restriction may be further refined during the development of
Version 3.

In HM (& SCM) the software function
TComTrQuant::crossComponentPrediction still needs to be slightly
modified: the pResiL[x] should be cast to an Intermediate_Int
and a clipping stage should be added to the software to prevent
the modified residual exceeding the range of a Pel.

For reference, the calculation is:

Let YCBitDiff be the difference between the luma and chroma bit-depths

For each sample in the block:

Let chromaResi be the (restricted) chroma residual value.
Let lumaResi be the (restricted) luma residual value.
Let shiftedLuma be lumaResi shifted left/right by YCBitDiff.
result = chromaResi + ((alpha * shiftedLuma)>>3)

The worst case of the CCP residual calculation
for extended-precision disabled is when luma's
bit depth is 8, and chroma's bit depth is 16. In
this case lumaResi (eg -32768) will be left shifted
8 places to be 24-bit (shiftedLuma=-8388608).
The result then gets multiplied by alpha, and
is potentially 28-bit (-8*-8388608=67108864).

The worst case of the CCP residual calculation
for extended-precision enabled is 23+8+4=35 bit.

Hence 32-bit (or 64-bit for extended-precision)
must be used for the calculation, and a clipping
stage is required to protect the Pel type.

As indicated previously, the constraint might be further refined by
the JCT-VC.

For example, does the case of 16-bit with extended precision disabled
actually work?

Constraining the inputs to CCP to the range of -32768 to 32767 does
not seem like a good solution in this case (given the valid range
of lossless residuals at the output of CCP would be -65535 to 65535).

Perhaps restricting the values at the input to CCP to bitDepthY+4
and bitDepthC+4 signed-bits might be a more appropriate solution?

This would also prevent the size of shiftedLuma in the above
calculation becoming too large.

• Note that I had an error in my previous update to this bug-report: I believe the worst case of the CCP residual calculation for extended-precision enabled is 16+8+4=28 bit, not 35 bit.

Software fix applied in r4227, assuming that inputs for 12-bit and below will not exceed 16-bit signed and beyond 12-bit will not exceed 23-bit signed. Modifying ticket to become a SCC text ticket as a reminder to editors.

The meeting notes (JCTVC-T1000) indicated the following during the review of JCTVC-T0003:

This issue was discussed at the previous JCT-VC meeting in Strasbourg and it was agreed that a constraint could be added to the next version of the HEVC specification. (However, it was not yet incorporated into a draft text output document.)

...

Decision: As previously planned, for RExt purposes, specify a normative constraint that limits inputs to the inverse CCP to the range of the quantized coefficients CoeffMinY/C to CoeffMaxY/C. (Include in SCC draft output.)

Constraint added to HEVC v4.