HM 5.0 decoder crashes in AI_HE configuration in sequences Kimono,ChinaSpeed,SlidShows at QP=12 with the following assertion failed message:

Void TComInputBitstream::read (UInt uiNumberOfBits, UInt& ruiBits) {

............. assert(m_fifo_idx + num_bytes_to_load < m_fifo->size()); .............

}

Minor Bug

SAD and HAD called during Inter where pixel values may be out of range, negative or exceed upper limit of given bit depth due to TComYuv::removeHighFreq(...)

Within TEncSearch::xMotionEstimation(..), when bBi is enabled, TcomYuv::removeHighFreq(...) is called which attempts to average out the Dst with the Src pix values as follows:

pDst[x ] = (pDst[x ]<<1) - pSrc[x ] ;

However, in not all cases are they similar values, and so it is possible to that a negative values or that exceeding the bit-depth range would be stored.

Previously, in Ticket 175 the issue was raised that clipping was hindering performance and hence the code checks if the flag for disabling clip has been enabled.

#if DISABLING_CLIP_FOR_BIPREDME

The issue with clipping is it only stops if it exceeds the bit-depth range and does not handle if the value is negative.

As a consequence, when SAD or Hadamard Distortion Metrics are called within TcomRdCost, block pairs may contain invalid pixel values, resulting in lower or highter distortion scores including exceeding the maximum for the given bit depth. As such, for those block pairs that are incorrectly scored, this can affect the choices undertaken by the encoder.

Taking observations of when Dst is assigned values that are invalid, test conditions were produced. Src would be assessed against and upper and lower condition in order to decide average out Dst with Src.

The condition are linear equations, adding minimal additional complexity. The lower limit test is y > 2x + 1, where x is Dst and Src is y, the upper condition is y <= 2x – 2^BitDepth.

int Dst2 = pDst[x]<<1; pDst[x ] = pSrc[x ] > (Dst2 + 1) && pSrc[x] <= Dst2-iMaxValBitDepthY ?

Dst2 - pSrc[x ] : pDst[x ];

where iMaxValBitDepthY is defined prior to the loop as:

int iMaxValBitDepthY = 2<<g_bitDepthY;

This can be extended to the chroma parts in a similar fashion.

(Before Loop)

Added to checking pSrc int iMaxValBitDepthC = 2<<g_bitDepthC;

(Within Loop)

Fix - Ensures Pixel Values are within Range int DstU2 = pDstU[x]<<1; int DstV2 = pDstV[x]<<1;

pDstU[x ] = pSrcU[x ] > (DstU2 + 1) && pSrcU[x] <= DstU2-iMaxValBitDepthC ? DstU2 - pSrcU[x ] : pDstU[x ];

pDstV[x ] = pSrcV[x ] > (DstV2 + 1) && pSrcV[x] <= DstV2-iMaxValBitDepthC ? DstV2 - pSrcV[x ] : pDstV[x ];

The solution was tested on the first three frames of an 8 bit, 10 bit and 12 bit video, where no subsequent invalid pixel values captured after new Dst value has been assigned within TcomYuv::removeHighFreq(...).

Below is a table of a series of brief tests conducted with the differences between original and that with the modified removeHighFreq() method. A short GOP of 5 was used.

Video Resolution BitDepth Frames Bit-Rate Y-PSNR Time RaceHorses 416x240 8 30 1.04% -0.0579 0.49% OldTown 1920x1080 10 20 0.07% -0.0008 -0.48% Traffic 2560x1600 12 10 0.13% 0.001 -0.24%

Minor Bug

Changes to TComSlice.h required in order to compile with GCC 4.8.1

When compiling with GCC 4.8.1 TComSlice.h is flagged up multiple times with a single compiler error ("array subscript is above array bounds").

In effect, the compiler does not like it that tLayer (or tlayer is is called elsewhere) is assumed to be valid, it could be abused and thus unsafe.

To resolve this fear, test tLayer against getMaxTLayers to decide if it is safe prior to accepting or reverting to the maximum possible value. e.g.

m_numReorderPics[(tLayer = tLayer > getMaxTLayers ()? getMaxTLayers () : tLayer)] = v;

Making the following modifications will suppress these errors and the codebase will compile.

------------- Around Line 450 Change from

Void setNumReorderPics(UInt v, UInt tLayer) { m_numReorderPics[tLayer] = v; }

To

Void setNumReorderPics(UInt v, UInt tLayer) { m_numReorderPics[(tLayer = tLayer > getMaxTLayers ()? getMaxTLayers () : tLayer)] = v; }

Change from

Void setMaxDecPicBuffering(UInt v, UInt tLayer) { m_uiMaxDecPicBuffering[tLayer] = v; } Void setMaxDecPicBuffering(UInt v, UInt tLayer) { m_uiMaxDecPicBuffering[(tLayer = tLayer > getMaxTLayers ()? getMaxTLayers () : tLayer)] = v; }

Change from

Void setMaxLatencyIncrease(UInt v, UInt tLayer) { m_uiMaxLatencyIncrease[tLayer] = v; }

To

Void setMaxLatencyIncrease(UInt v, UInt tLayer) { m_uiMaxLatencyIncrease[(tLayer = tLayer > getMaxTLayers ()? getMaxTLayers () : tLayer)] = v; }

-------------

------------- Around Line 820 Change from Void setNumReorderPics(Int i, UInt tlayer) { m_numReorderPics[tlayer] = i; } To

Void setNumReorderPics(Int i, UInt tlayer) { m_numReorderPics[(tlayer = tlayer > getMaxTLayers ()? getMaxTLayers () : tlayer)] = i; }

Change from

Void setMaxDecPicBuffering ( UInt ui, UInt tlayer ) { m_uiMaxDecPicBuffering[tlayer] = ui; }

To

Void setMaxDecPicBuffering ( UInt ui, UInt tlayer ) { m_uiMaxDecPicBuffering[(tlayer = tlayer > getMaxTLayers ()? getMaxTLayers () : tlayer)] = ui; }

Change from

Void setMaxLatencyIncrease ( UInt ui , UInt tlayer) { m_uiMaxLatencyIncrease[tlayer] = ui; }

To

Void setMaxLatencyIncrease ( UInt ui , UInt tlayer) { m_uiMaxLatencyIncrease[(tlayer = tlayer > getMaxTLayers ()? getMaxTLayers () : tlayer)] = ui; }

-------------