According to 8.7.3.2. “Coding tree block modification process” SAO filtering should be disabled for current CU if cu_transquant_bypass_flag is equal to 1 regardless of CU encoding mode:

HEVC SCC Spec:

If one or more of the following conditions are true, saoPicture[ xSi ][ ySj ] is not modified:

1. pcm_loop_filter_disabled_flag and pcm_flag[ xYi ][ yYj ] are both equal to 1.
2. cu_transquant_bypass_flag is equal to 1.
3. SaoTypeIdx[ cIdx ][ rx ][ ry ] is equal to 0.

HM decoder (encoder) performs SAO filtering for CU encoded in palette mode even if cu_transquant_bypass_flag is equal to 1. (see function xPCMSampleRestoration which restored pixels in case of PCM or if cu_transquant_bypass_flag is equal to 1)

HM code:

Void TComSampleAdaptiveOffset::xPCMSampleRestoration (TComDataCU* pcCU, UInt uiAbsZorderIdx, UInt uiDepth, const ComponentID compID) {

if (pcCU->getPaletteModeFlag(uiAbsZorderIdx)) {

return;

} TComPicYuv* pcPicYuvRec = pcCU->getPic()->getPicYuvRec();

To check if palette_mode_flag should be inserted to bitstream or not HM software compared CU size with fixed value <= 32 (or < 64) while spec compared CU size with max transform size:

HEVC SCC Spec:

if( palette_mode_enabled_flag && CuPredMode[ x0 ][ y0 ] = = MODE_INTRA &&

log2CbSize <= MaxTbLog2SizeY ) palette_mode_flag[ x0 ][ y0 ]

HM code:

================================

Void TDecEntropy::decodePaletteModeInfo( TComDataCU* pcCU, UInt uiAbsPartIdx, UInt uiDepth, Bool& bCodeDQP, Bool& isChromaQpAdjCoded ) {

Note: the condition is log2CbSize < MaxTbLog2SizeY in 7.3.8.5 of JCTVC-T1005-v2 if( pcCU->getSlice()->getSPS()->getSpsScreenExtension().getUsePaletteMode() && pcCU->isIntra( uiAbsPartIdx ) && (pcCU->getSlice()->getSPS()->getMaxCUWidth() >> uiDepth) < 64 ) {

m_pcEntropyDecoderIf->parsePaletteModeFlag( pcCU, uiAbsPartIdx, uiDepth ); if ( pcCU->getPaletteModeFlag( uiAbsPartIdx ) ) {

m_pcEntropyDecoderIf->parsePaletteModeSyntax( pcCU, uiAbsPartIdx, uiDepth, 3, bCodeDQP, isChromaQpAdjCoded ); pcCU->saveLastPaletteInLcuFinal( pcCU, uiAbsPartIdx, MAX_NUM_COMPONENT );

}

}

}

According to “Note” HM developers aware of this difference

I used VS2013 to compile the latest HM (HM-dev r4815) into x64 Debug version and Release version. The command line I used was TAppEncoder -c encoder_lowdelay_main.cfg -c BQSquare.cfg -f 1

The coding result of Release build is POC 0 TId: 0 ( I-SLICE, nQP 32 QP 32 ) 91576 bits [Y 33.5217 dB U 39.4202 dB V 40.0321 dB] [ET 1 ] [L0 ] [L1 ] The coding result of Debug build is POC 0 TId: 0 ( I-SLICE, nQP 32 QP 32 ) 89352 bits [Y 33.3534 dB U 39.4079 dB V 39.8650 dB] [ET 8 ] [L0 ] [L1 ]

I've checked other versions of HM. r4793 seems to be OK (Debug and Release version generate identical results, which is the same as the Release result above). But r4794 has the same problem. I've also checked that disabling SHARP_LUMA_DELTA_QP in r4815 can also solve the problem.