I just decoded with HM-9.1. As far as I know, these offsets are not used in the decoder. Are you using a modified software like the one in #942?

stream with entropy sync enabled in which HM 9.1 decoder crashes after decoding one CTB row. The docoding of second row starts at wrong offset as it ignores the emuation prevention bytes in the first CTB row.

I am sorry i had attached a wrong file. I have replaced the older file with bitstream which crashes when tried decoding with HM 9.1 and the description holds good for this file. I confirm that i am using unmodified version of HM 9.1 decoder. The following code snippet is used to extract substream based on entry_point_offset.

Function - Void TDecGop::decompressSlice(TComInputBitstream* pcBitstream, TComPic*& rpcPic)

  UInt uiNumSubstreams = pcSlice->getPPS()->getEntropyCodingSyncEnabledFlag() ? pcSlice->getNumEntryPointOffsets()+1 : pcSlice->getPPS()->getNumSubstreams();

  // init each couple {EntropyDecoder, Substream}
  UInt *puiSubstreamSizes = pcSlice->getSubstreamSizes();
  ppcSubstreams    = new TComInputBitstream*[uiNumSubstreams];
  m_pcSbacDecoders = new TDecSbac[uiNumSubstreams];
  m_pcBinCABACs    = new TDecBinCABAC[uiNumSubstreams];
  for ( UInt ui = 0 ; ui < uiNumSubstreams ; ui++ )
  {
    m_pcSbacDecoders[ui].init(&m_pcBinCABACs[ui]);
    ppcSubstreams[ui] = pcBitstream->extractSubstream(ui+1 < uiNumSubstreams ? puiSubstreamSizes[ui] : pcBitstream->getNumBitsLeft());
  }

The value of entry_point_offset is used to extract the substream and HM 9.1 decoder does not include emulation prevention bytes.

Would it be possible to share the exact location of the emulation prevention bytes (in the NALU) and the CTB row sizes as seen by the encoder (in entropy_sync_enabled.265)? If the bitstream is generated using HM, could you please share the HM version and encoder configuration used so that the problem can be reproduced.

Replying to kiranmisra:

Would it be possible to share the exact location of the emulation prevention bytes (in the NALU) and the CTB row sizes as seen by the encoder (in entropy_sync_enabled.265)? If the bitstream is generated using HM, could you please share the HM version and encoder configuration used so that the problem can be reproduced.

I am not using HM encoder to generate bitstream. I am using a fixed CTB size of 64x64 and encoding entire frame as single slice. The uploaded file is single IDR frame(blank content) The offset of first emulation prevention bytes in slice data (we can ignore emulation prevention bytes in header data as this will not affect entry_point_offset) is listed below
First emulation prevention byte offset from frame start - 150
FIrst emulation prevention byte offset from start of slice data - 18
This emualtion prevention byte is part of the first CTB row (which was successfully parsed).

I have tested our encoder's entropy sync feature for other contents and i dont see any mismatch between our encoder and HM decoder(version 9.1). I did not observe emulation prevention bytes in slice data of streams which passed the test. However when i try to encode blank frame( this increases possiblity of emulation prevention bytes) i see mismatch in trace of decoder at start of second row. The decoder starts decoding the second row at wrong offset, the difference in offset of start of actual second row data and offset where decoder starts decoding is same as number of emulation prevention bytes inserted in first CTB row.

Proposed patch for substream entry point information

Thank you for the explanation of the contents within the bitstream entropy_sync_enabled.265. Attached is a proposed patch which keeps track of emulation prevention bytes within a NALU. The decoder uses this information to update the entry point information. The proposed patch also updates the encoder to include emulation prevention bytes when determining entry point information for wavefront substreams.

Thank you for the patch. The problem seems to have got resolved with this patch. Please let me know when this change gets reflected in main release of HM.

I have adapted the patch to HM-10.0-dev and applied it in r3363

The applied patch has bug that results in failure to count consecutive start code emulations properly.

The code has been fixed in r3376. The fix also resolves the same issue for tiles.