Opened 3 years ago

Last modified 3 years ago

#1304 new defect

Unclear decoder DPB behaviour

Reported by: ralfw Owned by:
Priority: minor Milestone:
Component: HM Version: HM-14.0
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

We have generated a simple bitstream using our encoder with the following sequence parameters:
sps_max_dec_pic_buffering_minus1 = 3 ( = DPB size 4 )
sps_max_num_reorder_pics = 2
sps_max_latency_increase_plus1 = 0

The stream has a GOP structure like this:

Decode Order:

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

POC:

0 4 2 1 3 8 6 5 7 12 10 9 11 16 13 13 15

Decoding the stream with HM-15.0 produces the following output ( MD5 values removed ):

HM software: Decoder Version [15.0][Windows][VS 1600][64 bit]

POC 0 TId: 0 ( I-SLICE, QP 25 ) [DT 0.015] [L0 ] [L1 ] [MD5:,(OK)]
POC 4 TId: 0 ( B-SLICE, QP 25 ) [DT 0.042] [L0 0 ] [L1 0 ] [MD5:,(OK)]
POC 2 TId: 0 ( B-SLICE, QP 25 ) [DT 0.006] [L0 0 4 ] [L1 4 0 ] [MD5:,(OK)]
POC 1 TId: 0 ( B-SLICE, QP 25 ) [DT 0.006] [L0 0 2 4 ] [L1 2 4 0 ] [MD5:,(OK)]
POC 3 TId: 0 ( B-SLICE, QP 39 ) [DT 0.013] [L0 2 0 4 ] [L1 4 2 0 ] [MD5:,(OK)]
POC 8 TId: 0 ( B-SLICE, QP 40 ) [DT 0.004] [L0 4 2 0 ] [L1 4 2 0 ] [MD5:,(OK)]
POC 6 TId: 0 ( B-SLICE, QP 39 ) [DT 0.036] [L0 4 2 8 ] [L1 8 4 2 ] [MD5:,(OK)]
POC 5 TId: 0 ( B-SLICE, QP 38 ) [DT 0.003] [L0 4 6 8 ] [L1 6 8 4 ] [MD5:,(OK)]
POC 7 TId: 0 ( B-SLICE, QP 39 ) [DT 0.003] [L0 6 4 8 ] [L1 8 6 4 ] [MD5:,(OK)]
POC 12 TId: 0 ( B-SLICE, QP 38 ) [DT 0.005] [L0 8 6 4 ] [L1 8 6 4 ] [MD5:,(OK)]
POC 10 TId: 0 ( B-SLICE, QP 39 ) [DT 0.004] [L0 8 6 12 ] [L1 12 8 6 ] [MD5:,(OK)]
POC 9 TId: 0 ( B-SLICE, QP 39 ) [DT 0.017] [L0 8 10 12 ] [L1 10 12 8 ] [MD5:,(OK)]
POC 11 TId: 0 ( B-SLICE, QP 38 ) [DT 0.004] [L0 10 8 12 ] [L1 12 10 8 ] [MD5:,(OK)]
POC 16 TId: 0 ( B-SLICE, QP 36 ) [DT 0.006] [L0 12 10 8 ] [L1 12 10 8 ] [MD5:,(OK)]
POC 14 TId: 0 ( B-SLICE, QP 37 ) [DT 0.005] [L0 12 10 16 ] [L1 16 12 10 ] [MD5:,(OK)]
POC 13 TId: 0 ( B-SLICE, QP 37 ) [DT 0.005] [L0 12 14 16 ] [L1 14 16 12 ] [MD5:,(OK)]
POC 15 TId: 0 ( B-SLICE, QP 37 ) [DT 0.004] [L0 14 12 16 ] [L1 16 14 12 ] [MD5:,(OK)]

Total Time: 0.215 sec.

For the picture with POC=8 the HM decoder executes the following code in TDecTop::xGetNewPicBuffer:

  if ( !bBufferIsAvailable )
  {
    //There is no room for this picture, either because of faulty encoder or dropped NAL. Extend the buffer.
    m_iMaxRefPicNum++;
    rpcPic = new TComPic();
    m_cListPic.pushBack( rpcPic );
  }

The comment in the source code suggests a fault in our encoder.

The RPS for the picture with POC=8 is as follows:
Number of Negative Pictures: 3
Delta POCs: -4 -6 -8
Number of Positive Pictures: 0
No Longterm.

At this point ( before the execution of the code ) the decoder DPB contains the following 4 pictures:
0: POC=0, Referenced=true, Needed for Output = false
1: POC=2, Referenced=true, Needed for Output = false
2: POC=3, Referenced=false, Needed for Output = true
3: POC=4, Referenced=true, Needed for Output = true

The previous picture output by the decoder had POC=2.

We are currently interpetrating the point of code to be in the decode process after parsing the slice header of the first slice of the picture with POC=8 but before decoding of the picture with POC=8. ( The RPS has already been applied ).

We are currently also considering the possibility that the HM-15.0 decoder slightly inaccurate because, if the full output and removal of pictures from the DPB process of C.5.2.2 had been applied after the application of the RPS but before the call to TDecTop::xGetNewPicBuffer, we believe the picture with POC=3 might has been output and removed from the DPB.

We find this a possibility because C.5.2.2 states that for the non IRAP picture case the bumping process in C.5.2.4 is invoked if the number of pictures in the DPB is greater than or equal to sps_max_dec_pic_buffering_minus1 + 1, which, so we think, should have output the picture with POC=3 and marked it as not needed for output so it is removed from the DPB.

We tested this hypothesis in a somewhat crude way by calling the TAppDecTop::xWriteOutput method from inside TDecTop::xDecodeSlice after the call to m_apcSlicePilot->applyReferencePictureSet(m_cListPic, m_apcSlicePilot->getRPS()); but before the call to xGetNewPicBuffer (m_apcSlicePilot, pcPic); and, for this case, checking in xWriteOutput for dpbFullness >= maxDecPicBufferingHighestTid. The result of this experiment is that the picture with POC=3 is output by the call to xWriteOutput() and removed from the DPB after applying the RPS of the picture with POC=8.

The modified decoder also did not execute the code in question above for the whole sequence.

Creating a stream with the standard HM configuration files for random access and a sequence like so:
"TAppEncoder -c encoder_randomaccess_main.cfg -c Kimono.cfg" also results in a stream that let the decoder execute the code in question.

So if our assumption is correct the decoder seems to be implementing the bumping/output process of pictures from the DPB slightly inefficiently and might confuse people looking for a reference example.

Attachments (1)

gop4_dpb4_reordering2.hevc (28.6 KB) - added by ralfw 3 years ago.
our example stream

Download all attachments as: .zip

Change History (5)

comment:1 Changed 3 years ago by DefaultCC Plugin

  • Cc fbossen ksuehring davidf jct-vc@… added

Changed 3 years ago by ralfw

our example stream

comment:2 Changed 3 years ago by ksuehring

  • Type changed from technical change to defect

comment:3 Changed 3 years ago by adarsh

I agree that the example brings out the inefficient implementation of the bumping process in HM.

In SHM, (branch SHM-dev: https://hevc.hhi.fraunhofer.de/svn/svn_SHVCSoftware/branches/SHM-dev), the DPB is implemented such that the bumping process/picture output is called after the decoding of the RPS. I tried decoding this bitstream ( gop4_dpb4_reordering2.hevc) with the SHM code and it decodes fine without expanding the "DPB."

I can volunteer to provide a patch for HM (similar to SHM) to implement the DPB such that the the bumping is done after the RPS derivation. I will need some time to do it, though.

Adarsh

comment:4 Changed 3 years ago by ksuehring

Thanks Adarsh for volunteering. Any help is appreciated.

I'm not sure if there is any difference in the RExt branch in this part of the software. But it might be a good idea to work directly on that branch, which is intended to be merged into mainline HM soon.

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • Adarsh Krishnan Ramasubramonian(Participant)
  • David Flynn(Subscriber, Always)
  • Frank Bossen(Subscriber)
  • jct-vc@…(Subscriber)
  • Karl Sharman(Always)
  • Karsten Suehring(Subscriber, Participant, Always)
  • ralfw(Reporter)