Opened 11 years ago

Closed 9 years ago

Last modified 9 years ago

#1070 closed defect (fixed)

HM performs loop filtering after replacing SPS/PPS

Reported by: pieterkapsenberg Owned by:
Priority: major Milestone: HM-16.3
Component: HM Version: HM-10.0
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

In HM-10.0, loop filtering and SEI digest checking is performed after the first slice header of the next picture is decoded. By then, the SPS and PPS needed to deblock & SAO-filter the finished picture may have already been replaced by new ones with the same ID.

A patch is attached that fixed the issue. It changes the overal decoder flow a bit:

  • The slice decode updates a boolean that indicates if the decompressed slice was the last of the pic, as detected by last CU address being equal to numCUsInFrame-1.
  • If the boolean is true, perform loop filtering.
  • When SEI digest message arrives, perform the check then. Digest code was moved from TDecGop to TDecTop

Attachments (4)

lf_fix.patch (14.0 KB) - added by pieterkapsenberg 11 years ago.
lf_fix_2.patch (15.5 KB) - added by pieterkapsenberg 11 years ago.
modified to handle dependent slices and the case where SEI arrives before the last slice of the picture
TDecTop.cpp.patch (1.4 KB) - added by pieterkapsenberg 11 years ago.
comments out SEI parameter sets
lf_fix_hm10dev.patch (15.6 KB) - added by pieterkapsenberg 11 years ago.
loop filter patch updated for HM10.0-dev

Download all attachments as: .zip

Change History (25)

comment:1 Changed 11 years ago by DefaultCC Plugin

  • Cc fbossen ksuehring davidf jct-vc@… added

Changed 11 years ago by pieterkapsenberg

Changed 11 years ago by pieterkapsenberg

modified to handle dependent slices and the case where SEI arrives before the last slice of the picture

comment:2 Changed 11 years ago by ksuehring

I think that creates an issue when a decoded picture hash message arrives between two VCL NAL units of a picture.

SPS and PPS are not directly overwritten, see ParametersetManagerDecoder. They are stored in a "prefetch"-buffer and only overwrite the old parameter sets when applyPrefetchedPS() is called. This is called only with the activation of a new parameter set, which should happen in decoding, not parsing.

There exist two functions for accessing either a new parameter set (getPrefetched{VSP}PS) or an old one (get{VSP}PS).

comment:3 Changed 11 years ago by pieterkapsenberg

The second patch should handle the case of the SEI showing up between two slices. Regarding the operation of the ParametersetManagerDecoder, I think current HM still ends up replacing the PPS/SPS before loop filtering when a parameter activation SEI message shows up.

Changed 11 years ago by pieterkapsenberg

comments out SEI parameter sets

comment:4 Changed 11 years ago by pieterkapsenberg

I suppose a simpler fix is to just have HM ignore parameter activation SEIs, since SEIs are ignorable. Patch that does this is attached, and also solves the issue I was seeing.

comment:5 Changed 11 years ago by ksuehring

Unfortunately for a stream that contains buffering and timing SEI the startup won't work when ignoring the SEI (i.e. an SPS need to be active before buffering and timing can be parsed). Maybe we can ignore the SEI in the middle of the bitstream.

comment:6 Changed 11 years ago by pieterkapsenberg

That would be ok, it is still simpler than the second patch that initiates loop filtering after the picture completes, although I do think that seems more "correct".

comment:7 Changed 11 years ago by ksuehring

I would like to test lf_fix_2. Filtering at the end of the picture seems to be helpful to also avoid reading the slice header twice. Unfortunately the patch does not apply to the current HM-10.0-dev branch. Could you please update it?

Changed 11 years ago by pieterkapsenberg

loop filter patch updated for HM10.0-dev

comment:8 Changed 11 years ago by pieterkapsenberg

Attached

comment:9 Changed 11 years ago by pieterkapsenberg

ok the patch needs a small tweak, otherwise SEIs of skipped leading pictures will try to get applied to the previously decoded IRAP.

In TDecTop::xDecodeSlice, add the m_digestCanBeChecked assignments:

  if (isRandomAccessSkipPicture(iSkipFrame, iPOCLastDisplay))
  {
    m_digestCanBeChecked = false;
    return false;
  }
  // Skip TFD pictures associated with BLA/BLANT pictures
  if (isSkipPictureForBLA(iPOCLastDisplay))
  {
    m_digestCanBeChecked = false;
    return false;
  }

comment:10 follow-up: Changed 11 years ago by conrad.ho

The activation SEI can activate VPS, SPS, but not PPS. VPS and PPS do not contain SAO and DBF related parameters. So I'm wondering why activation SEI will make it wrong?

comment:11 in reply to: ↑ 10 Changed 11 years ago by pieterkapsenberg

Replying to conrad.ho:

The activation SEI can activate VPS, SPS, but not PPS. VPS and PPS do not contain SAO and DBF related parameters. So I'm wondering why activation SEI will make it wrong?

The real problem is that loop filter code gets the pic height & width from the newly activated SPS, which isn't parsed yet (all the values are garbage).

comment:12 Changed 11 years ago by pieterkapsenberg

Sorry, my previous statement is incorrect. The slice's pointer to the old SPS refers to de-allocated memory by the time the loop filter runs.

comment:13 Changed 11 years ago by pieterkapsenberg

  • Milestone changed from HM-10.1 to HM-10.2

comment:14 Changed 11 years ago by ksuehring

  • Resolution set to fixed
  • Status changed from new to closed

The patch has been applied in r3497.

There are alternative ways to solve the issue, but they might require more work.

comment:15 Changed 11 years ago by ksuehring

  • Resolution fixed deleted
  • Status changed from closed to reopened

Reopening. The patch was reverted in trunk r3508 because it breaks decoding of conformance streams SLIST_B and SLIST_D

comment:16 Changed 11 years ago by ksuehring

A severe effect is that the output picture is wrong from the beginning while the MD5 check fails only much later. We should make sure this does not happen.

comment:17 Changed 11 years ago by pieterkapsenberg

Perhaps each slice can simply allocate and keep a duplicate copy of its SPS/PPS, and deallocate in the destructor.

comment:18 Changed 11 years ago by pieterkapsenberg

  • Milestone changed from HM-11.0 to HM-11.1

comment:19 Changed 10 years ago by ksuehring

  • Milestone HM-11.1 deleted

comment:20 Changed 9 years ago by karlsharman

  • Resolution set to fixed
  • Status changed from reopened to closed

Fixed in r4219 by placing a constant instance of the SPS and PPS within
TComPicSym. The SPS/PPS pointers within a slice point to the instance
within the TComPicSym. The decoder will abort if the active PPS/SPS
changes whilst decoding a picture.

comment:21 Changed 9 years ago by karlsharman

  • Milestone set to HM-16.3

Batch modify of recently closed tickets to set milestone to HM-16.3.

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • Conrad Ho(Participant)
  • David Flynn(Subscriber)
  • Frank Bossen(Subscriber)
  • jct-vc@…(Subscriber)
  • Karl Sharman(Participant)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Participant, Always)
  • Pieter Kapsenberg(Reporter, Participant)