Opened 13 years ago

Closed 10 years ago

#255 closed defect (fixed)

Potential Bug in HM (both in 4.0 and 5.0)

Reported by: NaZhang Owned by:
Priority: minor Milestone: HM-16.3
Component: HM Version: HM-5.0
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

When we did expeiments on HM4.0, we observed that the following section of the code does not seem to behave in the correct way.In the function "encodeResAndCalcRdInterCU" in the file "TEncSearch.cpp".

In the section "Residual coding", after calculating the best RD cost.

if ( pcCU->isSkipped(0) )

{

pcCU->setCbfSubParts( 0, 0, 0, 0, pcCU->getDepth( 0 ) );

}


To my understanding, this is similar to the case when the value of bSkipRes is true. Therefore, it seems that the above code should be changed to
if ( pcCU->isSkipped(0) )

{

pcCU->setCbfSubParts( 0, 0, 0, 0, pcCU->getDepth( 0 ) );
pcCU->setTrIdxSubParts( 0, 0, pcCU->getDepth(0) );

}


Change History (5)

comment:1 Changed 13 years ago by DefaultCC Plugin

  • Cc fbossen ksuehring davidf added

comment:2 Changed 13 years ago by ksuehring

  • Version set to HM-5.0

comment:3 Changed 13 years ago by davidf

  • Cc jct-vc@… added

comment:4 Changed 11 years ago by fbossen

In r3772, line 4551 and following in TEncSearch.cpp

comment:5 Changed 10 years ago by karlsharman

  • Milestone set to HM-16.3
  • Resolution set to fixed
  • Status changed from new to closed

As identified, the transformIdx data structure can contain undefined values for a 2Nx2N skipped block (when all CBFs will be 0).

Fortunately, since the CBFs are all 0, the other places in the code that transformIdx are used are not affected by the undefined values (although the loop filter may test unnecessary TU edges before discovering that they belong to the same PU).

However, to avoid future confusion, this has been tidied in r4242.

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber, Participant)
  • Frank Bossen(Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • Karl Sharman(Participant)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Participant, Always)
  • NaZhang(Reporter)