Opened 11 years ago

Closed 10 years ago

#1252 closed defect (fixed)

Mismatched coordinates in transform_unit()

Reported by: jackh Owned by:
Priority: minor Milestone: RExt D7
Component: RExt text Version: RExt D6 (P1005) v1
Keywords: Cc: davidf, joel, teruhiko, jct-vc@…

Description

In O1005, the following passage appears in transform_unit:

else if( blkIdx = = 3 ) {
   for( tIdx = 0; tIdx < ( ChromaArrayType = = 2 ? 2 : 1 ); tIdx++ )
      if( cbf_cb[ xBase ][ yBase ][ trafoDepth ] )
         residual_coding( xBase, yBase + ( tIdx << log2TrafoSize ), log2TrafoSize, 1 )
   for( tIdx = 0; tIdx < ( ChromaArrayType = = 2 ? 2 : 1 ); tIdx++ )
      if( cbf_cr[ xBase ][ yBase ][ trafoDepth ] )
         residual_coding( xBase, yBase + ( tIdx << log2TrafoSize ), log2TrafoSize, 2 )
}

In P1005, this has been changed to:

else if( blkIdx = = 3 ) {
   for( tIdx = 0; tIdx < ( ChromaArrayType = = 2 ? 2 : 1 ); tIdx++ )
      if( cbf_cb[ xBase ][ yBase + ( tIdx << log2TrafoSizeC ) ][ trafoDepth ] )
         residual_coding( xBase, yBase + ( tIdx << log2TrafoSize ), log2TrafoSize, 1 )
   for( tIdx = 0; tIdx < ( ChromaArrayType = = 2 ? 2 : 1 ); tIdx++ )
      if( cbf_cr[ xBase ][ yBase + ( tIdx << log2TrafoSizeC ) ][ trafoDepth ] )
         residual_coding( xBase, yBase + ( tIdx << log2TrafoSize ), log2TrafoSize, 2 )
}

i.e. the 4:2:2 y offsets in the cbf_xx y indices have been changed to chroma coordinates rather than luma coordinates. As this is being done in the block which executes on a 4x4 luma block, log2TrafoSizeC=1. So for instance, in a 4x4 luma block with its origin at 4,4, this would end up accessing cbf_cb[0][2] instead of cbf_cb[0][4]. As this location will never have been filled in, this seems broken - I think the P1005 version was correct.

Change History (5)

comment:1 Changed 11 years ago by DefaultCC Plugin

  • Cc davidf joel teruhiko jct-vc@… added

comment:2 Changed 11 years ago by jackh

Sorry - that last sentence should say that I think the O1005 version was correct!

comment:3 Changed 11 years ago by jackh

Further to this, I've now done some more in-depth debugging and discovered that the problem actually lies in the lack of cbf_cb and cbf_cr defaults in the spec for the second half of 4:2:2 chroma sub-TUs, which presumably should get copied from the layer above in the same way that the first half ones do.

However, that still leaves these flags spaced 2 locations apart instead of 4, which seems a bit counter-intuitive as the blocks are spaced 4 apart. Perhaps a better solution might be to read the flags directly from the layer above in the log2TrafoSize==2 case?

comment:4 Changed 11 years ago by davidf

  • Milestone changed from RExt D6 to RExt D7

comment:5 Changed 10 years ago by davidf

  • Resolution set to fixed
  • Status changed from new to closed

This is related to a more general cbf issue. I believe this has been fixed in Q1005.

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber, Participant)
  • Jack Haughton(Reporter, Participant)
  • jct-vc@…(Subscriber)
  • Joel(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Always)
  • Teruhiko Suzuki(Subscriber)