Opened 11 years ago

Closed 11 years ago

#1234 closed defect (fixed)

8.4.5.1 uses incorrect coordinate system assigning to xTb1 and yTb1

Reported by: jackh Owned by:
Priority: minor Milestone: RExt D6
Component: RExt text Version: RExt D5 (O1005) v3
Keywords: Cc: davidf, joel, teruhiko, jct-vc@…

Description

Following on from ticket #1208, I think the following section in 8.4.5.1:

  1. The variables xTb1 and yTb1 are derived as follows:

– The variable xTb1 is set equal to xTb0 + ( 1 << ( log2TrafoSize − 1 ) ).
– The variable yTb1 is set equal to yTb0 + ( 1 << ( log2TrafoSize − 1 ) ).

should say:

  1. The variables xTb1 and yTb1 are derived as follows:

– The variable xTb1 is set equal to xTb0 + ( ( 1 << ( log2TrafoSize ) ) / SubWidthC ).
– The variable yTb1 is set equal to yTb0 + ( ( 1 << ( log2TrafoSize ) ) / SubHeightC ).

as xTb1 and yTb1 are in chroma coordinates.

Change History (6)

comment:1 Changed 11 years ago by DefaultCC Plugin

  • Cc davidf joel teruhiko jct-vc@… added

comment:2 Changed 11 years ago by davidf

  • Milestone changed from RExt D5 to RExt D6
  • Resolution set to fixed
  • Status changed from new to closed

Fixed

comment:3 Changed 11 years ago by jackh

  • Resolution fixed deleted
  • Status changed from closed to reopened

Apologies, this correction was incomplete. The previous equations were only incorrect for chroma blocks, so the derivation should say:

  1. The variables xTb1 and yTb1 are derived as follows:

– The variable xTb1 is set equal to xTb0 + ( (cIdx>0) ? ( ( 1 << ( log2TrafoSize ) ) / SubWidthC ) : ( 1 << ( log2TrafoSize − 1 ) ) ).

– The variable yTb1 is set equal to yTb0 + ( (cIdx>0) ? ( ( 1 << ( log2TrafoSize ) ) / SubHeightC ) : ( 1 << ( log2TrafoSize − 1 ) ) ).

comment:4 Changed 11 years ago by davidf

I am not sure that this is correct. As i understand, the issue is for 4:2:2 where the non square tree is currently using incorrect vertical co-ordinates.

For 4:2:0, this should revert to the previous text.
For 4:4:4, the recursion needs to halve the transform size (as per the previous text).

The following seems to describe this behaviour:

The variables xTb1 and yTb1 are derived as follows:

– If either cIdx is equal to 0 or ChromaArrayType is not equal to 2, the following applies:

– The variable xTb1 is set equal to xTb0 + ( 1 << ( log2TrafoSize − 1 ) ).
– The variable yTb1 is set equal to yTb0 + ( 1 << ( log2TrafoSize − 1 ) ).

– Otherwise (ChromaArrayType is equal to 2 and cIdx is greater than 0), the following applies:

– The variable xTb1 is set equal to xTb0 + ( 1 << ( log2TrafoSize − 1 ) ).
– The variable yTb1 is set equal to yTb0 + ( 2 << ( log2TrafoSize − 1 ) ).

comment:5 Changed 11 years ago by jackh

I agree - the code I suggested won't work for 4:4:4. Yours is correct.

comment:6 Changed 11 years ago by davidf

  • Resolution set to fixed
  • Status changed from reopened to closed

Fixed in P1005_v4

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber, Participant)
  • Jack Haughton(Reporter, Participant)
  • jct-vc@…(Subscriber)
  • Joel(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Always)
  • Teruhiko Suzuki(Subscriber)