Opened 12 years ago

Closed 12 years ago

#339 closed defect (fixed)

Out of bounds index computation in Angular Intra prediction

Reported by: pieterkapsenberg Owned by: bbross
Priority: minor Milestone: WD6
Component: Text Version: D6 (H1003) dI/dJ/dK
Keywords: intra angular index Cc: bbross, wjhan, jct-vc@…


In the WD, subclause "Specification of Intra_Angular (2..9, 11..25, 27..34) prediction mode" the following equation is shown (8-48):
refMain[ x ] = p[ −1, −1+( ( x*invAngle+128 )>>8 ) ], with x=( nS*intraPredAngle ) >>5..−1

For the case of an intra 4x4 block with an intra mode of 25, intraPredAngle is -2 and invAngle is -4096 (table 8-5 and 8-6). Then, the index range goes from (4*-2)>>1 .. -1, which is -1..-1. This implies that the equation is applied once, with x = -1. In that case we index array p with [-1,-1+((-1*-4096+128)>>8)] = [-1,15]. However, array P is specified to be sized from [-1..2*ns-1,-1..2*ns-1], [-1..7,-1..7] in this case.

HM seems to avoid this by just skipping the loop altogether, since it has the following code:

      Int invAngleSum    = 128;       // rounding for (shift by 8)
      for (k=-1; k>blkSize*intraPredAngle>>5; k--)
        invAngleSum += invAngle;
        refMain[k] = refSide[invAngleSum>>8];

In this case, the exit condition is met immediately, since k is not larger than -1 (it is equal to -1). If the HM model followed the WD exactly, the loop exit condition would have been a great-or-equal operator like so:

      for (k=-1; k>=blkSize*intraPredAngle>>5; k--)

Change History (3)

comment:1 Changed 12 years ago by DefaultCC Plugin

  • Cc bbross wjhan jct-vc@… added

comment:2 Changed 12 years ago by bbross

Adding the following condition between "– If intraPredAngle is less than 0," and "refMain[ x ] = ..." for both equations would avoid the invalid indices and match draft text with HM software.

  • When ( nS*intraPredAngle ) >>5 is less than −1,

comment:3 Changed 12 years ago by bbross

  • Resolution set to fixed
  • Status changed from new to closed

Fixed in JCTVC-I1003_d9.

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • Benjamin Bross(Owner, Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Always)
  • Pieter Kapsenberg(Reporter)
  • Woo-Jin Han(Subscriber)