Opened 11 years ago Closed 10 years ago #339 closed defect (fixed)Out of bounds index computation in Angular Intra prediction
Description
In the WD, subclause 8.4.3.1.6 "Specification of Intra_Angular (2..9, 11..25, 27..34) prediction mode" the following equation is shown (848):
For the case of an intra 4x4 block with an intra mode of 25, intraPredAngle is 2 and invAngle is 4096 (table 85 and 86). Then, the index range goes from (4*2)>>1 .. 1, which is 1..1. This implies that the equation is applied once, with x = 1. In that case we index array p with [1,1+((1*4096+128)>>8)] = [1,15]. However, array P is specified to be sized from [1..2*ns1,1..2*ns1], [1..7,1..7] in this case.
HM seems to avoid this by just skipping the loop altogether, since it has the following code: Int invAngleSum = 128; // rounding for (shift by 8) for (k=1; k>blkSize*intraPredAngle>>5; k) { invAngleSum += invAngle; refMain[k] = refSide[invAngleSum>>8]; }
In this case, the exit condition is met immediately, since k is not larger than 1 (it is equal to 1). If the HM model followed the WD exactly, the loop exit condition would have been a greatorequal operator like so: for (k=1; k>=blkSize*intraPredAngle>>5; k) Change History (3)comment:1 Changed 11 years ago by DefaultCC Plugin
comment:2 Changed 10 years ago by bbrosscomment:3 Changed 10 years ago by bbross
Fixed in JCTVCI1003_d9. Note: See
TracTickets for help on using
tickets.  This list contains all users that will be notified about changes made to this ticket. These roles will be notified: Reporter, Owner, Subscriber, Participant

Adding the following condition between "– If intraPredAngle is less than 0," and "refMain[ x ] = ..." for both equations would avoid the invalid indices and match draft text with HM software.