Opened 12 years ago

Closed 12 years ago

#934 closed defect (fixed)

g_auiRasterToZscan Address Calculation out of range in TComPattern::initPattern

Reported by: yul Owned by:
Priority: minor Milestone: HM-9.2
Component: HM Version: HM-9.1
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

In TComPattern::initPattern function,

if( uiCurrPicPelY != 0 )
{

UInt uiNumPartInWidth = ( uiWidth/pcPic->getMinCUWidth() );
uiOffsetAbove = 1;


if( uiCurrPicPelX + uiWidth < pcCU->getSlice()->getSPS()->getPicWidthInLumaSamples() )
{

if( ( g_auiZscanToRaster[uiAbsZorderIdx] + uiNumPartInWidth ) % pcPic->getNumPartInWidth() ) Not CU boundary
{

if( g_auiRasterToZscan[ (Int)g_auiZscanToRaster[uiAbsZorderIdx] - (Int)pcPic->getNumPartInWidth() + (Int)uiNumPartInWidth ] < uiAbsZorderIdx )
{

uiOffsetRight = 1;

}

}
...

The g_auiRasterToZscan[ (Int)g_auiZscanToRaster[uiAbsZorderIdx] - (Int)pcPic->getNumPartInWidth() + (Int)uiNumPartInWidth ] may have out of range problem, due to the calculation of (Int)g_auiZscanToRaster[uiAbsZorderIdx] - (Int)pcPic->getNumPartInWidth() + (Int)uiNumPartInWidth may be < 0. This will cause the visiting address of g_auiRasterToZscan[] is out of range and random value is obtained. It should be changed as follows:

if( uiCurrPicPelY != 0 )
{

UInt uiNumPartInWidth = ( uiWidth/pcPic->getMinCUWidth() );
uiOffsetAbove = 1;


if( uiCurrPicPelX + uiWidth < pcCU->getSlice()->getSPS()->getPicWidthInLumaSamples() )
{

if( ( g_auiZscanToRaster[uiAbsZorderIdx] + uiNumPartInWidth ) % pcPic->getNumPartInWidth() ) Not CU boundary
{

if ((Int)g_auiZscanToRaster[uiAbsZorderIdx] - (Int)pcPic->getNumPartInWidth() + (Int)uiNumPartInWidth >= 0 )
{

if( g_auiRasterToZscan[ (Int)g_auiZscanToRaster[uiAbsZorderIdx] - (Int)pcPic->getNumPartInWidth() + (Int)uiNumPartInWidth ] < uiAbsZorderIdx )
{

uiOffsetRight = 1;

}

}

}
...

Change History (2)

comment:1 Changed 12 years ago by DefaultCC Plugin

  • Cc fbossen ksuehring davidf jct-vc@… added

comment:2 Changed 12 years ago by fbossen

  • Milestone set to HM-9.2
  • Resolution set to fixed
  • Status changed from new to closed

Fixed in r3167
Removed unused variable instead of fixing its computation

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber)
  • Frank Bossen(Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)