Opened 10 years ago Closed 10 years ago #76 closed defect (fixed)TComDataCU::getBaseColCU has signed/unsigned error
Description
The following bit of code in function TComDataCU::getBaseColCU does not handle negative values properly. Int iBX = (((uiPelX - leftStartL)*g_posScalingFactor[refLayerIdc][0] + (1<<15)) >> 16) + windowRL.getWindowLeftOffset(); Int iBY = (((uiPelY - topStartL )*g_posScalingFactor[refLayerIdc][1] + (1<<15)) >> 16) + windowRL.getWindowTopOffset();
The unsigned variables, uiPelX and uiPelY, need to be cast to signed variables as follows. Int iBX = ((((Int)uiPelX - leftStartL)*g_posScalingFactor[refLayerIdc][0] + (1<<15)) >> 16) + windowRL.getWindowLeftOffset(); Int iBY = ((((Int)uiPelY - topStartL )*g_posScalingFactor[refLayerIdc][1] + (1<<15)) >> 16) + windowRL.getWindowTopOffset(); Change history (2)comment:1 Changed 10 years ago by DefaultCC Plugin
comment:2 Changed 10 years ago by Vadim
Note: See TracTickets for help on using tickets. | This list contains all users that will be notified about changes made to this ticket. These roles will be notified: Reporter, Owner, Subscriber, Participant
|
fixed with rev 1074