Opened 9 years ago

Closed 9 years ago

#76 closed defect (fixed)

TComDataCU::getBaseColCU has signed/unsigned error

Reported by: johnnyVidyo Owned by: Vadim
Priority: critical Milestone: SHM-9.0
Component: SHM software Version:
Keywords: Cc: Vadim, jct-vc@…

Description

The following bit of code in function TComDataCU::getBaseColCU does not handle negative values properly.

  Int iBX = (((uiPelX - leftStartL)*g_posScalingFactor[refLayerIdc][0] + (1<<15)) >> 16) + windowRL.getWindowLeftOffset();
  Int iBY = (((uiPelY - topStartL )*g_posScalingFactor[refLayerIdc][1] + (1<<15)) >> 16) + windowRL.getWindowTopOffset();

The unsigned variables, uiPelX and uiPelY, need to be cast to signed variables as follows.

  Int iBX = ((((Int)uiPelX - leftStartL)*g_posScalingFactor[refLayerIdc][0] + (1<<15)) >> 16) + windowRL.getWindowLeftOffset();
  Int iBY = ((((Int)uiPelY - topStartL )*g_posScalingFactor[refLayerIdc][1] + (1<<15)) >> 16) + windowRL.getWindowTopOffset();

Change history (2)

comment:1 Changed 9 years ago by DefaultCC Plugin

  • Cc Vadim jct-vc@… added

comment:2 Changed 9 years ago by Vadim

  • Resolution set to fixed
  • Status changed from new to closed

fixed with rev 1074

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • Gerhard Tech(Always)
  • jct-vc@…(Subscriber)
  • johnnyVidyo(Reporter)
  • Karsten Suehring(Always)
  • Vadim Seregin(Owner, Subscriber, Participant)