Opened 13 years ago

Closed 13 years ago

#1072 closed defect (fixed)

numDU = 0 causes divide by zero, run time error

Reported by: gregcoppa Owned by:
Priority: minor Milestone: HM-11.0
Component: HM Version: HM-10.0
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

In TEncGOP::compressGOP(...) numDU is initialized to zero for SliceMode != 1:

UInt numDU = ( m_pcCfg->getSliceMode() == 1 ) ? ( pcPic->getNumCUsInFrame() / maxCU ) : ( 0 );

this leads to a divide by zero when SliceMode == 0 in the following call:

pcSlice->getSPS()->setHrdParameters( m_pcCfg->getFrameRate(), numDU, m_pcCfg->getTargetBitrate(), ( m_pcCfg->getIntraPeriod() > 0 ) );

...

ducpbSizeValue = bitRate/numDU;

Not sure how to best prevent this - greg

Change History (3)

comment:1 Changed 13 years ago by DefaultCC Plugin

  • Cc fbossen ksuehring davidf jct-vc@… added

comment:2 Changed 13 years ago by gregcoppa

  • Summary changed from HM to numDU = 0 causes divide by zero, run time error

comment:3 Changed 13 years ago by fbossen

  • Milestone changed from HM-10.1 to HM-11.0
  • Resolution set to fixed
  • Status changed from new to closed

Fixed in r3478

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber)
  • Frank Bossen(Subscriber, Participant)
  • gregc(Reporter, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)