Opened 12 years ago

Closed 10 years ago

#502 closed defect (fixed)

HM Encoder crash with divide by 0 error in deblocking for min transform size = 32

Reported by: kumarsanjeev123 Owned by:
Priority: minor Milestone:
Component: HM Version: HM-6.0
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

if ( (iEdge % ( (DEBLOCK_SMALLEST_BLOCK<<1)/uiPelsInPart ) ) == 0 )

Above line in TComLoopFilter.cpp has divide by 0 error for following settings in config file

MaxCUWidth : 64 # Maximum coding unit width in pixel
MaxCUHeight : 64 # Maximum coding unit height in pixel
MaxPartitionDepth : 1 # Maximum coding unit depth
QuadtreeTULog2MaxSize : 5 # Log2 of maximum transform size for

# quadtree-based TU coding (2...6)

QuadtreeTULog2MinSize : 5 # Log2 of minimum transform size for

# quadtree-based TU coding (2...6)

QuadtreeTUMaxDepthInter : 1
QuadtreeTUMaxDepthIntra : 1

Change History (2)

comment:1 Changed 12 years ago by DefaultCC Plugin

  • Cc fbossen ksuehring davidf jct-vc@… added

comment:2 Changed 10 years ago by fbossen

  • Resolution set to fixed
  • Status changed from new to closed

Could not reproduce issue with recent HM version

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber)
  • Frank Bossen(Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)