Opened 9 years ago

Closed 9 years ago

#565 closed defect (fixed)

Divide by zero error on streams 3 or less LCUs in height or width when ALF is enabled

Reported by: pieterkapsenberg Owned by:
Priority: minor Milestone: HM-7.1
Component: HM Version: HM-7.0
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

The calculation of the ALF regions involves a division by a variable called xInterval and yInterval in TComAdaptiveLoopFilter.cpp, line 752 and 756. This value evaluates to 0 when the picture width/height in LCUs is 3 or less, causing a divide-by-zero.

This is also an issue in the suggested text in I0157

Change History (5)

comment:1 Changed 9 years ago by DefaultCC Plugin

  • Cc fbossen ksuehring davidf jct-vc@… added

comment:2 Changed 9 years ago by chiayang_tsai

This is a duplicate ticket. The fix is in #543.

comment:3 Changed 9 years ago by yamakage

Since this ticket also reports a defect of the specification text, the following modification in section 8.7.3.3 of the specification text is suggested so that the text and ticket #543 become consistent:

=======
(Old: JCTVC-I0603_CDTexts_r3.doc)

xIdx = Min( 3, Floor( xC / ( (( PicWidthInCtbs + 1 ) >> 2) << Log2CtbSize ) ) ) (8 374)
yIdx = Min( 3, Floor( yC / ( (( PicHeightInCtbs + 1 ) >> 2) << Log2CtbSize ) ) ) (8 374)

(Modified)

xInterval = ( (( PicWidthInCtbs + 1 ) >> 2) << Log2CtbSize ) ) (8 374)
If xInterval is equal to 0, xIdx is set equal to 3.
Otherwise, xIdx is set equal to Min(3, Floor(xC / xInterval) ).
yInterval = ( (( PicHeightInCtbs + 1 ) >> 2) << Log2CtbSize ) ) (8 374)
If yInterval is equal to 0, yIdx is set equal to 3.
Otherwise, yIdx is set equal to Min(3, Floor(yC / yInterval) ).

=======

comment:4 Changed 9 years ago by bbross

The text is fixed with integration of JCTVC-I0603 ALF text in draft7 JCTVC-I1003_d5 (also in JCTVC-I0603_CDText by reviewer name "Ticket565").

comment:5 Changed 9 years ago by fbossen

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • Benjamin Bross(Participant)
  • Chia-Yang Tsai(Participant)
  • David Flynn(Subscriber)
  • Frank Bossen(Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)
  • Pieter Kapsenberg(Reporter)
  • Tomoo Yamakage(Participant)