Opened 13 years ago

Closed 13 years ago

#196 closed defect (fixed)

Assertion failure in TEncAdaptiveLoopFilter::gnsSolveByChol()

Reported by: hao Owned by:
Priority: blocker Milestone: HM-4.0
Component: HM Version: HM-3.3
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

HM encoders fail to encode images whose pixel values are uniform or near-uniform.
HM-3.2 or lower hangs up and HM-3.3 or higher stops with an assertion failure.

An example command line for replicating the bug is:

TAppEncoder.exe -c cfg/encoder_intra.cfg --InputFile=gray.yuv --SourceWidth=1280 --SourceHeight=720 --FrameRate=1 --FrameToBeEncoded=1 -q 37

where gray.yuv is the uniform byte array of 128.

The error message from HM-3.4-dev r1246) is:

Assertion failed: singular == 1, file ..\..\source\Lib\TLibEncoder\TEncAdaptiveLoopFilter.cpp, line 4287

Change History (6)

comment:1 Changed 13 years ago by hao

  • Priority changed from minor to major

comment:2 Changed 13 years ago by fbossen

  • Milestone set to HM-4.0
  • Priority changed from major to blocker

comment:3 Changed 13 years ago by fbossen

  • Resolution set to fixed
  • Status changed from new to closed

Fixed in r1320

comment:4 Changed 13 years ago by davidf

  • Component set to HM

Updating component after adding WD (Text) tickets

comment:5 Changed 13 years ago by davidf

  • Cc fbossen ksuehring davidf added

comment:6 Changed 13 years ago by davidf

  • Cc jct-vc@… added
Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber, Participant)
  • Frank Bossen(Subscriber, Participant)
  • Hirofumi Aoki(Reporter, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)