Opened 12 years ago

Closed 10 years ago

#269 closed defect (wontfix)

Inconsistent signalling of merge flags triggers asserts

Reported by: stworrall Owned by:
Priority: minor Milestone:
Component: HM Version: HM-5.0
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…


When HM-5.0 is run using RAHE standard conditions, but with Merge turned off (e.g. -0 MRG), an assert is triggered in TComPrediction::xWeightedAverage. This is caused by iRefIdx values not being properly set.

Example command line:
-c encoder_randomaccess.cfg -i BasketballPass_416x240_50.yuv -wdt 416 -hgt 240 -fr 50 -fs 0 -f 500 -0 MRG --ReconFile=nomerge.yuv --IntraPeriod=48 --InputBitDepth=8 -q 22 -b basketpass_qp22_no_merge_rahe.bin

This issue seems to be caused by inconsistent signalling of whether Merge should be used for AMP or not.

In xCompressCU, AMP Merge is always checked, even if the Merge option is switched off. This becomes an issue in TEncSearch::predInterSearch, where the main merge flag is checked (pcCU->getSlice()->getSPS()->getUseMRG()), rather than the supplied function parameter bUseMRG. The result is that the iRefIdx's are not properly set.

There are two fixes for this problem. Either will prevent the assert being triggered. But it may be a good idea to apply both. Please see attached patch file for details.

Fix 1

Fix TEncCU.cpp, add some lines to xCompressCU to avoid AMP merge when the main merge flag is off. This means that the merge flag in the config file also switches AMP merge off.

Fix 2

Fix TEncSearch::predInterSearch so that it only uses the supplied function parameter bUseMRG.

Attachments (1)

merge_mode_mods.patch (1.1 KB) - added by stworrall 12 years ago.
Patch file containing suggested fixes

Download all attachments as: .zip

Change History (4)

comment:1 Changed 12 years ago by DefaultCC Plugin

  • Cc fbossen ksuehring davidf added

Changed 12 years ago by stworrall

Patch file containing suggested fixes

comment:2 Changed 12 years ago by davidf

  • Cc jct-vc@… added

comment:3 Changed 10 years ago by fbossen

  • Resolution set to wontfix
  • Status changed from new to closed

MRG parameter doesn't exist anymore

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber, Participant)
  • Frank Bossen(Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)
  • Stewart Worrall(Reporter)