Opened 13 years ago Closed 11 years ago #269 closed defect (wontfix)Inconsistent signalling of merge flags triggers asserts
Description
When HM-5.0 is run using RAHE standard conditions, but with Merge turned off (e.g. -0 MRG), an assert is triggered in TComPrediction::xWeightedAverage. This is caused by iRefIdx values not being properly set.
Example command line:
This issue seems to be caused by inconsistent signalling of whether Merge should be used for AMP or not.
In xCompressCU, AMP Merge is always checked, even if the Merge option is switched off. This becomes an issue in TEncSearch::predInterSearch, where the main merge flag is checked (pcCU->getSlice()->getSPS()->getUseMRG()), rather than the supplied function parameter bUseMRG. The result is that the iRefIdx's are not properly set.
There are two fixes for this problem. Either will prevent the assert being triggered. But it may be a good idea to apply both. Please see attached patch file for details.
Fix 1
Fix TEncCU.cpp, add some lines to xCompressCU to avoid AMP merge when the main merge flag is off. This means that the merge flag in the config file also switches AMP merge off.
Fix 2
Fix TEncSearch::predInterSearch so that it only uses the supplied function parameter bUseMRG. Attachments (1)Change History (4)comment:1 Changed 13 years ago by DefaultCC Plugin
Changed 13 years ago by stworrallcomment:2 Changed 13 years ago by davidf
comment:3 Changed 11 years ago by fbossen
MRG parameter doesn't exist anymore Note: See
TracTickets for help on using
tickets. | This list contains all users that will be notified about changes made to this ticket. These roles will be notified: Reporter, Owner, Subscriber, Participant
|
Patch file containing suggested fixes