Opened 10 years ago

Closed 9 years ago

#1345 closed defect (fixed)

AdaptiveQp activity calculation error

Reported by: rikallen Owned by:
Priority: minor Milestone:
Component: HM Version: HM-16.0
Keywords: Cc: ksuehring, davidf, karlsharman, jct-vc@…


In TEncPreanalyzer::xPreanalyze(...) (TEncPreanalyzer.cpp)

The sum of luma sample values and squared values is accumulated for each quadrant of a 2Nx2N cu - so each accumulates over NxN values, but the average is then calculated by dividing through by "uiNumPixInAQPart" which counts values over the whole 2Nx2N.

i.e. - for a 16x16 Cu, the 4 sums are accumulated over 64 values each, but the average value is calculated by dividing by 256. This gives an incorrect value for the variance of the blocks.

Possible fix -

Delete ", uiNumPixInAQPart++" in each of the 4 inner loops and insert at line 121

uiNumPixInAQPart = uiCurrAQPartWidth*uiCurrAQPartHeight/4;

Change History (2)

comment:1 Changed 10 years ago by DefaultCC Plugin

  • Cc ksuehring davidf karlsharman jct-vc@… added

comment:2 Changed 9 years ago by karlsharman

  • Resolution set to fixed
  • Status changed from new to closed

Fixed in r4591.

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber)
  • jct-vc@…(Subscriber)
  • Karl Sharman(Subscriber, Participant)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)
  • Rik Allen(Reporter)