Opened 14 years ago

Closed 14 years ago

#64 closed defect (fixed)

AMVRES 1/8th 12-tap DCT-IF bug (QC_AMVRES_LOW_COMPLEXTY=0 case)

Reported by: wjhan Owned by:
Priority: minor Milestone:
Component: HM Version:
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

[AMVRES=1 (QC_AMVRES macro = 1), QC_AMVRES_LOW_COMPLEXTY macro = 0]

5/8th position of 12-tap DCT-IF filter is strange: (it should be symmetric values to 3/8th position)

Here is the changes:

Current 3/8:

{ -3, 9, -15, 27, -51, 200, 119, -44, 24, -14, 7, -3 }, 3/8

Current 5/8:

{ -1, 6, -12, 20, -40, 119, 195, -44, 22, -14, 6, -1 }, 5/8

Fixed 5/8: (symmetric to 3/8)

{ -3, 7, -14, 24, -44, 119, 200, -51, 27, -15, 9, -3 }, 5/8

Fortunately, default value of QC_AMVRES_LOW_COMPLEXTY = 1, so no performance change in default setting with this change.

Attachments (1)

patch_Ticket64.patch​ (397 bytes) - added by wjhan 14 years ago.
Patch file for #64

Download all attachments as: .zip

Change History (7)

Changed 14 years ago by wjhan

Patch file for #64

comment:1 follow-up: Changed 14 years ago by fbossen

Which one should actually be corrected? The 3/8 looks a bit funny since the first and last values are -3 whereas these are -1 for all other fractional positions.

comment:2 in reply to: ↑ 1 Changed 14 years ago by wjhan

Replying to fbossen:

Which one should actually be corrected? The 3/8 looks a bit funny since the first and last values are -3 whereas these are -1 for all other fractional positions.

Patch is correct one. Here is simply rounded version of DCT-IF using the equation in A124.

FracMV = 1/ 8

{ -1, 4, -7, 13, -26, 249, 35, -16, 9, -6, 3, -1,},

FracMV = 2/ 8

{ -2, 7, -13, 22, -44, 230, 76, -31, 18, -11, 6, -2,},

FracMV = 3/ 8

{ -3, 8, -16, 27, -52, 201, 121, -44, 24, -15, 8, -3,},

FracMV = 4/ 8

{ -3, 9, -16, 28, -52, 162, 162, -52, 28, -16, 9, -3,},

FracMV = 5/ 8

{ -3, 8, -15, 24, -44, 121, 201, -52, 27, -16, 8, -3,},

FracMV = 6/ 8

{ -2, 6, -11, 18, -31, 76, 230, -44, 22, -13, 7, -2,},

FracMV = 7/ 8

{ -1, 3, -6, 9, -16, 35, 249, -26, 13, -7, 4, -1,},

And the code is:

{ -1, 4, -7, 13, -27, 249, 36, -16, 9, -6, 3, -1 }, 1/8
{ -1, 5, -12, 20, -40, 229, 76, -32, 16, -8, 4, -1 }, Quarter0
{ -3, 9, -15, 27, -51, 200, 119, -44, 24, -14, 7, -3 }, 3/8
{ -1, 8, -16, 24, -48, 161, 161, -48, 24, -16, 8, -1 }, Half

#if BUGFIX64

{ -3, 7, -14, 24, -44, 119, 200, -51, 27, -15, 9, -3 }, 5/8

#else

{ -1, 6, -12, 20, -40, 119, 195, -44, 22, -14, 6, -1 }, 5/8

#endif

{ -1, 4, -8, 16, -32, 76, 229, -40, 20, -12, 5, -1 }, Quarter1
{ -1, 3, -6, 9, -16, 36, 249, -27, 13, -7, 4, -1 } 7/8

So first and last values of half, 3/8 and 5/8 should be -3 and those of quarter should be -2, but the current coefficients of half and quarter were designed to remove multiplication operation easily with small loss of coding efficiency. (already in CfP submission, A124)

Small differences of other positions between generated one and one in the source code are due to filter normalization.

comment:3 Changed 14 years ago by fbossen

  • Resolution set to fixed
  • Status changed from new to closed

resolved in r178

comment:4 Changed 13 years ago by davidf

  • Component set to HM

Updating component after adding WD (Text) tickets

comment:5 Changed 13 years ago by davidf

  • Cc fbossen ksuehring davidf added

comment:6 Changed 13 years ago by davidf

  • Cc jct-vc@… added
Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber, Participant)
  • Frank Bossen(Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)
  • Woo-Jin Han(Reporter, Participant)