Opened 13 years ago

Closed 13 years ago

# AMVRES 1/8th 12-tap DCT-IF bug (QC_AMVRES_LOW_COMPLEXTY=0 case)

Reported by: Owned by: wjhan minor HM fbossen, ksuehring, davidf, jct-vc@…

### Description

[AMVRES=1 (QC_AMVRES macro = 1), QC_AMVRES_LOW_COMPLEXTY macro = 0]

5/8th position of 12-tap DCT-IF filter is strange: (it should be symmetric values to 3/8th position)

Here is the changes:

Current 3/8:

{ -3, 9, -15, 27, -51, 200, 119, -44, 24, -14, 7, -3 }, 3/8

Current 5/8:

{ -1, 6, -12, 20, -40, 119, 195, -44, 22, -14, 6, -1 }, 5/8

Fixed 5/8: (symmetric to 3/8)

{ -3, 7, -14, 24, -44, 119, 200, -51, 27, -15, 9, -3 }, 5/8

Fortunately, default value of QC_AMVRES_LOW_COMPLEXTY = 1, so no performance change in default setting with this change.

### Changed 13 years ago by wjhan

Patch file for #64

### comment:1 follow-up: ↓ 2 Changed 13 years ago by fbossen

Which one should actually be corrected? The 3/8 looks a bit funny since the first and last values are -3 whereas these are -1 for all other fractional positions.

### comment:2 in reply to: ↑ 1 Changed 13 years ago by wjhan

Which one should actually be corrected? The 3/8 looks a bit funny since the first and last values are -3 whereas these are -1 for all other fractional positions.

Patch is correct one. Here is simply rounded version of DCT-IF using the equation in A124.

FracMV = 1/ 8

{ -1, 4, -7, 13, -26, 249, 35, -16, 9, -6, 3, -1,},

FracMV = 2/ 8

{ -2, 7, -13, 22, -44, 230, 76, -31, 18, -11, 6, -2,},

FracMV = 3/ 8

{ -3, 8, -16, 27, -52, 201, 121, -44, 24, -15, 8, -3,},

FracMV = 4/ 8

{ -3, 9, -16, 28, -52, 162, 162, -52, 28, -16, 9, -3,},

FracMV = 5/ 8

{ -3, 8, -15, 24, -44, 121, 201, -52, 27, -16, 8, -3,},

FracMV = 6/ 8

{ -2, 6, -11, 18, -31, 76, 230, -44, 22, -13, 7, -2,},

FracMV = 7/ 8

{ -1, 3, -6, 9, -16, 35, 249, -26, 13, -7, 4, -1,},

And the code is:

{ -1, 4, -7, 13, -27, 249, 36, -16, 9, -6, 3, -1 }, 1/8
{ -1, 5, -12, 20, -40, 229, 76, -32, 16, -8, 4, -1 },
Quarter0
{ -3, 9, -15, 27, -51, 200, 119, -44, 24, -14, 7, -3 }, 3/8
{ -1, 8, -16, 24, -48, 161, 161, -48, 24, -16, 8, -1 },
Half

#if BUGFIX64

{ -3, 7, -14, 24, -44, 119, 200, -51, 27, -15, 9, -3 }, 5/8

#else

{ -1, 6, -12, 20, -40, 119, 195, -44, 22, -14, 6, -1 }, 5/8

#endif

{ -1, 4, -8, 16, -32, 76, 229, -40, 20, -12, 5, -1 }, Quarter1
{ -1, 3, -6, 9, -16, 36, 249, -27, 13, -7, 4, -1 }
7/8

So first and last values of half, 3/8 and 5/8 should be -3 and those of quarter should be -2, but the current coefficients of half and quarter were designed to remove multiplication operation easily with small loss of coding efficiency. (already in CfP submission, A124)

Small differences of other positions between generated one and one in the source code are due to filter normalization.

### comment:3 Changed 13 years ago by fbossen

• Resolution set to fixed
• Status changed from new to closed

resolved in r178

### comment:4 Changed 12 years ago by davidf

• Component set to HM

Updating component after adding WD (Text) tickets

### comment:5 Changed 12 years ago by davidf

• Cc fbossen ksuehring davidf added

### comment:6 Changed 12 years ago by davidf

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

• David Flynn(Subscriber, Participant)
• Frank Bossen(Subscriber, Participant)
• jct-vc@…(Subscriber)
• karl.sharman@…(Always)
• Karsten Suehring(Subscriber, Always)
• Woo-Jin Han(Reporter, Participant)