Opened 11 years ago Closed 11 years ago #321 closed defect (fixed)mismatch between HM and WD on ALF chroma coeff prediction
Description
There are inconsistency between HM and WD on ALF chroma prediction.
HM's prediction properly works assuming that "sum of ALF coeff is equal to or near to 1.0", so WD should follows HM with the following fixs.
(1) fix1
(2) fix2
sum = alf_coeff_chroma[ AlfCodedLengthChroma – 2 ] + sigma_j( alf_coeff_chroma[ j ] << 1 ) (8 469)
with
sum = sigma_j ( alf_coeff_chroma[ j ] << 1 ) (8 469)
The corresponding function is Change History (4)comment:1 Changed 11 years ago by DefaultCC Plugin
comment:2 Changed 11 years ago by wjhan
comment:3 Changed 11 years ago by Tomohiro Ikai
(2) is related to H0483.
In my understanding,
HM5.0: val = pred  code (decoder)
HM6.0: val = pred + code (decoder)
Therefore
cC[ i ] = (256 – sum) – alf_coeff_chroma[ i ]
In WD6.0 (after JCTVCH0483 applied):
cC[ i ] = (256 – sum) + alf_coeff_chroma[ i ]
And luma part also have the same bug
cL[ i ][ j ] = alf_coeff_luma[ i ][ j ] + 255 – k(2*alf_coeff_luma[ i ][ k ]) – 2*cL[ i ][ j – 1 ] (8 465)
cL[ i ][ j ] = alf_coeff_luma[ i ][ j ] + 255 – ( k2*alf_coeff_luma[ i ][ k ] ) >> 2 (8 465) comment:4 Changed 11 years ago by wjhan
Ok, since H0483 is not integrated yet, I use HM5.0 method and fixe 255 bugs for both luma and chroma. When H0483 is integrated, first issue will be cleared according to the provided text. Note: See
TracTickets for help on using
tickets.  This list contains all users that will be notified about changes made to this ticket. These roles will be notified: Reporter, Owner, Subscriber, Participant

(2) is confirmed.
In the source code, (1) seems like:
cC[ i ] = 256 – ( sum – alf_coeff_chroma[ i ] ) (8 468)
Is it correct?