Opened 10 years ago Closed 10 years ago #282 closed defect (invalid)Strange behaviour observed in the HM5.0 with quantization matrix
Description
When using the quantization matrix the initial quantization scale (denoted as uiQ) is modified to obtain the scaled quantization scale (denoted as uiQW) according the following formula:
uiQW = (uiQ<<4)/scaling_list[n].
where n denotes the coefficient position inside a given BxB TU. The shift by four bits (i.e. the multiplication by 16) is needed because the quantization matrix values are computed by multiplying their floating point values (picked by the HVS model that SONY is currently considering) by 16 and rounding them the nearest integer.
In this setting, the following issue has been observed when the quantization matrices are used (i.e. ScalingList=1). Suppose to provide the encoder with quantization matrices whereby all the values are equal to 16, from the previous formula it yields that uiQW = uiQ. In this case, the encoder that runs with these (special) quantization matrices must produce the same reconstructed sequence (i.e equal pixelbypixel) that would be produced when no quantization matrix is used. This happens to not be the case for the current HM5.0 release. Attached there are also the log files for the first 8 frames of race_horses sequence, QP = 32, ld_he. As may be noted the PSNR for both case is different. The rate is different too but this is expectable as the encoder that uses quantization matrices must transmit them and thus ends up with a higher bitrate. Attachments (1)Change History (5)comment:1 Changed 10 years ago by DefaultCC Plugin
Changed 10 years ago by matteoncomment:2 Changed 10 years ago by ksuehringcomment:3 Changed 10 years ago by matteon
Ok, I have done the test with ScalingList=2 and now it works! Thanks! comment:4 Changed 10 years ago by davidf
Note: See
TracTickets for help on using
tickets.  This list contains all users that will be notified about changes made to this ticket. These roles will be notified: Reporter, Owner, Subscriber, Participant

ScalingList=1 uses the stored default quantization matrices. ScalingList=2 uses the values provided in the parameter file. I think this needs better documentation.