Opened 14 years ago

Closed 14 years ago

#28 closed defect (fixed)

Incompatibility, HHI_TRANSFORM_CODING=0, ROT=0

Reported by: wjhan Owned by: fbossen
Priority: minor Milestone:
Component: HM Version:
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

It seems large loss when HHI_TRANSFORM_CODING=0, ROT=0.

Maybe due to the wrong quantization scaling matrix for this case.

Attachments (1)

patch_Ticket28_TComTrQuant.patch​ (1.5 KB) - added by wjhan 14 years ago.
Patch file for Ticket #28, TComTrQuant.cpp

Download all attachments as: .zip

Change History (7)

Changed 14 years ago by wjhan

Patch file for Ticket #28, TComTrQuant.cpp

comment:1 Changed 14 years ago by wjhan

[NOTE: this patch can be tested after applying Ticket #30 patch (compilation error patch)]

In some cases of HHI_TRANSFORM_CODING=0, even when ROT=0, fwd ROT is applied but inv ROT is not applied, and vice versa. It is mainly due to the bug in HHI_ALLOW_ROT_SWITCH macro implementation. This patch solves the problem. Here is performance impact:

Before fix (HHI_TRANSFORM_CODING=0, ROT=0)
POC 0 ( I-SLICE, QP 32 ) 67192 bits [Y 34.4852 dB U 28.5087 dB V 27.8281 dB]

After fix (HHI_TRANSFORM_CODING=0, ROT=0)
POC 0 ( I-SLICE, QP 32 ) 77712 bits [Y 35.1053 dB U 38.0066 dB V 38.3377 dB]

You can see that 0.7dB gain in luma PSNR and 10dB gain in chroma PSNR.

I attach the patch for TComTrQuant.cpp file.

comment:2 Changed 14 years ago by fbossen

  • Owner set to fbossen
  • Status changed from new to accepted

comment:3 Changed 14 years ago by fbossen

  • Resolution set to fixed
  • Status changed from accepted to closed

Fixed in r131

comment:4 Changed 12 years ago by davidf

  • Component set to HM

Updating component after adding WD (Text) tickets

comment:5 Changed 12 years ago by davidf

  • Cc fbossen ksuehring davidf added

comment:6 Changed 12 years ago by davidf

  • Cc jct-vc@… added
Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber, Participant)
  • Frank Bossen(Owner, Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)
  • Woo-Jin Han(Reporter, Participant)