TMuC supports a new context modeling for the transform coefficients and an adaptive scanning methods. But now they are controlled by a single macro, HHI_TRANSFORM_CODING, although they are different coding tools and can be separated easily.

We suggest to separate one macro into two macros: HHI_TRANSFORM_SCAN and HHI_TRANSFORM_CONTEXT_MODEL to improve the configurability of the software. Or, to have a backward compatibility, adding HHI_TRANSFORM_SCAN to turn on/off adaptive scanning inside the original MACRO is also ok.

According to the residual quadtree concept, for the given CU, various TU size is evaluated in the quadtree style. In the configuration file, two parameters are given:

QuadtreeTULog2MaxSize QuadtreeTULog2MinSize

where they specify the maximum and minimum possible transform sizes, respectively. It is a good feature, however, it can be improved further. For example, for 64x64 PU, all transform sizes of 4x4, 8x8, 16x16, 32x32 and 64x64 are evaluated when RQT=1. It slows encoder a lot.

We suggest to add one another way to configure residual quadtree: maximum allowed depth of residual quadtree such as:

QuadtreeTUMaxDepth

If we set QuadtreeTUMaxDepth=3, 64x64, 32x32 and 16x16 are used for 64x64 CU while 16x16, 8x8 and 4x4 are used for 16x16 CU. It is helpful to reduce the encoder complexity.

It should be noted that it does not change RQT itself, but just encoder behaviour. And if we set QuadtreeTUMaxDepth sufficiently large value, it should have the same results in the previous version.

This feature is also useful in TE9 to check variable aspects of the TU structure.

Issue seems related to HHI_TRANSFORM_CODING. See attached trace.