According to the residual quadtree concept, for the given CU, various TU size is evaluated in the quadtree style. In the configuration file, two parameters are given:

QuadtreeTULog2MaxSize
QuadtreeTULog2MinSize

where they specify the maximum and minimum possible transform sizes, respectively. It is a good feature, however, it can be improved further. For example, for 64x64 PU, all transform sizes of 4x4, 8x8, 16x16, 32x32 and 64x64 are evaluated when RQT=1. It slows encoder a lot.

We suggest to add one another way to configure residual quadtree: maximum allowed depth of residual quadtree such as:

QuadtreeTUMaxDepth

If we set QuadtreeTUMaxDepth=3, 64x64, 32x32 and 16x16 are used for 64x64 CU while 16x16, 8x8 and 4x4 are used for 16x16 CU. It is helpful to reduce the encoder complexity.

It should be noted that it does not change RQT itself, but just encoder behaviour. And if we set QuadtreeTUMaxDepth sufficiently large value, it should have the same results in the previous version.

This feature is also useful in TE9 to check variable aspects of the TU structure.