Opened 13 years ago Closed 11 years ago #419 closed defect (fixed)transformSubdivFlag signaled when Log2CbSize=6 and Log2MaxTrafoSize=4 and implicit split
Description
In HM-6.0, when the transform tree of a CU with 64x64 luma coding block is processed and (not common conditions)
in Void TDecEntropy::xDecodeTransform, it occurs that for non-2Nx2N inter partitions, a transform block subdivision flag is signaled. Furthermore in HE10 configuration a encoder/decoder mismatch occurs.
This is reproducible with the following encoder calls: 1.a ../build/Release/TAppEncoder -c encoder_lowdelay_main.cfg -c per-sequence/BQSquare.cfg -b BQSquare_LD_main-2.bin -q 22 -f 8 --QuadtreeTULog2MaxSize=4 --QuadtreeTUMaxDepthInter=1 1.b ../build/Release/TAppEncoder -c ./encoder_lowdelay_main.cfg -c ./per-sequence/NebutaFestival_10bit.cfg -b str_main.bin -q 22 -f 8 --QuadtreeTULog2MaxSize=4 --MaxPartitionDepth=1 --QuadtreeTUMaxDepthInter=1 --QuadtreeTUMaxDepthIntra=1
With mismatch (he10): 2.a ../build/Release/TAppEncoder -c encoder_lowdelay_he10.cfg -c per-sequence/BQSquare.cfg -b BQSquare_LD_he10-2.bin -q 22 -f 8 --QuadtreeTULog2MaxSize=4 --QuadtreeTUMaxDepthInter=1 2.b ../build/Release/TAppEncoder -c ./encoder_lowdelay_he10.cfg -c ./per-sequence/NebutaFestival_10bit.cfg -b str_he10.bin -q 22 -f 10 --QuadtreeTULog2MaxSize=4 --MaxPartitionDepth=1 --QuadtreeTUMaxDepthInter=1 --QuadtreeTUMaxDepthIntra=1
and decoding the corresponding bitstreams with a breakpoint in l.633 in TDecEntropy.cpp m_pcEntropyDecoderIf->parseTransformSubdivFlag( uiSubdiv, uiDepth );
The corresponding bitstreams are attached.
This can easily be fixed by removing '- SplitFlag' in l.2055 in TComDataCU::getQuadtreeTULog2MinSizeInCU( UInt uiIdx ): uiLog2MinTUSizeInCU = m_pcSlice->getSPS()->getQuadtreeTULog2MaxSize();
After the fix is applied, the subdivision flag is not signaled anymore and the mismatch is gone and in config
After I decrypted what the function TComDataCU::getQuadtreeTULog2MinSizeInCU( UInt uiIdx ) is doing, especially w.r.t to draft text, I took the chance to clean the function up. The cleanup patch (bit exact) and the cleanup patch including the fix are attached. The fixed code matches with draft text.
Another observation in the code is that TComDataCU::getQuadtreeTULog2MinSizeInCU( UInt uiIdx ) only depends on the CU address AbsPartIdx/uiIdx and does not change during recursive call of TDecEntropy::xDecodeTransform/TDecEntropy::xDecodeTransformSubdiv. This function can be taken out of the recursion and passed, e.g. as a parameter uiLog2MinTUSizeInCU so that multiple calls of the function leading to the same result can be avoided. Attachments (4)Change History (7)comment:1 Changed 13 years ago by DefaultCC Plugin
Changed 13 years ago by bbrosscomment:2 follow-up: ↓ 3 Changed 13 years ago by ksuehringcomment:3 in reply to: ↑ 2 Changed 11 years ago by fbossen
Note: See
TracTickets for help on using
tickets. | This list contains all users that will be notified about changes made to this ticket. These roles will be notified: Reporter, Owner, Subscriber, Participant
|
1.a bitstream