Opened 14 years ago

Closed 14 years ago

#151 closed defect (fixed)

Decoding mismatch/crash when RDOQ off

Reported by: Xiang Li Owned by:
Priority: major Milestone:
Component: HM Version: HM-3.0
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

Decoding mismatch/crash was observed when RDOQ off (HM3.0). For example for BasketballPass_416x240_50.yuv, QP=32, 9 frames, decoding mismatch occurs. The encoding and decoding logs are as follows.

Encoding:
HM software: Encoder Version [3.0][Windows][VS 1500][32 bit]

Input File : E:\Sequences\JCTVC_Seqs\D01_BasketballPass_416x240_50.yuv
Bitstream File : str\HM2.0_IntraLoCo_D01_BasketballPass_416x240_QP32.bin
Reconstruction File : yuv\HM2.0_IntraLoCo_D01_BasketballPass_416x240_QP32.rec
Real Format : 416x240 50Hz
Internal Format : 416x240 50Hz
Frame index : 0 - 8 (9 frames)
Number of Ref. frames (P) : 4
Number of Ref. frames (B_L0) : 2
Number of Ref. frames (B_L1) : 2
Number of Reference frames : 4
CU size / depth : 64 / 4
RQT trans. size (min / max) : 4 / 32
Max RQT depth inter : 3
Max RQT depth intra : 3
Motion search range : 64
Intra period : -1
Decoding refresh type : 0
QP : 32.00
GOP size : 1
Rate GOP size : 4
Internal bit depth : 8
Entropy coder : VLC

TOOL CFG: ALF:0 IBD:0 HAD:1 SRD:0 RDQ:0 SQP:0 ASR:0 PAD:0 LDC:1 NRF:0 BQP:1 GPB:1 LComb:1 LCMod:0 FEN:1 RQT:1 MRG:1 LMC:1 Slice:0 EntropySlice:0 CIP:0 SAO:1

POC 0 ( I-SLICE, QP 32 ) 39776 bits [Y 35.7291 dB U 39.5039 dB V 39.3209 dB] [ET 1 ] [L0 ] [L1 ] [MD5:d9a6ef5e818feaa199a227af58784ca6]
POC 1 ( B-SLICE, QP 35 ) 1520 bits [Y 34.3308 dB U 39.2839 dB V 38.5440 dB] [ET 1 ] [L0 0 ] [L1 0 ] [LC 0 ] [MD5:b3df8c513e3209a0b14307c9d9ea09b1]
POC 2 ( B-SLICE, QP 34 ) 2712 bits [Y 34.1265 dB U 39.1618 dB V 38.3636 dB] [ET 2 ] [L0 1 0 ] [L1 1 0 ] [LC 1 0 ] [MD5:c880ad3d1cb40000b4082b53f250265a]
POC 3 ( B-SLICE, QP 35 ) 2144 bits [Y 33.6282 dB U 38.9792 dB V 37.9861 dB] [ET 2 ] [L0 2 1 0 ] [L1 2 1 0 ] [LC 2 1 0 ] [MD5:2d270756a661bef00b4d560a65e05786]
POC 4 ( B-SLICE, QP 33 ) 8544 bits [Y 35.1661 dB U 39.2272 dB V 38.8317 dB] [ET 3 ] [L0 3 2 1 0 ] [L1 3 2 1 0 ] [LC 3 2 1 0 ] [MD5:3cff5217f5e8a583aa2ff81cc8eec5ae]
POC 5 ( B-SLICE, QP 35 ) 1208 bits [Y 34.1652 dB U 39.0674 dB V 38.6094 dB] [ET 3 ] [L0 4 3 2 1 ] [L1 4 3 2 1 ] [LC 4 3 2 1 ] [MD5:a7475351d7b8d45d96915216386435ec]
POC 6 ( B-SLICE, QP 34 ) 2472 bits [Y 33.9374 dB U 38.9547 dB V 38.1424 dB] [ET 3 ] [L0 5 4 3 2 ] [L1 5 4 3 2 ] [LC 5 4 3 2 ] [MD5:d4eb37372ad5e4e08a25bfe27c138125]
POC 7 ( B-SLICE, QP 35 ) 2080 bits [Y 33.5086 dB U 38.7835 dB V 37.7761 dB] [ET 3 ] [L0 6 5 4 3 ] [L1 6 5 4 3 ] [LC 6 5 4 3 ] [MD5:8ac93966370d07fe738631dc62842e43]
POC 8 ( B-SLICE, QP 33 ) 9008 bits [Y 35.0873 dB U 39.2397 dB V 38.9473 dB] [ET 3 ] [L0 7 6 5 4 ] [L1 7 6 5 4 ] [LC 7 6 5 4 ] [MD5:7083472bb2123aa971a996e3fcfd2ddb]

SUMMARY --------------------------------------------------------

Total Frames | Bitrate Y-PSNR U-PSNR V-PSNR

9 a 385.9111 34.4088 39.1335 38.5024

I Slices--------------------------------------------------------

Total Frames | Bitrate Y-PSNR U-PSNR V-PSNR

1 i 1988.8000 35.7291 39.5039 39.3209

P Slices--------------------------------------------------------

Total Frames | Bitrate Y-PSNR U-PSNR V-PSNR

0 p -1.#IND -1.#IND -1.#IND -1.#IND

B Slices--------------------------------------------------------

Total Frames | Bitrate Y-PSNR U-PSNR V-PSNR

8 b 185.5500 34.2437 39.0872 38.4001

RVM: 0.000

Total Time: 18.750 sec.

Decoding:

HM software: Decoder Version [3.0][Windows][VS 1500][32 bit]

POC 0 ( I-SLICE, QP 32 ) [DT 0.015] [L0 ] [L1 ] [MD5:d9a6ef5e818feaa199a227af58784ca6,(OK)]
POC 1 ( B-SLICE, QP 35 ) [DT 0.016] [L0 0 ] [L1 0 ] [LC 0 ] [MD5:b3df8c513e3209a0b14307c9d9ea09b1,(OK)]
POC 2 ( B-SLICE, QP 34 ) [DT 0.000] [L0 1 0 ] [L1 1 0 ] [LC 1 0 ] [MD5:c880ad3d1cb40000b4082b53f250265a,(OK)]
POC 3 ( B-SLICE, QP 35 ) [DT 0.000] [L0 2 1 0 ] [L1 2 1 0 ] [LC 2 1 0 ] [MD5:2d270756a661bef00b4d560a65e05786,(OK)]
POC 4 ( B-SLICE, QP 33 ) [DT 0.000] [L0 3 2 1 0 ] [L1 3 2 1 0 ] [LC 3 2 1 0 ] [MD5:3cff5217f5e8a583aa2ff81cc8eec5ae,(OK)]
POC 5 ( B-SLICE, QP 35 ) [DT 0.000] [L0 4 3 2 1 ] [L1 4 3 2 1 ] [LC 4 3 2 1 ] [MD5:a7475351d7b8d45d96915216386435ec,(OK)]
POC 6 ( B-SLICE, QP 34 ) [DT 0.016] [L0 5 4 3 2 ] [L1 5 4 3 2 ] [LC 5 4 3 2 ] [MD5:d4eb37372ad5e4e08a25bfe27c138125,(OK)]
POC 7 ( B-SLICE, QP 35 ) [DT 0.000] [L0 6 5 4 3 ] [L1 6 5 4 3 ] [LC 6 5 4 3 ] [MD5:9fa944844475f06d77ad1fbbbff687d3,(***ERROR***)] [rxMD5:8ac93966370d07fe738631dc62842e43]
POC 8 ( B-SLICE, QP 33 ) [DT 0.000] [L0 7 6 5 4 ] [L1 7 6 5 4 ] [LC 7 6 5 4 ] [MD5:415f4387d617f0381f9eef090665b6c5,(***ERROR***)] [rxMD5:7083472bb2123aa971a996e3fcfd2ddb]

*ERROR* A decoding mismatch occured: signalled md5sum does not match

Total Time: 0.078 sec.

Sometimes, decoding crash was also observed. So far, the bug has not been located. Fortunately, this bug does not affect behavior under common conditions.

Change History (9)

comment:1 Changed 14 years ago by Vadim

I think problem is in chroma quantization, in VLC case it is up to 8x8 low frequency and also scan dependent. But, with RDOQ off it simply quantizes coefficients in zigzag order and clears after 64 position.

Conditional check should be added to xQuantLTR function in the same manner as it is done in codeCoeffNxN

comment:2 Changed 14 years ago by Xiang Li

Yeah, it seems that this bug only affects VLC and chroma. I used CABAC instead of CAVLC in the above simulation and the mismatch disappeared. Moreover, the first mismatch in the above simulation was on chroma component.

By the way, we have not tried cases other than lowdelay_loco.

comment:3 Changed 14 years ago by Vadim

I believe this code should fix a problem, insert it in xQuantLTR function just before loop
for( Int n = 0; n < iWidth*iHeight; n++ )

if( m_iSymbolMode == 0 && eTType != TEXT_LUMA && iWidth > 8 )
{
	memset( piQCoef, 0, sizeof( TCoeff ) * iWidth*iHeight );
	const UInt uiScanIdx = pcCU->getCoefScanIdx(uiAbsPartIdx, iWidth, eTType==TEXT_LUMA, pcCU->isIntra(uiAbsPartIdx));
	UInt uiBlSize = 8; //Min(8,iWidth);
	UInt uiLog2BlkSize = g_aucConvertToBit[ pcCU->isIntra( uiAbsPartIdx ) ? iWidth : uiBlSize ] + 2;
	UInt uiNoCoeff = uiBlSize*uiBlSize;
	UInt uiBlSizeInBit = g_aucConvertToBit[uiBlSize] + 2;
	UInt uiWidthInBit = g_aucConvertToBit[iWidth] + 2;

	for( Int n = 0; n < uiNoCoeff; n++ )
	{
		Long iLevel;
		Int  iSign;
		UInt uiBlockPos;

		uiBlockPos = g_auiSigLastScan[uiScanIdx][uiLog2BlkSize-1][n];

		if( !pcCU->isIntra( uiAbsPartIdx ) )
		{
			uiBlockPos = ((uiBlockPos>>uiBlSizeInBit) <<uiWidthInBit) + (uiBlockPos&(uiBlSize-1));
		}

		iLevel  = (Long) piCoef[uiBlockPos];
		iSign   = (iLevel < 0 ? -1: 1);      

		iLevel = (abs(iLevel) * uiQ + iAdd ) >> iQBits;

		uiAcSum += iLevel;
		iLevel *= iSign;        
		piQCoef[uiBlockPos] = iLevel;
	} // for n
}
else
	for( Int n = 0; n < iWidth*iHeight; n++ )
		...

comment:4 follow-up: Changed 14 years ago by Xiang Li

With Vadim's code, I didn't observe any mismatch in RAHE, RALC, LDHE and LDLC (JCTVC-E700 conditions except that first 105 frames of each sequence were coded). The performance seems reasonable. Compared with RDOQ off, RDOQ on provides gains as follows

Y U V

RAHE: -6.42% -2.49% -2.71%
RALC: -5.25% -7.08% -7.48%
LDHE: -5.41% -1.42% -1.20%
LDLC: -3.42% -7.05% -6.49%

Can we put Vadim's bug fix into HM-3.0-dev-bugfix branch? We have two cases in CE4 to be tested with RDOQ off. Thanks!

comment:5 in reply to: ↑ 4 Changed 14 years ago by Xiang Li

Sorry for the wrong numbers. The gains should be

Y U V

RAHE: -5.78% -0.42% -0.22%
RALC: -4.22% -5.74% -5.88%
LDHE: -4.47% 0.59% 1.26%
LDLC: -3.42% -7.05% -6.49%

comment:6 Changed 14 years ago by fbossen

  • Resolution set to fixed
  • Status changed from new to closed

Fixed in r850
The fix that was integrated is different from the one proposed here but should generate identical results (as indicated by limited testing). It was selected because it is simpler and does not involve code duplication.

comment:7 Changed 13 years ago by davidf

  • Component set to HM

Updating component after adding WD (Text) tickets

comment:8 Changed 13 years ago by davidf

  • Cc fbossen ksuehring davidf added

comment:9 Changed 13 years ago by davidf

  • Cc jct-vc@… added
Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber, Participant)
  • Frank Bossen(Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)
  • Vadim Seregin(Participant)
  • Xiang Li(Reporter, Participant)