Opened 13 years ago

Closed 12 years ago

#235 closed defect (wontfix)

Encoding crash when QuadtreeTUMaxDepthInter=4 (related to NSQT)

Reported by: Xiang Li Owned by:
Priority: minor Milestone:
Component: HM Version: HM-5.0
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

An encoding crash (release version) was observed at second frame on windows when QuadtreeTUMaxDepthInter was set to 4. Please find my config file in the attachment.

According to valgrind, invalid write occurred several times such as in function xIT. However, the bug has not been located.

It is believed that the bug is related to NSQT since the crash disappears when NSQT is disabled in Typedef.h.

Change History (7)

comment:1 in reply to: ↑ description Changed 13 years ago by Xiang Li

encoder_lowdelay.cfg

#InputFile                     : ../../../Sequences/CIF/crew.yuv
#InputFile                     : ../../../Sequences/WVGA/RaceHorses_832x480_30.yuv
InputFile                     : ../../../Sequences/WQVGA/BlowingBubbles_416x240_50.yuv
FrameRate                     : 30          # Frame Rate per second
FrameSkip                     : 0           # Number of frames to be skipped in input
SourceWidth                   : 416         # Input  frame width
SourceHeight                  : 240         # Input  frame height
FrameToBeEncoded              : 17           # Number of frames to be coded
LogLevel											: 31

#======== File I/O =====================
BitstreamFile                 : str.bin
ReconFile                     : rec.yuv

#======== Unit definition ================
MaxCUWidth                    : 64          # Maximum coding unit width in pixel
MaxCUHeight                   : 64          # Maximum coding unit height in pixel
MaxPartitionDepth             : 4           # Maximum coding unit depth
QuadtreeTULog2MaxSize         : 5           # Log2 of maximum transform size for
                                            # quadtree-based TU coding (2...6)
QuadtreeTULog2MinSize         : 2           # Log2 of minimum transform size for
                                            # quadtree-based TU coding (2...6)
QuadtreeTUMaxDepthInter       : 4
QuadtreeTUMaxDepthIntra       : 3

#======== Coding Structure =============
IntraPeriod                   : -1          # Period of I-Frame ( -1 = only first)
DecodingRefreshType           : 0           # Random Accesss 0:none, 1:CDR, 2:IDR
GOPSize                       : 1           # GOP Size (number of B slice = GOPSize-1)
RateGOPSize                   : 4           # GOP size used for QP assignment
NumOfReference                : 4           # Number of reference frames
NumOfReferenceB_L0            : 2           # Number of reference frames for L0 for B-slices
NumOfReferenceB_L1            : 2           # Number of reference frames for L1 for B-slices
HierarchicalCoding            : 0           # Hierarchical B coding ON/OFF
LowDelayCoding                : 1           # Low-delay coding structure
GPB                           : 1           # Replace P-slice by B-slice using two same directions
NRF                           : 0           # Mark non-reference for highest temporal layer
BQP                           : 1           # Use hier-B style QP assignment for hier-P structure
ListCombination               : 1           # Use combined list for uni-prediction in B-slices
DisableInter4x4               : 1           # Disable Inter4x4 (1: Disable Inter4x4, 0: Enable Inter4x4)
UsingNewRefSetting            : 1           # Using 1+X reference frame settting for LD
#=========== Motion Search =============
FastSearch                    : 1           # 0:Full search  1:EPZS
SearchRange                   : 64          # (0: Search range is a Full frame)
BipredSearchRange             : 4           # Search range for bi-prediction refinement
HadamardME                    : 1           # Use of hadamard measure for fractional ME
FEN                           : 1           # Fast encoder decision

#======== Quantization =============
QP                            : 22          # Quantization parameter(0-51)
MaxDeltaQP                    : 0           # CU-based multi-QP optimization
MaxCuDQPDepth                 : 0           # Max depth of a minimum CuDQP for sub-LCU-level delta QP
DeltaQpRD                     : 0           # Slice-based multi-QP optimization
RDOQ                          : 1           # RDOQ

#=========== Entropy Coding ============
SymbolMode                    : 1           # 0:LCEC, 1:CABAC
#=========== Deblock Filter ============
LoopFilterDisable             : 0           # Disable loop filter in slice header (0=Filter, 1=No Filter)
LoopFilterAlphaC0Offset       : 0           # Range: -26 ~ 26
LoopFilterBetaOffset          : 0           # Range: -26 ~ 26

#=========== Misc. ============
InternalBitDepth              : 10          # internal bit-depth = BitDepth + BitIncrement

#=========== Coding Tools =================
MRG                           : 1           # Merge mode                      (0: OFF, 1: ON)
SAO                           : 1           # Sample adaptive offset  (0: OFF, 1: ON)
ALF                           : 1           # Adaptive loop filter  (0: OFF, 1: ON)
ALFEncodePassReduction        : 0           # Reduce ALF encoding passes (0:Original 16-pass, 1: 1-pass, 2: 2-pass encoding)

#============ Slices ================
SliceGranularity         : 0                # Slice granularity relative the LCU size
SliceMode                : 0                # 0: Disable all slice options.
                                            # 1: Enforce maximum number of LCU in an slice,
                                            # 2: Enforce maximum number of bytes in an 'slice'
SliceArgument            : 1500             # Argument for 'SliceMode'.
                                            # If SliceMode==1 it represents max. SliceGranularity-sized blocks per slice.
                                            # If SliceMode==2 it represents max. bytes per slice.

LFCrossSliceBoundaryFlag : 1                # In-loop filtering, including ALF and DB, is across or not across slice boundary.
                                            # 0:not across, 1: across

EntropySliceMode         : 0                # 0: Disable all entropy slice options.
                                            # 1: Enforce maximum number of LCU in an entropy slice,
                                            # 2: Enforce maximum number of bins/bits in an 'entropy slice'
EntropySliceArgument     : 180000           # Argument for 'EntropySliceMode'.
                                            # If EntropySliceMode==1 it represents max. SliceGranularity-sized blocks per slice.
                                            # If EntropySliceMode==2 it represents max. bins per entropy slice.

#============ PCM ================
PCMLog2MinSize                      : 7                # Log2 of minimum PCM block size.
PCMInputBitDepthFlag                : 1                # 0: PCM bit-depth is internal bit-depth. 1: PCM bit-depth is input bit-depth.
PCMFilterDisableFlag                : 0                # 0: Enable loop filtering on I_PCM samples. 1: Disable loop filtering on I_PCM samples.

### DO NOT ADD ANYTHING BELOW THIS LINE ###
### DO NOT DELETE THE EMPTY LINE BELOW ###

Last edited 13 years ago by ksuehring (previous) (diff)

comment:2 Changed 13 years ago by davidf

Updating component after adding WD (Text) tickets

comment:3 Changed 13 years ago by Xiang Li

It has been confirmed by the NSQT proponent that the bug is related two NSQT functions. The bug also exists in HM-5.0. Fortunately, it has no impact on common test conditions. The NSQT proponent promised to provide a bug fix in one week. They would also solve the "z-order" issue with NSQT later.

comment:4 Changed 13 years ago by Xiang Li

  • Version changed from HM-4.0 to HM-5.0

comment:5 Changed 13 years ago by davidf

  • Cc fbossen ksuehring davidf added

comment:6 Changed 13 years ago by davidf

  • Cc jct-vc@… added

comment:7 Changed 12 years ago by davidf

  • Resolution set to wontfix
  • Status changed from new to closed

NSQT was removed from the spec

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber, Participant)
  • Frank Bossen(Subscriber)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Participant, Always)
  • Xiang Li(Reporter, Participant)