Opened 11 years ago

Closed 11 years ago

Last modified 11 years ago

#1209 closed defect (fixed)

segmentation fault with custom GOP structure

Reported by: dominik.wojt Owned by:
Priority: major Milestone: HM-13.0
Component: HM Version: HM-12.1
Keywords: GOP segfault Cc: karlsharman, davidf, ksuehring, fbossen, jct-vc@…

Description (last modified by ksuehring)

I get a segmentation fault from encoder when using custom GOP structure.

#======== Coding Structure =============
IntraPeriod                   : 800          # Period of I-Frame ( -1 = only first)
DecodingRefreshType           : 1           # Random Accesss 0:none, 1:CDR, 2:IDR
GOPSize                       : 8           # GOP Size (number of B slice = GOPSize-1)
#        Type POC QPoffset QPfactor tcOffsetDiv2 betaOffsetDiv2 temporal_id #ref_pics_active #ref_pics reference pictures     predict deltaRPS #ref_idcs reference idcs 
Frame1:  P    1   0        0.5      0            0              0           0                1         -1                     0
Frame2:  P    3   0        0.5      0            0              0           1                2         -3 -2                  0
Frame3:  B    2   0        0.5      0            0              0           1                3         -2 -1 1                0
Frame4:  P    8   0        0.5      0            0              0           1                1         -8                     0
Frame5:  B    4   0        0.5      0            0              0           1                2         -4 4                   0
Frame6:  P    7   0        0.5      0            0              0           1                1         -3                     0
Frame7:  B    6   0        0.5      0            0              0           1                2         -2 1                   0
Frame8:  B    5   0        0.5      0            0              0           1                2         -1 1                   0

Attachments (2)

test.cfg​ (11.5 KB) - added by dominik.wojt 11 years ago.
test.yuv​ (75.9 KB) - added by dominik.wojt 11 years ago.

Download all attachments as: .zip

Change History (9)

comment:1 Changed 11 years ago by DefaultCC Plugin

  • Cc davidf karlsharman jct-vc@… added

Changed 11 years ago by dominik.wojt

Changed 11 years ago by dominik.wojt

comment:2 Changed 11 years ago by karlsharman

Please can you confirm that your configuration also seg-faults with HM12.1 (not RExt).

comment:3 Changed 11 years ago by dominik.wojt

I first encountered the problem in my modified version based on 12.1-dev (non RExt) branch.
Next I checked with unmodified version of latest RExt.
Now I checked against this version:
​https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-12.1/

The problem is reproducible in all three above cases.

The output of encoder is:

~/Downloads/HM-12.1/bin/TAppEncoderStatic -c test.cfg 

HM software: Encoder Version [12.1][Linux][GCC 4.8.2][64 bit] 
******************************************************************
** WARNING: --SEIDecodedPictureHash is now disabled by default. **
**          Automatic verification of decoded pictures by a     **
**          decoder requires this option to be enabled.         **
******************************************************************
***************************************************************************
** WARNING: For conforming bitstreams a valid Profile value must be set! **
***************************************************************************
***************************************************************************
** WARNING: For conforming bitstreams a valid Level value must be set!   **
***************************************************************************

Input          File          : test.yuv
Bitstream      File          : test.h265
Reconstruction File          : test_rec.yuv
Real     Format              : 72x72 30Hz
Internal Format              : 72x72 30Hz
Frame/Field                  : Frame based coding
Frame index                  : 0 - 19 (20 frames)
CU size / depth              : 32 / 3
RQT trans. size (min / max)  : 4 / 32
Max RQT depth inter          : 4
Max RQT depth intra          : 4
Min PCM size                 : 8
Motion search range          : 64
Intra period                 : 800
Decoding refresh type        : 1
QP                           : 10.00
Max dQP signaling depth      : 2
Cb QP Offset                 : 0
Cr QP Offset                 : 0
QP adaptation                : 0 (range=0)
GOP size                     : 8
Internal bit depth           : (Y:8, C:8)
PCM sample bit depth         : (Y:8, C:8)
RateControl                  : 0
Max Num Merge Candidates     : 5

TOOL CFG: IBD:0 HAD:1 SRD:1 RDQ:1 RDQTS:1 RDpenalty:0 SQP:1 ASR:0 FEN:1 ECU:0 FDM:1 CFM:0 ESD:0 RQT:1 TransformSkip:1 TransformSkipFast:1 Slice: M=0 SliceSegment: M=0 CIP:0 SAO:1 PCM:1 LosslessCuEnabled:0 WPP:0 WPB:0 PME:2  WaveFrontSynchro:0 WaveFrontSubstreams:1 ScalingList:1 TMVPMode:1 AQpS:0 SignBitHidingFlag:1 RecalQP:0

POC    0 TId: 0 ( I-SLICE, nQP 11 QP 11 )      16424 bits [Y 54.0324 dB    U 54.2189 dB    V 54.6030 dB] [ET     3 ] [L0 ] [L1 ]
Segmentation fault


n.b. to use the supplied makefile I had to disable "-Werror" option, as otherwise compilation failed with this message:

In file included from /home/dmw/Downloads/HM-12.1/build/linux/lib/TLibDecoder/../../../../source/Lib/TLibDecoder/TDecEntropy.h:43:0,
                 from /home/dmw/Downloads/HM-12.1/build/linux/lib/TLibDecoder/../../../../source/Lib/TLibDecoder/TDecCAVLC.h:45,
                 from /home/dmw/Downloads/HM-12.1/build/linux/lib/TLibDecoder/../../../../source/Lib/TLibDecoder/TDecCAVLC.cpp:38:
/home/dmw/Downloads/HM-12.1/build/linux/lib/TLibDecoder/../../../../source/Lib/TLibCommon/TComSlice.h: In member function ‘virtual Void TDecCavlc::parseVPS(TComVPS*)’:
/home/dmw/Downloads/HM-12.1/build/linux/lib/TLibDecoder/../../../../source/Lib/TLibCommon/TComSlice.h:450:96: error: array subscript is above array bounds [-Werror=array-bounds]
   Void    setMaxDecPicBuffering(UInt v, UInt tLayer)            { m_uiMaxDecPicBuffering[tLayer] = v;    }
                                                                                                ^
/home/dmw/Downloads/HM-12.1/build/linux/lib/TLibDecoder/../../../../source/Lib/TLibCommon/TComSlice.h:447:90: error: array subscript is above array bounds [-Werror=array-bounds]
   Void    setNumReorderPics(UInt v, UInt tLayer)                { m_numReorderPics[tLayer] = v;    }
                                                                                          ^
/home/dmw/Downloads/HM-12.1/build/linux/lib/TLibDecoder/../../../../source/Lib/TLibCommon/TComSlice.h:453:96: error: array subscript is above array bounds [-Werror=array-bounds]
   Void    setMaxLatencyIncrease(UInt v, UInt tLayer)            { m_uiMaxLatencyIncrease[tLayer] = v;    }
                                                                                                ^
/home/dmw/Downloads/HM-12.1/build/linux/lib/TLibDecoder/../../../../source/Lib/TLibCommon/TComSlice.h: In member function ‘virtual Void TDecCavlc::parseSPS(TComSPS*)’:
/home/dmw/Downloads/HM-12.1/build/linux/lib/TLibDecoder/../../../../source/Lib/TLibCommon/TComSlice.h:863:87: error: array subscript is above array bounds [-Werror=array-bounds]
   Void setMaxDecPicBuffering  ( UInt ui, UInt tlayer ) { m_uiMaxDecPicBuffering[tlayer] = ui;   }
                                                                                       ^
/home/dmw/Downloads/HM-12.1/build/linux/lib/TLibDecoder/../../../../source/Lib/TLibCommon/TComSlice.h:812:84: error: array subscript is above array bounds [-Werror=array-bounds]
   Void setNumReorderPics(Int i, UInt tlayer)              { m_numReorderPics[tlayer] = i;    }
                                                                                    ^
/home/dmw/Downloads/HM-12.1/build/linux/lib/TLibDecoder/../../../../source/Lib/TLibCommon/TComSlice.h:865:87: error: array subscript is above array bounds [-Werror=array-bounds]
   Void setMaxLatencyIncrease  ( UInt ui , UInt tlayer) { m_uiMaxLatencyIncrease[tlayer] = ui;      }
Version 1, edited 11 years ago by ksuehring (previous) (next) (diff)

comment:4 follow-up: Changed 11 years ago by ksuehring

  • Cc ksuehring fbossen added
  • Component changed from HM RExt to HM
  • Description modified (diff)
  • Milestone HM+RExt-4.2 deleted
  • Version changed from RExt-4.1 (HM-12.0) to HM-12.1

comment:5 in reply to: ↑ 4 Changed 11 years ago by bmandel

Replying to ksuehring: I was just switching from RExt 12.0 4.1 to RExt 12.1 5.1 today and came across the same makefile, compile error on array bounds mentioned in the prior comment. Removing -Werror allows compilation to proceed. I've also switched from a 32 bit OS (slackware 14.0) to 64 bit OS (slackware 14.1). So I tried recompiling 4.1 again and get the same array bounds error with that too. I suspect this is some issue with libraries used with this system as on AWS/Ubuntu I had no issue compiling 4.1 in the past.

comment:6 Changed 11 years ago by fbossen

Is it intentional to have ref_pics_active set to 0 in the Frame1 configuration parameter? Shouldn't it be at least 1 for a P slice?

comment:7 Changed 11 years ago by fbossen

  • Milestone set to HM-13.0
  • Resolution set to fixed
  • Status changed from new to closed

Added support for specifying I slices in GOP structure in r3777
Slice type should be changed to I for Frame1 parameter in provided configuration file

Last edited 11 years ago by fbossen (previous) (diff)
Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • Bill Mandel(Participant)
  • David Flynn(Subscriber)
  • Dominik Wójt(Reporter, Participant)
  • Frank Bossen(Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • Karl Sharman(Subscriber, Participant)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Participant, Always)