Crash Configuration

See ticket #256 for suggested patch.

After patch from ticket #256 is applied, the encoder crashes at this point:

Assertion failed: pcSlice->getMaxNumMergeCand()<=MRG_MAX_NUM_CANDS_SIGNALED, fil
e ..\..\source\Lib\TLibEncoder\TEncCavlc.cpp, line 797

Did you have a look at #258? This seems to be related.

I could reproduce that with patches for #256 and #258 applied.

The new crash seems to be related to the implementation of JCTVC-G091.

In addition to the NULL pointer patch which was provided to fix ticket #256, it may be also good to change the getPULeft and getPUAbove function arguments within the DBF_DQP macro as below (see SLICEBOUNDARY_PATCH macro, which is to be applied to functions TComLoopFilter::xEdgeFilterLuma and TComLoopFilter::xEdgeFilterChroma). This makes the usage of getPULeft and getPUAbove functions similar as in functions TComLoopFilter::xGetBoundaryStrengthSingle and TComLoopFilter::xSetLoopfilterParam.

#if DBF_DQP
      iQP_Q = pcCU->getQP( uiBsAbsIdx );
      uiPartQIdx = uiBsAbsIdx;
      // Derive neighboring PU index
#if SLICEBOUNDARY_PATCH
      if (iDir == EDGE_VER)
      {
        pcCUP = pcCUQ->getPULeft (uiPartPIdx, uiPartQIdx, !pcCU->getSlice()->getSPS()->getLFCrossSliceBoundaryFlag(), false);
      }
      else  // (iDir == EDGE_HOR)
      {
        pcCUP = pcCUQ->getPUAbove(uiPartPIdx, uiPartQIdx, !pcCU->getSlice()->getSPS()->getLFCrossSliceBoundaryFlag(), false);
      }
#else
      if (iDir == EDGE_VER)
      {
        pcCUP = pcCUQ->getPULeft (uiPartPIdx, uiPartQIdx);
      }
      else  // (iDir == EDGE_HOR)
      {
        pcCUP = pcCUQ->getPUAbove(uiPartPIdx, uiPartQIdx);
      }
#endif
#if TILEBOUNDARYINDEPENDENCE_PATCH
      if (!pcCUP)
      {
        return;
      }
#endif
      iQP_P = pcCUP->getQP(uiPartPIdx);

      if(pcCU->getIPCMFlag(uiPartQIdx)) 
      {
        iQP_Q = 0; 
      }
      if(pcCUP->getIPCMFlag(uiPartPIdx)) 
      {
        iQP_P = 0; 
      }

      iQP = (iQP_P + iQP_Q + 1) >> 1;
#endif

fixed in r2126 and r2127, see also #299