Opened 13 years ago

Closed 12 years ago

#553 closed defect (fixed)

Transform Subdiv flag uses CNU context when LCU size < 64x64

Reported by: pieterkapsenberg Owned by:
Priority: minor Milestone: HM-7.1
Component: HM Version: HM-7.0
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

Because transform skip flag can be coded at depth 0 when LCU is not 64x64, the context increment can be 0, which is CNU. A proper init value should be chosen (maybe the default of 154 is ok?)

Change History (5)

comment:1 Changed 13 years ago by DefaultCC Plugin

  • Cc fbossen ksuehring davidf jct-vc@… added

comment:2 Changed 13 years ago by fbossen

Can the trailing 6 contexts marked as CNU be removed?

static const UChar 
INIT_TRANS_SUBDIV_FLAG[3][NUM_TRANS_SUBDIV_FLAG_CTX] = 
{
#if SLICE_TYPE_ORDER
{ CNU,  153,  138,  138,  CNU,  CNU,  CNU,  CNU,  CNU,  CNU, }, 
{ CNU,  124,  138,   94,  CNU,  CNU,  CNU,  CNU,  CNU,  CNU, }, 
{ CNU,  224,  167,  122,  CNU,  CNU,  CNU,  CNU,  CNU,  CNU, }, 
#else
{ CNU,  224,  167,  122,  CNU,  CNU,  CNU,  CNU,  CNU,  CNU, }, 
{ CNU,  124,  138,   94,  CNU,  CNU,  CNU,  CNU,  CNU,  CNU, }, 
{ CNU,  153,  138,  138,  CNU,  CNU,  CNU,  CNU,  CNU,  CNU, }, 
#endif
};

comment:3 Changed 13 years ago by pieterkapsenberg

I think so, I don't think transform depth can ever be more than 3

comment:4 Changed 13 years ago by pieterkapsenberg

Maybe we can instead fix the context increment to be ctxinc = trafodepth + (6-log2maxCUSize). That way context 1 is always used for 32x32 split, context 2 always for 16x16 split, etc

comment:5 Changed 12 years ago by fbossen

  • Resolution set to fixed
  • Status changed from new to closed

Issue not present in recent HM version

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber)
  • Frank Bossen(Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)
  • Pieter Kapsenberg(Reporter, Participant)