Opened 9 years ago

Closed 6 years ago

#1415 closed defect (fixed)

tsShift calculation with extended_precision_processing_flag

Reported by: Vadim Owned by:
Priority: minor Milestone: v5
Component: RExt text Version: v4 (12/2016)
Keywords: Cc: davidf, joel, teruhiko, jct-vc@…

Description

Currently, transform skip shift is calculated as follows:

tsShift = ( extended_precision_processing_flag ? Min( 5, bdShift − 2 ) : 5 ) + Log2( nTbS )	(8-300)

and seems there is no dependency on extended_precision_processing_flag, since

bdShift = Max( 20 − bitDepth, extended_precision_processing_flag ? 11 : 0 )	(8-299)

and bdShift is equal at least to 11 when extended_precision_processing_flag is enabled, so Min( 5, bdShift − 2 ) will be equal to 5.

Then, spec can be simplified as 

tsShift = 5 + Log2( nTbS )

or I missed something?

Change History (3)

comment:1 Changed 9 years ago by karlsharman

Yes this expression could have been simplified. The complication was due to early changes that increased precision only for the transform-skip path. Later, the accuracy of the transformed coefficients path was also changed, altering the expression for bdShift.

comment:2 Changed 7 years ago by bbross

  • Milestone set to v5
  • Version set to v4 (12/2016)

Agreed by the authors of the AHG2 report JCTVC-AD0002.
Fix:
Replace Equation 8-29 with
tsShift = 5 + Log2( nTbS )

comment:3 Changed 6 years ago by bbross

  • Resolution set to fixed
  • Status changed from new to closed

fixed in v5 (2018/02)

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • Benjamin Bross(Participant)
  • David Flynn(Subscriber)
  • jct-vc@…(Subscriber)
  • Joel(Subscriber)
  • Karl Sharman(Participant)
  • karl.sharman@…(Always)
  • Karsten Suehring(Always)
  • Teruhiko Suzuki(Subscriber)
  • Vadim Seregin(Reporter)