Opened 2 years ago

Last modified 2 years ago

#1415 new defect

tsShift calculation with extended_precision_processing_flag

Reported by: Vadim Owned by:
Priority: minor Milestone:
Component: RExt text Version:
Keywords: Cc: davidf, joel, teruhiko, jct-vc@…

Description

Currently, transform skip shift is calculated as follows:

tsShift = ( extended_precision_processing_flag ? Min( 5, bdShift − 2 ) : 5 ) + Log2( nTbS )	(8-300)

and seems there is no dependency on extended_precision_processing_flag, since

bdShift = Max( 20 − bitDepth, extended_precision_processing_flag ? 11 : 0 )	(8-299)

and bdShift is equal at least to 11 when extended_precision_processing_flag is enabled, so Min( 5, bdShift − 2 ) will be equal to 5.

Then, spec can be simplified as

tsShift = 5 + Log2( nTbS )

or I missed something?

Change History (1)

comment:1 Changed 2 years ago by karlsharman

Yes this expression could have been simplified. The complication was due to early changes that increased precision only for the transform-skip path. Later, the accuracy of the transformed coefficients path was also changed, altering the expression for bdShift.

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber, Always)
  • jct-vc@…(Subscriber)
  • Joel(Subscriber)
  • Karl Sharman(Participant, Always)
  • Karsten Suehring(Always)
  • Teruhiko Suzuki(Subscriber)
  • Vadim Seregin(Reporter)