Opened 12 years ago

Closed 12 years ago

#839 closed defect (fixed)

Error in strong intra smoothing formulas.

Reported by: bheng Owned by: bbross
Priority: minor Milestone:
Component: Text Version: D9 (K1003) v9
Keywords: Cc: bbross, wjhan, jct-vc@…

Description

There is a mismatch between the WD and the HM software regarding strong intra smoothing. Specifically, the +32 for rounding in the following formulas should be added after the multiplication.

The additional parenthesis below would resolve the mismatch.

pF[ −1 ][ y ] = p[ −1 ][ −1 ] + (( y + 1 )*(p[ −1 ][ 63 ] − p[ −1 ][ −1 ]) + 32 ) >> 6 for y = 0..62 (8-30)

pF[ x ][ −1 ] = p[ −1 ][ −1 ] + (( x + 1 )*(p[ 63 ][ −1 ] − p[ −1 ][ −1 ]) + 32 ) >> 6 for x = 0..62 (8-32)

Change History (2)

comment:1 Changed 12 years ago by DefaultCC Plugin

  • Cc bbross wjhan jct-vc@… added

comment:2 Changed 12 years ago by bbross

  • Resolution set to fixed
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • Benjamin Bross(Owner, Subscriber, Participant)
  • Brian Heng(Reporter)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Always)
  • Woo-Jin Han(Subscriber)