Opened 7 years ago

Closed 7 years ago

#1480 closed defect (fixed)

Data Overflow at Adaptive Color Transform HM Implementation

Reported by: swong10 Owned by:
Priority: major Milestone:
Component: HM SCC Version:
Keywords: Adaptive Color Transform Cc: ksuehring, davidf, karlsharman, jct-vc@…

Description

There is not enough precision in the internal stages of the Adaptive Color Transform.
Solution : The intermediate ACT stages must use more than 16b to avoid overflow, when not using HIGH_BIT_DEPTH_SUPPORT.

Section 8.6.8.2 (H.265 12/2016 spec)
– Residual samples rY[ x ][ y ], rCb[ x ][ y ] and rCr[ x ][ y ] are modified as follows:
tmp = rY[ x ][ y ] − ( rCb[ x ][ y ] >> 1 ) (8-338)
rY[ x ][ y ] = tmp + rCb[ x ][ y ] (8-339)
rCb[ x ][ y ] = tmp − ( rCr[ x ][ y ] >> 1 ) (8-340)
rCr[ x ][ y ] += rCb[ x ][ y ] (8-341)

When without HIGH_BIT_DEPTH_SUPPORT, HM C-model using only Short (16b; typedef Short Pel) to store pDst0/1/2.

HM-16.15+SCM-8.5
File : TComYuv.cpp, function Void TComYuv::convert()
Pel* pDst0

Int t = yo - (cg>>1);
pDst0[x] = cg + t;
pDst1[x] = t - (co>>1);
pDst2[x] = co + pDst1[x];

Thanks
Sam.

Change History (1)

comment:1 Changed 7 years ago by libin

  • Component changed from HM to HM SCC
  • Resolution set to fixed
  • Status changed from new to closed

Fixed in r4933.

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • Bin Li(Participant)
  • David Flynn(Subscriber)
  • jct-vc@…(Subscriber)
  • Karl Sharman(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)
  • Samuel Wong(Reporter)