Opened 12 years ago

Closed 12 years ago

#972 closed defect (fixed)

The difference of motion vector out of valid range of "short integer" type

Reported by: jeunder Owned by:
Priority: minor Milestone: HM-10.0
Component: HM Version: HM-9.2
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

In the following code to calculate boundary strength for deblocking process (in TComLoopFilter.cpp, xGetBoundaryStrengthSingle function)

pcMvP0 -= pcMvQ0; pcMvP1 -= pcMvQ1;
uiBs = (pcMvP0.getAbsHor() >= 4) | (pcMvP0.getAbsVer() >= 4) |
(pcMvP1.getAbsHor() >= 4) | (pcMvP1.getAbsVer() >= 4);

The horizontal and vertical component of motion vector was stored as "short integer" type in class TComMv, this will cause problem in the extreme case if the absolute difference between the horizontal or vertical component of motion vectors is greater than 32767 (the maximum value of "short integer" type)

Change History (3)

comment:1 Changed 12 years ago by DefaultCC Plugin

  • Cc fbossen ksuehring davidf jct-vc@… added

comment:2 Changed 12 years ago by jeunder

In the case of MVP+MVD, in 8.5.3.1 Derivation process for motion vector components and reference indices, the following rule applied

uLX[ 0 ] = (mvpLX[ 0 ] + mvdLX[ 0 ] + 216) % 216
mvLX[ 0 ] = ( uLX[ 0 ] >= 215 ) ? ( uLX[ 0 ] − 216 ) : uLX[ 0 ]
uLX[ 1 ] = (mvpLX[ 1 ] + mvdLX[ 1 ] + 216) % 216
mvLX[ 1 ] = ( uLX[ 1 ] >= 215 ) ? ( uLX[ 1 ] − 216 ) : uLX[ 1 ]

This is equivalent to

mvLX[ 0 ] = (Short)(mvpLX[ 0 ] + mvdLX[ 0 ]);
mvLX[ 1 ] = (Short)(mvpLX[ 1 ] + mvdLX[ 1 ]);

It may need to add similar rule to subtraction of motion vectors.

Last edited 12 years ago by jeunder (previous) (diff)

comment:3 Changed 12 years ago by fbossen

  • Milestone set to HM-10.0
  • Resolution set to fixed
  • Status changed from new to closed

Fixed in r3305

Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Subscriber)
  • Frank Bossen(Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)