Opened 14 years ago

Closed 14 years ago

#15 closed defect (fixed)

negative values to unsigned int

Reported by: n.shlyakhov@… Owned by: davidf
Priority: minor Milestone:
Component: HM Version:
Keywords: Cc: fbossen, ksuehring, davidf, jct-vc@…

Description

in TComRom.cpp file
const UInt g_auiLumaRun8x8[29][2][64]

contains -1s, which gives tons of warnings

Change History (6)

comment:1 Changed 14 years ago by davidf

  • Owner set to davidf
  • Status changed from new to accepted

It looks as if the -1 values aren't used by the actual implementation and exist solely to pad the table. However, no checking is done in the code that uses g_auiLumaRun8x8 for these large values (they would get passed straight to the entropy coder).

I can suggest two solutions:

  • Remove the -1 values, implicitly replacing them with 0 (there is no need to explicitly pad the array).
  • Replace the -1 values with INT_MAX, and add an assert test for their absence.

comment:2 Changed 14 years ago by anonymous

UINT_MAX would be OK as I think.

comment:3 Changed 14 years ago by fbossen

  • Resolution set to fixed
  • Status changed from accepted to closed

Fixed in r594

comment:4 Changed 13 years ago by davidf

  • Component set to HM

Updating component after adding WD (Text) tickets

comment:5 Changed 13 years ago by davidf

  • Cc fbossen ksuehring davidf added

comment:6 Changed 13 years ago by davidf

  • Cc jct-vc@… added
Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber, Participant

  • David Flynn(Owner, Subscriber, Participant)
  • Frank Bossen(Subscriber, Participant)
  • jct-vc@…(Subscriber)
  • karl.sharman@…(Always)
  • Karsten Suehring(Subscriber, Always)
  • n.shlyakhov@…(Reporter)