id summary reporter owner description type status priority milestone component version resolution keywords cc 1085 Insufficient bits for the result of luma quarter pel interpolation in HM code. ManojJames " The data type of the interpolated pixels in Short *dst is insufficient for a implementation of the filter conforming to JCTVC-L1003_v34 'short' can be 16 bits depending upon the platform and compiler used. template Void TComInterpolationFilter::filter(Int bitDepth, Short const *src, Int srcStride, Short *dst, Int dstStride, Int width, Int height, Short const *coeff) Examples of case when 16 bits is insufficient is the quarter-sample location 'j' (8-209). The maximum and minimum interpolated values of quarter-sample location 'j' depends on the range of 'b' (8-200). Assume eight bit input. In equation (8-200) The maximum of 'b' occurs when A(−3,0) = 0, A(−2,0) = 255, A(−1,0) = 0, A(0,0) = 255, A(1,0) = 255, A(2,0) =0, A(3,0) = 255, A(4,0) = 0. The maximum of 'b' is 88*255. The minimum of 'b' occurs when A(−3,0) = 255, A(−2,0) = 0, A(−1,0) = 255, A(0,0) = 0, A(1,0) = 0, A(2,0) =255, A(3,0) = 0, A(4,0) = 255. The minimum of 'b' is -24*255. For quarter-sample location 'j', in equation (8-209) The maximum of 'j' occurs when b(0,-3) = -24*255, b(0,−2) = 88*255, b(0,−1) = -24*255, b(0,0) = 88*255, b(0,1) = 88*255, b(0,2) = -24*255, b(0,3) = 88*255, b(0,4) = -24*255. The maximum of 'j' is 33150 The minimum of 'j' occurs when b(0,-3) = 88*255, b(0,−2) = -24*255, b(0,−1) = 88*255, b(0,0) = -24*255, b(0,1) = -24*255, b(0,2) = 88*255, b(0,3) = -24*255, b(0,4) = 88*255. The minimum value is -16830. The maximum of 'j' is greater than 32767(the maximum that can be represented by 16 bits). The current implementation of the filter does not confrom to the JCTVC-L1003_v34 draft of the standard when bi-prediction is used. " defect closed minor HM-10.1 HM HM-10.0 invalid fbossen ksuehring davidf jct-vc@…