Custom Query (1442 matches)

Filters
 
Or
 
  
 
Columns

Show under each result:


Results (28 - 30 of 1442)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Ticket Resolution Summary Owner Reporter
#946 fixed 32-bit Dequantization Overflow when QP=47 bheng
Description

The non-normative clipping described in H0541, and implemented in HM, attempts to avoid 32-bit overflow of intermediate results by clipping coefficients to a 15-bit range prior to dequantization for 4x4 blocks when scaling lists are used and QP>=48.

However, the intermediate results can still overflow when QP=47.

For example, using the largest weight and coefficient level:

(L * IQ * W) << (QP/6 - iShift) = (-32768 * 72 * 255) << 2 = -2406481920 (33 bit signed value)

In HM, the resulting dequantized coefficients overflow the 32 bit integers used to store these results. I believe the attached patch would fix the problem.

#1294 fixed 444, derivation of scanIdx and predModeIntra, mismatch between specification and software dominik.wojt
Description

In section 7.4.9.11, "Residual coding semantics" in derivation of scanIdx the following condition:

- If CuPredMode[ x0 ][ y0 ] is equal to MODE_INTRA and one or more of the following conditions are true:
    - log2TrafoSize is equal to 2.
    - log2TrafoSize is equal to 3 and cIdx is equal to 0.

differs from the one in reference software HM-14.0+RExt-7.2, when chromaArrayType is 3, log2TrafoSize is 3, and cIdx is different from 0. Condition in specification gives false, while condition in software gives true. Following change should fix the issue:

- If CuPredMode[ x0 ][ y0 ] is equal to MODE_INTRA and one or more of the following conditions are true:
    - log2TrafoSize is equal to 2.
    - log2TrafoSize is equal to 3 and cIdx is equal to 0.
    - log2TrafoSize is equal to 3 and chromaArrayType is equal to 3.
#1225 fixed 4:2:2 and 4:4:4 chroma cbf coding kazui.kimihiko@…
Description

In both 4:2:2 and 4:4:4 the transform tree syntax doesn't provide for non non-zero TBs.

(Reminder), For 4:2:0, an 8x8 TU that is split to four 4x4 TUs will have a single 4x4 chroma TB coded. The chroma CBF for this single 4x4 TB is that from the level above (ie 8x8).

(1) In the case of 4:2:2, which follows the 4:2:0 design, there are two 'leaf' 4x4 chroma TBs. The current spec has the issue that both chroma TBs inherit the CBF from the level above, requiring both to contain sig coefs.

(2) In the case of 4:4:4, which allows the equal splitting of luma and chroma blocks (ie an 8x8CU can be split into four 4x4 luma TBs and four 4x4 Cb TBs), all 4x4 chroma TBs will inherit the CBF from the level above.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Note: See TracQuery for help on using queries.