Opened 11 years ago Closed 11 years ago #1219 closed defect (fixed)SAO code for different InternalBitDepth(C)
Description
HM-12.1+RExt-5.1 encoder crashes with segmantation fault when InternalBitDepth is less than InternalBitDepthC. It seems that the current SAO code has an issue in the case. By disabling SAO, the encoder works without error.
Example of failure:
It seems the issue is related to #1192 Attachments (1)Change History (4)comment:1 Changed 11 years ago by DefaultCC Plugin
Changed 11 years ago by karlsharmancomment:2 Changed 11 years ago by karlsharman
comment:3 Changed 11 years ago by ksuehring
The patch has been applied in r3810.
I'm still curious what the performance impact of using a sign function instead of the lookup table would be. Note: See
TracTickets for help on using
tickets. | This list contains all users that will be notified about changes made to this ticket. These roles will be notified: Reporter, Owner, Subscriber, Participant
|
The problem also affects HM.
In the latest SAO code, a table has been used to replace the sgn function. The table is initialised to be the size of the luma [is it really necessary to replace the sgn function with a look-up table, especially given the size of the table for higher bit depths?]
eg
The following produces encoder/decoder mismatch and, if you are lucky, a stack-trace being output from the encoder on exit.
./bin/TAppEncoderStatic -c cfg/encoder_randomaccess_main10.cfg -c cfg/per-sequence/BasketballDrillText.cfg -q 32 -f 9 --SEIDecodedPictureHash=1 --InternalBitDepth=8 --InternalBitDepthC=10
Patch attached for HM13.0