In chapter 8.4.4.2.2, the substitution is described as following :

"The values of the samples p[ x ][ y ] with x = −1, y = −1..nT*2−1 and x = 0..nT*2−1, y = −1 are modified as follows:
– If all samples p[ x ][ y ] with x = −1, y = −1..nT*2−1 and x = 0..nT*2−1, y = −1 are marked as "not available for intra prediction," the value ( 1 << ( BitDepthY − 1 ) ) is substituted for the values of all samples p[ x ][ y ]."

This is applied for luma and chroma pixels.
For chroma pixels, I think the value to be used should be : ( 1 << ( BitDepthC − 1 ) )