= Subjective Viewing = == Preparing material for subjective viewing == The following describes the setup for CE8.f. The general principal can apply to other subjective viewing sessions and some notes are provided at the end as to changing the test configuration. The subjective test to be performed is of the form "Reference-A-B-Vote". The reference is the uncompressed original; A and B are candidates under test. Scoring is performed by comparing against the Reference. === Video data === Please provide a directory per proposal or reference containing: - decoded 8bit yuv files, using the naming convention of the original material on the uni-hannover ftp site. Eg, "BQMall_832x480_60.yuv". [[BR]] This will allow scripts to automatically determine the display parameters. - if the same sequence at multiple QPs is to be tested, place video files in directories called qpNN. - this data should be copied to {{{Z:\XXX\SomeUsefulIdentifierEgCE11}}} on the playout computer. - please avoid using spaces in file or directory names. For example, assuming two proposals, "dut1" and "dut2", and the uncompressed references "orig" we should see the following: {{{ $ ls -R dut1/ dut1/qp22/BQMall_832x480_60.yuv dut1/qp22/ParkScene_1920x1080_24.yuv ... dut1/qp27/BQMall_832x480_60.yuv dut1/qp27/ParkScene_1920x1080_24.yuv ... $ ls -R dut2/ dut2/qp22/BQMall_832x480_60.yuv dut2/qp22/ParkScene_1920x1080_24.yuv ... dut2/qp27/BQMall_832x480_60.yuv dut2/qp27/ParkScene_1920x1080_24.yuv ... $ ls -R orig/ orig/BQMall_832x480_60.yuv orig/ParkScene_1920x1080_24.yuv ... }}} NB, the script is designed to look for the yuv file for the "Reference" set ("orig/" in the above example) without any extra directory parts. === Preparing a playlist === Given a number of directories and a text file that lists the test points, a script will generate a playlist in the correct form to do a randomized viewing. Two test configurations are possible: - [[http://ftp.kw.bbc.co.uk/davidf/jctvc/gen-playlist-Ref-A-B.pl|gen-playlist-Ref-A-B.pl]]: Test points use the form "Reference-A-B-Vote". - [[http://ftp.kw.bbc.co.uk/davidf/jctvc/gen-playlist-A-B-A-B.pl|gen-playlist-A-B-A-B.pl]]: Test points use the form "A-B-A-B-Vote". 1. Create a textfile that names the order you wish to view the test points. Eg, {{{ $ cat order.txt qp22/BQMall_832x480_60.yuv qp27/BQMall_832x480_60.yuv ... qp22/ParkScene_1920x1080_24.yuv qp27/ParkScene_1920x1080_24.yuv ... }}} Note the lack of any "dut1" or "dut2" prefix. 1. Create a similar text file that names three test points for use in training at the start of the session. {{{ $ cat training.txt qp27/Cactus_1920x1080_50.yuv qp37/RaceHorses_832x480_30.yuv qp32/BasketballDrill_832x480_50.yuv }}} 1. Run [[http://ftp.kw.bbc.co.uk/davidf/jctvc/gen-playlist-Ref-A-B.pl|gen-playlist-Ref-A-B.pl]] to generate a playlist: [[BR]] {{{ $ gen-playlist-Ref-A-B.pl /path/to/orig /path/to/dut1 /path/to/dut2 training.txt order.txt > out.plst }}} 1. For a subjective test with multiple viewing sessions, it is advisable to generate multiple copies of the playlist, one for each viewing session. Each viewing session is then randomized differently. [[BR]] For example, {{{ $ for N in 1 2 3 4 5 6; do gen-playlist-Ref-A-B.pl /path/to/orig /path/to/dut1 /path/to/dut2 training.txt order.txt > session$N.plst done }}} === Running the test === Launch the [[http://ftp.kw.bbc.co.uk/davidf/videoplayer/|videoplayer]] using: {{{ $ /path/to/videoplayer --vt.readahead=128 --vdu-fps=60 --quit --full --osdstate=2 --osdtextcolour=0 $ANYOPTIONS_IVE_MISSED out.plst }}} The player will quit automatically when it reaches the end of the playlist. //DF: this could be made simpler if we create a small script and associate the .plst file extension with it. Then people only have to double click on the correct playlist// == Notes on test configuration == - The [[http://ftp.kw.bbc.co.uk/davidf/jctvc/gen-playlist-Ref-A-B.pl|gen-playlist-Ref-A-B.pl]] script will produce a playlist, with test points in the form "Reference-A-B-Vote". - A and B are randomized for each test point. - The ordering of test points is randomized. - Each test point is shown twice. Once in the first half of the test, once in the second. - The [[http://ftp.kw.bbc.co.uk/davidf/jctvc/gen-playlist-A-B-A-B.pl|gen-playlist-A-B-A-B.pl]] script with two directories will produce a playlist, with test points in the form "A-B-A-B-Vote" - A and B are randomized for each test point. - The ordering of test points is randomized. - The [[http://ftp.kw.bbc.co.uk/davidf/jctvc/gen-playlist-A-B-A-B.pl|gen-playlist-A-B-A-B.pl]] script with three or more directories will produce a playlist, with test points in the form "A-A-B-B-C-C-Vote" - A, B and C are randomized for each test point. - The ordering of test points is randomized. The scripts provide a means for customizing the style of test generated. See {{{%options}}}. == Notes on black & white level == By default, the videoplayer outputs R'G'B' at 16-235, however on a computer monitor, this will look wrong. The following options will set the output range to 0-255: {{{ --out-black=0 --out-range=256 }}} Alternatively, one can interactively toggle between the two to see the difference by using the 'n' and 'b' keys. //DF: what range has the monitor been configured for?