Subjective Viewing

Preparing material for subjective viewing

Video data

Please provide a directory per proposal or reference containing:

• decoded 8bit yuv files, using the naming convention of the original material on the uni-hannover ftp site. Eg, "BQMall_832x480_60.yuv".
  This will allow scripts to automatically determine the display parameters.

• if the same sequence at multiple QPs is to be tested, place video files in directories called qpNN.

• this data should be copied to Z:\XXX\SomeUsefulIdentifierEgCE11 on the playout computer.

• please avoid using spaces in file or directory names.

For example, assuming two proposals, "REF" and "DUT" we should see the following:

$ ls -R REF/
REF/qp22/BQMall_832x480_60.yuv
REF/qp22/ParkScene_1920x1080_24.yuv
...
REF/qp27/BQMall_832x480_60.yuv
REF/qp27/ParkScene_1920x1080_24.yuv
...

$ ls -R DUT/
DUT/qp22/BQMall_832x480_60.yuv
DUT/qp22/ParkScene_1920x1080_24.yuv
...
DUT/qp27/BQMall_832x480_60.yuv
DUT/qp27/ParkScene_1920x1080_24.yuv
...

Preparing a playlist

We have a script that will create a playlist in the correct form to do a randomized A-B-A-B viewing. It requires an input file to list the sequences that are of interest and the order in which to evaluate them:

1. Create a textfile that names the order you wish to view the test points. Eg,

   $ cat order.txt
   qp22/BQMall_832x480_60.yuv
   qp27/BQMall_832x480_60.yuv
   ...
   qp22/ParkScene_1920x1080_24.yuv
   qp27/ParkScene_1920x1080_24.yuv
   ...
   

   Note the lack of any "DUT" or "REF" prefix.
2. Run ​gen-playlist-shuffled.pl to generate a playlist:
   

   $ gen-playlist-shuffled.pl /path/to/REF /path/to/DUT order.txt > out.plst
   

   NB, the path given to REF and DUT should be fully qualified.
3. For a subjective test with multiple viewing sessions, it is advisable to generate multiple copies of the playlist, one for each viewing session. Each viewing session is then randomized differently.
   For example,

   $ for N in 1 2 3 4 5 6; do
        gen-playlist-shuffled.pl /path/to/REF /path/to/DUT order.txt > session$N.plst
     done
   

Running the test

Launch the ​videoplayer using:

$ /path/to/videoplayer --vt.readahead=128 --vdu-fps=60 --quit --full --osdstate=2 $ANYOPTIONS_IVE_MISSED out.plst

The player will quit automatically when it reaches the end of the playlist.

DF: this could be made simpler if we create a small script and associate the .plst file extension with it. Then people only have to double click on the correct playlist

Notes on test configuration

• The ​gen-playlist-shuffled.pl script will produce an A-B-A-B stimulus, with A and B randomized for each test point.
  eg, for BQMall @ qp22, it might be REF-DUT-REF-DUT, but for qp27 it might be DUT-REF-DUT-REF.

• If the test involves multiple proposals, an A-B-C-D-E-..Z-A-B-C-D-E-..Z stimulus can be generated by providing multiple directories. The stimulus order for each test point is randomized. Eg, gen-playlist-shuffled.pl REF DUT1 DUT2 DUT3 DUT4 order.txt
  DF: this seems kind of pointless, surely it should be A-A-B-B-C-C-D-D-E-E-...Z-Z, which is what we've done before

Notes on black & white level

By default, the videoplayer outputs R'G'B' at 16-235, however on a computer monitor, this will look wrong. The following options will set the output range to 0-255: --out-black=0 --out-range=256

Alternatively, one can interactively toggle between the two to see the difference by using the 'n' and 'b' keys.

DF: what range has the monitor been configured for?