wiki:SanJoséSubjectiveViewing

Version 1 (modified by davidf, 9 years ago) (diff)

--

Subjective Viewing

Preparing material for subjective viewing

Video data

Please provide a directory per proposal or reference containing:

  • decoded 8bit yuv files, using the naming convention of the original material on the uni-hannover ftp site. Eg, "BQMall_832x480_60.yuv".
    This will allow scripts to automatically determine the display parameters.
  • if the same sequence at multiple QPs is to be tested, place video files in directories called qpNN.
  • this data should be copied to Z:\XXX\SomeUsefulIdentifierEgCE11 on the playout computer.
  • please avoid using spaces in file or directory names.

For example, assuming two proposals, "REF" and "DUT" we should see the following:

$ ls -R REF/
REF/qp22/BQMall_832x480_60.yuv
REF/qp22/ParkScene_1920x1080_24.yuv
...
REF/qp27/BQMall_832x480_60.yuv
REF/qp27/ParkScene_1920x1080_24.yuv
...

$ ls -R DUT/
DUT/qp22/BQMall_832x480_60.yuv
DUT/qp22/ParkScene_1920x1080_24.yuv
...
DUT/qp27/BQMall_832x480_60.yuv
DUT/qp27/ParkScene_1920x1080_24.yuv
...

Preparing a playlist

We have a script that will create a playlist in the correct form to do a randomized A-B-A-B viewing. It requires an input file to list the sequences that are of interest and the order in which to evaluate them:

  1. Create a textfile that names the order you wish to view the test points. Eg,
    $ cat order.txt
    qp22/BQMall_832x480_60.yuv
    qp27/BQMall_832x480_60.yuv
    ...
    qp22/ParkScene_1920x1080_24.yuv
    qp27/ParkScene_1920x1080_24.yuv
    ...
    
    Note the lack of any "DUT" or "REF" prefix.
  2. Run gen-playlist-shuffled.pl to generate a playlist:
    $ gen-playlist-shuffled.pl /path/to/REF /path/to/DUT order.txt > out.plst
    
    NB, the path given to REF and DUT should be fully qualified.
  3. For a subjective test with multiple viewing sessions, it is advisable to generate multiple copies of the playlist, one for each viewing session. Each viewing session is then randomized differently.
    For example,
    $ for N in 1 2 3 4 5 6; do
         gen-playlist-shuffled.pl /path/to/REF /path/to/DUT order.txt > session$N.plst
      done
    

Running the test

Launch the videoplayer using:

$ /path/to/videoplayer --vt.readahead=128 --vdu-fps=60 --quit --full --osdstate=2 $ANYOPTIONS_IVE_MISSED out.plst

The player will quit automatically when it reaches the end of the playlist.

DF: this could be made simpler if we create a small script and associate the .plst file extension with it. Then people only have to double click on the correct playlist

Notes on test configuration

  • The gen-playlist-shuffled.pl script will produce an A-B-A-B stimulus, with A and B randomized for each test point.
    eg, for BQMall @ qp22, it might be REF-DUT-REF-DUT, but for qp27 it might be DUT-REF-DUT-REF.
  • If the test involves multiple proposals, an A-B-C-D-E-..Z-A-B-C-D-E-..Z stimulus can be generated by providing multiple directories. The stimulus order for each test point is randomized. Eg, gen-playlist-shuffled.pl REF DUT1 DUT2 DUT3 DUT4 order.txt
    DF: this seems kind of pointless, surely it should be A-A-B-B-C-C-D-D-E-E-...Z-Z, which is what we've done before

Notes on black & white level

By default, the videoplayer outputs R'G'B' at 16-235, however on a computer monitor, this will look wrong. The following options will set the output range to 0-255: --out-black=0 --out-range=256

Alternatively, one can interactively toggle between the two to see the difference by using the 'n' and 'b' keys.

DF: what range has the monitor been configured for?