| Version 7 (modified by davidf, 13 years ago) (diff) |
|---|
Subjective Viewing
Preparing material for subjective viewing
The following describes the setup for CE8.f. The general principal can apply to other subjective viewing sessions and some notes are provided at the end as to changing the test configuration.
The subjective test to be performed is of the form "Reference-A-B-Vote". The reference is the uncompressed original; A and B are candidates under test. Scoring is performed by comparing against the Reference.
Video data
Please provide a directory per proposal or reference containing:
- decoded 8bit yuv files, using the naming convention of the original material on
the uni-hannover ftp site. Eg, "BQMall_832x480_60.yuv".
This will allow scripts to automatically determine the display parameters.
- if the same sequence at multiple QPs is to be tested, place video files in directories called qpNN.
- this data should be copied to Z:\XXX\SomeUsefulIdentifierEgCE11 on the playout computer.
TODO: update with correct location
- please avoid using spaces in file or directory names.
For example, assuming two proposals, "dut1" and "dut2", and the uncompressed references "orig" we should see the following:
$ ls -R dut1/ dut1/qp22/BQMall_832x480_60.yuv dut1/qp22/ParkScene_1920x1080_24.yuv ... dut1/qp27/BQMall_832x480_60.yuv dut1/qp27/ParkScene_1920x1080_24.yuv ... $ ls -R dut2/ dut2/qp22/BQMall_832x480_60.yuv dut2/qp22/ParkScene_1920x1080_24.yuv ... dut2/qp27/BQMall_832x480_60.yuv dut2/qp27/ParkScene_1920x1080_24.yuv ... $ ls -R orig/ orig/BQMall_832x480_60.yuv orig/ParkScene_1920x1080_24.yuv ...
NB, the script is designed to look for the yuv file for the "Reference" set ("orig/" in the above example) without any extra directory parts.
Preparing a playlist
Given a number of directories and a text file that lists the test points, a script will generate a playlist in the correct form to do a randomized viewing.
Two test configurations are possible:
- gen-playlist-Ref-A-B.pl: Test points use the form "Reference-A-B-Vote".
- gen-playlist-A-B-A-B.pl: Test points use the form "A-B-A-B-Vote".
- Create a textfile that names the order you wish to view the test points. Eg,
$ cat order.txt qp22/BQMall_832x480_60.yuv qp27/BQMall_832x480_60.yuv ... qp22/ParkScene_1920x1080_24.yuv qp27/ParkScene_1920x1080_24.yuv ...
Note the lack of any "dut1" or "dut2" prefix. - Create a similar text file that names three test points for use in training at
the start of the session.
$ cat training.txt qp27/Cactus_1920x1080_50.yuv qp37/RaceHorses_832x480_30.yuv qp32/BasketballDrill_832x480_50.yuv
- Run gen-playlist-Ref-A-B.pl
to generate a playlist:
$ gen-playlist-Ref-A-B.pl /path/to/orig /path/to/dut1 /path/to/dut2 training.txt order.txt > out.plst
- For a subjective test with multiple viewing sessions, it is advisable to generate
multiple copies of the playlist, one for each viewing session. Each viewing session
is then randomized differently.
For example,$ for N in 1 2 3 4 5 6; do gen-playlist-Ref-A-B.pl /path/to/orig /path/to/dut1 /path/to/dut2 training.txt order.txt > session$N.plst done
Running the test
Launch the videoplayer using:
$ /path/to/videoplayer --vt.readahead=128 --vdu-fps=60 --quit --full --osdvcentre --osdstate=2 --osdtextcolour=0 $ANYOPTIONS_IVE_MISSED out.plst
The player will quit automatically when it reaches the end of the playlist.
DF: this could be made simpler if we create a small script and associate the .plst file extension with it. Then people only have to double click on the correct playlist
Notes on test configuration
- The gen-playlist-Ref-A-B.pl script
will produce a playlist, with test points in the form "Reference-A-B-Vote".
- A and B are randomized for each test point.
- The ordering of test points is randomized.
- Each test point is shown twice. Once in the first half of the test, once in the second.
- The gen-playlist-A-B-A-B.pl script
with two directories will produce a playlist, with test points in the form "A-B-A-B-Vote"
- A and B are randomized for each test point.
- The ordering of test points is randomized.
- The gen-playlist-A-B-A-B.pl script
with three or more directories will produce a playlist, with test points in the form "A-A-B-B-C-C-Vote"
- A, B and C are randomized for each test point.
- The ordering of test points is randomized.
The scripts provide a means for customizing the style of test generated. See %options.
Notes on black & white level
By default, the videoplayer outputs R'G'B' at 16-235, however on a computer monitor, this will look wrong. The following options will set the output range to 0-255: --out-black=0 --out-range=256
Alternatively, one can interactively toggle between the two to see the difference by using the 'n' and 'b' keys.
DF: what range has the monitor been configured for?
