Methods of Quality Assessment for Large Sample Sets

Psychophysical evaluation of large sample sets was studied with reference to the International Newspaper Colour Quality Club Jury Evaluation, in which 150 prints of the same image are assessed by category judgement under 10 different quality attributes.

In a series of experiments using sub-sets of the CQC prints, some psychophysical techniques which could affect the reliability and precision of the results were evaluated. It was also possible to gain insights into the relationship between the different psychophysical methods.

On the basis of these results a number of modifications to the category judgement task used in the Jury Evaluation are proposed. These include the adoption of an anchor image whose scores on the quality attributes are defined in a preliminary observer task; and a reduction in the number of attributes and judgement categories.

The paper addresses a major problem in using psychophysics for objective visual assessment of samples, which is the difficulty in efficiently scaling up the number of samples in an experiment. Many studies have relatively limited numbers of samples or inadequate experimental design or data analysis methods, and this paper shows that anchored category judgement is an appropriate technique for larger sample sets.

The results provide a rigorous basis for large-scale quality evaluations carried out in industry and in research labs. The work applies to sample sets in which a single image is reproduced by many different methods or different printers. In a continuation of this work I am investigating methods of assessing reproduction quality where the number of different images is large. When complete, this will enable researchers to ensure that quality assessments are not image-dependent.

Green coordinated and performed psychophysical experiments and data analysis. S. Lindberg contributed statistical development of the work.

My research interests are in colour imaging generally, and specifically in colour management, colour difference, colour appearance, colour metrolology, device characterization, colour gamut mapping and colour image quality. Colour imaging is an extremely active area of research across the world, but there remain many outstanding problems and each advance in technology throws up new research and development opportunities.My current research projects include: metrics for quantifying the smoothness of colour transforms; development of a reference printer architecture for colour management; scalability of colour gamut mapping algorithms; colour stability of printing processes; colour tolerances across different media and different cultures; psychophysical methods for assessing image quality in large sample sets; effect of surround and background on colour appearance; and colour reproduction on chromatic substrates.I am interested in the implementation of colour transforms, and have developed the Colour Engineering Toolbox, which provides a comprehensive set of tools in the Matlab language for this purpose.At LCC we have outstanding laboratory facilities in colour measurement and colour imaging, and I and my students have been able to participate in numerous national and international research projects coordinated by bodies such as CIE and ICC. We have also worked on projects with researchers at large companies such as HP and IBM.