Four Groups Blog

Many psychometric reports will feedback a series of numbers, graphs and sten scores in a cacophony of quantification. While one cannot underestimate the impact of a good visual aid, I am often left asking the question ‘what do these numbers really mean‘?

Take a score of 6 out of 10 for Extroversion for example. My question in this instance is what does 6 out of 10 actually look like? Given that it has been assigned quantitative meaning, surely it should correspond to standard quantitative rules? In other words, I should be able to observe an Extroversion score of 3 out of 10, point to it and have a tangible sense of its qualities. Unfortunately, this is far easier said that done and as for taking a score and halving it – forget it!

Consider, on the other hand, the measurement of water. We all know what a litre of water looks like, how much it weighs and we can easily divide it in half and so forth. However, while water and other physical objects do have the genuine properties of being quantitative, it remains to be seen just how genuinely quantitative the information in psychometric reports really is.

While my own point is made having seen some of the thoughts from Paul Barrett, the following makes this case far more eloquently than I ever could. The following extract is taken from Michell (1997)

The attitude of psychologists to measurement is said to display the signs of a methodological thought disorder. In this paper, the axioms of quantitative measurement are explained – and the consequences made evident for psychologists who might claim to be making “quantitative measurement”.

The abstract from the paper continues;

It is argued that establishing quantitative science involves 2 research tasks: the scientific one of showing that the relevant attribute is quantitative; and the instrumental one of constructing procedures for numerically estimating magnitudes. In proposing quantitative theories and claiming to measure the attributes involved, psychologists are logically committed to both tasks. However, they have adopted their own, special, definition of measurement, one that deflects attention away from the scientific task. It is argued that this is not accidental. From G. T. Fechner (1860) onward, the dominant tradition in quantitative psychology ignored this task. S. S. Stevens’s (e.g., 1946, 1951) definition rationalized this neglect. The widespread acceptance of this definition within psychology made this neglect systemic, with the consequence that the implications of contemporary research in measurement theory for undertaking the scientific task are not appreciated. It is argued further that when the ideological support structures of a science sustain serious blind spots like this, then that science is in the grip of some kind of thought disorder.

References;
Michell, J. (1997) Quantitative science and the definition of measurement in Psychology. British Journal of Psychology, 88, 3, 355-383.