Four Groups

Continuing some of the themes picked up by Paul Barrett in his piece on Validity and Utility in I/O Psychology, he questions the use of disattenuating correlations and the impact this has on validity and practice.

In essence, a disattenuated (or corrected) correlation is a formula for deriving the true correlation with measurement error removed. While we could easily get bogged down in statistical argument and theory, especially if measuring an entire population, as opposed what happens in practice which is that a subset of a population is analysed, the key point, as posed by Barrett, is to ask what is the point of these corrections and what are the impacts on practice? In answer to the first question he writes;

To see what the maximum possible value for a relationship might be if there were no measurement error. This value cannot be used in practice – it yields a hypothetical value that is useful for theory purposes, and for examining the effects of measurement error on a relationship. That’s all.

Regarding the use of such information, Barrett then goes on to say;

These “operational validity” = disattenuated values are equivalent to those presented in the Hunter and Schmidt meta analysis of 1998… which just goes to show the problem faced by practitioners when using meta-analytic evidence to try and make sense of job performance within groups of employees within companies. i.e. great for knowing that engineers need a high level of ability as against a mechanic, but not much use for figuring out who is your best vs worst engineer.

In other words, using this sort of approach is fine when comparing two different skills or professions, for example accountants and sales staff or nurses and teachers but when used to identify exceptional people (on either extreme) from within the same skill set or role, the use of such data has little use. The implications of this research and its conclusions then have further impact on the use of competency based approaches to managing people and Human Capital Management which I’ll go on to look at in another post.

References cited by Barrett;

Schmidt, F.L., & Hunter, J.E. (1998) The Validity and Utility of Selection Methods in Personnel Psychology: practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 2, 262-
274.

Schmidt, F.L., & Hunter, J. (2004) General mental ability in the world of work: occupational attainment and Job Performance. Journal of Personality and Social Psychology, 88, 6, 162-173.

Salgado, J.F., Anderson, N., Moscoso, S., Bertua, C., de Fruyt, F., & Rolland, J.P. (2003) A meta-analytic study of general mental ability validity for different occupations in the European Community. Journal of
Applied Psychology, 88, 6, 1068-1081.