Main Content Area
Continued Misuse of 1-Tem Measures
It is typical in surveys by many practitioners and even some professors to use just one statement or question to measure an attitude, intention, or emotion of interest. Yet, measurement experts have long agreed that such measures tend to be unreliable. That is why scholars who hope to have their work published in the top academic journals tend to use multi-item scales, particularly for measuring the most important constructs in their studies. The greater sensitivity of the scales improves the accuracy of a study’s findings and the quality of conclusions based on them.
Let's backup and recall that much of the information that researchers want to gather from consumers is psychological in nature and there probably isn’t one, perfect item that fully measures the construct of interest. Thus, any 1-item measure has an unknown amount of error. The lack of reliability found in 1-item scales leads to even more problems when trying to measure the relationship between two constructs. The correlation is lower than what would be found if the relationship was measured using two multi-item scales of higher reliability. If all of that is not enough to scare you from using simplistic measures, recent research shows that the predictive validity of 1-item scales suffers compared to multi-item scales (Diamantopoulos et al. 2012).
Having said that, the many benefits of multi-item scales does not mean that dozens of items are necessary to measure something reliably. In fact, my experience shows that, long scales have their own problems. (That topic is worthy of its own Pet-Peeve at some point!) Although some academics don’t seem to understand it, the items in a scale are merely meant to be a “representative sample” of statements reflecting the construct rather than every possible statement that could be made. Thus, it is quite possible to develop a scale of 3-8 items that is reliable and valid (e.g., Bagozzi and Baumgartner 1994; Green and Rao 1970).
1-item measures have their place in a questionnaire but that is a topic for another posting. The point I am making here is that there are negative consequences of using 1-item scales for measuring psychological constructs, especially when they are of great importance to a study. Be suspicious of studies using nothing but 1-item measures of psychological constructs because their inherent low quality weaken the analyses and the conclusions based upon them. At the other extreme, it is NOT necessary to use long complicated scales to achieve reliability and validity. Admittedly, “brief” scales of high psychometric quality do take time to develop and validate. But, with the thousands of multi-item scales that have already been developed and tested, it is likely researchers can find something that will suit their purposes and will work better than quick-and-dirty 1-item measures..
Diamantopoulos, Adamantios, Marko Sarstedt, Christoph Fuchs, Petra Wilczynski, and Sebastian Kaiser (2012), "Guidelines for Choosing Between Multi-item and Single-item Scales for Construct Measurement: A Predictive Validity Perspective," Journal of the Academy of Marketing Science, 40 (3), 434-449.
Bagozzi, Richard P. and Hans Baumgartner (1994), “The Evaluation of Structural Equation Models and Hypothesis Testing,” in R.P. Bagozzi, editor, Principles of Marketing Research, Blackwell Publishers, Cambridge, MA (1994), pp. 386-422.
Green, Paul E. and Vithala R. Rao (1970), "Rating Scales and Information Recovery: How Many Scales and Response Categories to Use?" Journal of Marketing, 34 (July), 33-39.