MaxDiff
The MaxDiff is a long-established academic mathematical theory with very specific assumptions about how people make choices:[1] it assumes that respondents evaluate all possible pairs of items within the displayed set and choose the pair that reflects the maximum difference in preference or importance. It may be thought of as a variation of the method of Paired Comparisons. Consider a set in which a respondent evaluates four items: A, B, C and D. If the respondent says that A is best and D is worst, these two responses inform us on five of six possible implied paired comparisons:
- A > B, A > C, A > D, B > D, C > D
The only paired comparison that cannot be inferred is B vs. C. In a choice among five items, MaxDiff questioning informs on seven of ten implied paired comparisons.
Overview
In 1938 Richardson[2] introduced a choice method in which subjects reported the most alike pair of a triad and the most different pair. The component of this method involving the most different pair may be properly called "MaxDiff" in contrast to a "most-least" or "best-worst" method where both the most different pair and the direction of difference are obtained. Ennis, Mullen and Frijters (1988)[3] derived a unidimensional Thurstonian scaling model for Richardson's method of triads so that the results could be scaled under normality assumptions about the item percepts.
MaxDiff may involve multidimensional percepts, unlike most-least models that assume a unidimensional representation. MaxDiff and most-least methods belong to a class of methods that do not require the estimation of a cognitive parameter as occurs in the analysis of ratings data. This is one of the reasons for their popularity in applications. Other methods in this class include the 2- and 3-alternative forced choice methods, the triangular method which is a special case of Richardson's method, the duo-trio method and the specified and unspecified methods of tetrads. All of these methods have well developed Thurstonian scaling models as discussed recently in Ennis (2016)[4] which also includes a Thurstonian model for first-last or most-least choice and ranks with rank-induced dependencies. There are a number of possible processes through which subjects may make a most-least decision, including paired comparisons and ranking, but it is typically not known how the decision is reached.
Relationship to best–worst scaling
Maxdiff and best–worst scaling (BWS) have erroneously been considered synonyms.[5] Respondents can produce best-worst data in any of a number of ways, with a maxdiff process being but one. Instead of evaluating all possible pairs (the maxdiff model), they might choose the best from n items, the worst from the remaining n-1, or vice versa (sequential models). Or indeed they may use another method entirely. Thus it should be clear that maxdiff is a subset of BWS; maxdiff is BWS, but BWS is not necessarily maxdiff. Indeed, maxdiff might not be considered an attractive model on psychological and intuitive grounds: as the number of items increases, the number of possible pairs increases in a multiplicative fashion: n items produces n(n-1) pairs (where best-worst order matters). Assuming respondents do evaluate all possible pairs is a strong assumption. Early work did use the term maxdiff to refer to BWS, but with Marley's return to the field,[6] correct academic terminology has been disseminated throughout Europe and Asia-Pacific.
Commercial software named maxdiff
In North America the term maxdiff continues to be used for studies that are in fact BWS studies. This probably reflects the fact that Sawtooth Software in the United States implements a procedure it has named maxdiff. Indeed, it is unclear whether this procedure implements maxdiff procedures in estimating parameters of their models, or whether the simpler, sequential BWS model is used.[7]
Datagame has implemented an application called MaxDiff Rankifier which uses gamification techniques and interactive visuals to streamline the data collection process.
Conduct and analysis of a maxdiff survey
A maxdiff survey may be designed and analysed in two ways:[citation needed]
- Through manual design and researcher-led analysis using standard statistical software procedures (see BWS) or
- By utilising a commercial survey company which supports this type of discrete choice model.
Analysis of a MaxDiff survey will usually contain data that shows how many times each attribute was displayed, in addition to the numbers of times the attribute was chosen as best and worst. These three data points quantify the relative importance of each item. [8]
References
- ^ Marley, Anthony AJ; Louviere, Jordan J. (1 January 2005). "Some probabilistic models of best, worst, and best–worst choices". Journal of Mathematical Psychology. 49 (6): 464–480. doi:10.1016/j.jmp.2005.05.003.
- ^ Richardson, MW (1938). "Multimensional psychophysics". Psychological Bulletin. 35: 659–660. doi:10.1037/h0055433.
- ^ Ennis, Daniel M; Mullen, Kenneth; Frijters, Jan ER (1988). "Variants of the method of triads: Unidimensional Thursonian models". British Journal of Mathematical and Statistical Psychology. 41: 25–36. doi:10.1111/j.2044-8317.1988.tb00885.x.
- ^ Ennis, Daniel M (June 2016). Thurstonian Models: Categorical Decision Making in the Presence of Noise. The Institute for Perception. ISBN 9780990644606.
- ^ Louviere, Jordan J; Flynn, Terry N; Marley, A A J (September 2015). Best-Worst Scaling: Theory, Methods and Applications. Cambridge University Press. ISBN 9781107043152. Retrieved 2 October 2015.
- ^ Marley, A. A. J.; Louviere, J. J. (1 December 2005). "Some probabilistic models of best, worst, and best–worst choices". Journal of Mathematical Psychology. Special Issue Honoring Jean-Claude Falmagne: Part 1. 49 (6): 464–480. doi:10.1016/j.jmp.2005.05.003.
- ^ Flynn, Terry N.; Louviere, Jordan J.; Peters, Tim J.; Coast, Joanna (18 November 2008). "Estimating preferences for a dermatology consultation using Best-Worst Scaling: Comparison of various methods of analysis". BMC Medical Research Methodology. 8 (1): 76. doi:10.1186/1471-2288-8-76. ISSN 1471-2288. PMC 2600822. PMID 19017376.
{{cite journal}}: CS1 maint: unflagged free DOI (link)
- ^ "MaxDiff Analysis Explained". SurveyKing. Retrieved 29 January 2020.
