The Advanced MaxDiff HB test is a great way to compare many alternatives without overwhelming respondents by asking them to read and consider all items at once. It takes a list of your items to be compared, and shows them in a balanced order to each respondent 4 at a time.
The method is focused on collecting high-resolution individual-level data to be analysed by Hierarchical Bayesian model. In typical settings respondents would see
10-20 screens.
In deciding which items to show on the next screen of the MaxDiff, the system focuses on ensuring equal coverage of pairs of items among all respondents. In other words, items are chosen randomly, with bias toward pairs of items that were seen less frequently overall across respondents.
The core analysis of respondents' preferences is performed with help of Hierarchical Bayesian Multinomial Logit model. Bayesian model is estimated with Hybrid Gibbs Sampler with a random Metropolis step MCMC. The number of burn-in iterations is determined automatically when there's enough evidence for convergence.
The model takes into account properties of other items presented in a task when respondent makes a choice. The best/worst probabilities correspond to the logit transformation of the linear combination of utility scores of the packages in the task. Respondents are analysed individually, with their preference scores being a realization of pooled "average" opinion which follows a Normal distribution, at the same time reflecting their individual preferences. As a result, raw Logit coefficients are available for every respondent.
The statistics page has three display modes:
- Raw coefficients mode - aggregation of zero-minimized raw Logit coefficients. Score of 0 means the item was least preferred compared to all other items. The larger the score, the more preferable the item was.
- Average probabilities mode - aka Probability of Choice - probability of this item being selected in an exercise with two items, the second item being an average item. This statistics is calculated individually for each respondent, then aggregated using simple mean.
- Max Diff Scale % mode - is the average probabilities mode scaled down so that sum of probabilities equals 100%. This statistics is calculated individually for each respondent, then aggregated using simple mean.
When applying filters to a survey, Advanced MaxDiff HB questions will show aggregate statistics for the current subset of respondents.
Export includes:
- Raw coefficients export: zero-centered Logit coefficients for each respondent
- Raw data Export: data on what each respondent saw, and what decision was made on each task