A data collection technique that forces respondents to choose a best and worst option can help fine-tune your approach − and reveal some surprises.
By Dan Lowman
One of our recent clients, a highly-regarded liberal arts college, had developed a “Strategic Visioning” document, to be used as the basis for a 10-year college plan as well as the framework for a comprehensive campaign. The document was lengthy, and staff at the College sought to identify both the themes and the specific language that were most compelling to their alumni population for use in the final version.
The College commissioned GG+A to conduct focus groups with key donors, volunteers, and trustees. GG+A also conducted a broad-based online alumni survey to gather input from a larger (and generally less connected) portion of the alumni population.
GG+A employed a data collection technique known as “MaxDiff” that forces respondents to choose a best and worst option from several choices. For example, if a survey asks if a school should be supportive of its teachers, students, buildings, and curriculum, most respondents would indicate that all of these issues are important. When asked to choose the single most important, and the single least important, those distinctions become clearer.
As shown in the table above, each theme can be scored. The most highly scored themes were based on the future (“will continue,” “develop women and men,” “21st century solutions”). The low-rated themes were both in the present tense and self-congratulatory (“is extraordinary,” “is vital,” “is the best”). The results enabled the College to redesign the document to be focused on solutions and outcomes without falling back to aggrandizing terms that were perceived as complacent.
Another interesting finding from the survey was the use of the word “diversity.” Themes focused on ensuring or expanding diversity at the College scored well among relatively younger alumni, but scored poorly particularly from alumni graduating prior to 1970.
Yet, substantially similar themes that used different words were supported widely across all age bands. For example, “the College will admit the most qualified students from diverse backgrounds” scored much more poorly among older alumni than “the College will admit the most qualified students from any background.” Focus group interviews and free-text responses made clear that the term “diverse” was considered political and divisive among many older alumni. Importantly, however, that same group strongly supported use of financial aid to recruit the most qualified students, regardless of race, wealth, gender, or other similar factors.
Download a pdf of the sample survey.