Let's start by defining "data quality control" questions. We prefer to call them quality controls (QCs) because those questions are not "red herrings" and not "trap questions." We are not trying to mislead or trap survey respondents. The goal of QCs is to confirm that participants are carefully reading the questions and response options. QCs should measure attention, not other constructs related to memory or specific knowledge.

QCs can improve data quality, but if they are too convoluted, there is a risk of "over-cleaning" the data and losing valuable responses. We should also be considerate of the engaged respondents — who are the vast majority in every survey — and be respectful of the person sitting behind the screen by applying an empathetic approach to QC design.

Quality Control Questions: Don'ts

• Don't add QCs to the screening portion of the survey. Respondents usually pay full attention at the very beginning, and respondents who are terminated may affect the overall IR.
• Don't include multiple QCs unless the survey is longer than 20 minutes. Instead, use QCs in conjunction with other data quality metrics such as speeding, straightlining, open ends, conflicting answers, etc.
• Don't include QCs that relate to the actual survey topic — respondents will try to interpret the question in context of the questionnaire.
• Don't include math questions (such as "How much is 2+2?"). QCs are not quizzes or math tests, and it could be demeaning to some respondents.
• Don't include "red herrings" such as fake brands — respondents might be confused and think of a similar brand that actually exists.

Quality Control Questions: Dos

• Do keep QCs simple — the accurate response option should be only one (no multi-select, grid questions, etc.) so respondents can move on to the actual survey questions as easily as possible.
• Do include an indication in the question text that the question is for "quality control" purposes only. Otherwise, some respondents may think there is a mistake and will try to answer "correctly" based on the survey topic.
• Do make sure all response options are culturally neutral and applicable for multi-country surveys. All response options should have clear translations in all languages and should be native to each country.

Suggested Data Quality Check Templates

Template 1: For data quality control purposes, please select 'Strawberry' from the list below. [SINGLE CHOICE]

1. Apple
2. Banana
3. Peach
4. Orange
5. Strawberry

Template 2: The following question is for data quality control purposes only. Which of the following is NOT an animal? [RANDOMIZE, SINGLE CHOICE]

1. Cat
2. Dog
3. Car
4. Monkey

Template 3: Please select "very well" from the following list — this question is for data quality purposes only. [SINGLE CHOICE]

1. Extremely well
2. Very well
3. Somewhat well
4. Not at all well

Template 4: For data quality control purposes, please select the color 'green' from the list below. [SINGLE CHOICE]

1. Blue
2. Green
3. Orange
4. Red
5. White

Template 5: We very much appreciate your thoughtful responses to this survey so far. The following question has been added for data quality control purposes only. Please select "B" below. [SINGLE CHOICE]

1. A
2. B
3. C
4. D

Template 6: The following question is for data quality control purposes only. Please select the option 'disagree'. [SINGLE CHOICE]

1. Strongly Disagree
2. Disagree
3. Neither agree or disagree
4. Agree
5. Strongly Agree

Template 7: Please select the only answer option that is a color. (This question is for data quality purposes only.) [RANDOMIZE, SINGLE CHOICE]

1. Picture
2. Shirt
3. Purple
4. Ocean
5. River

Template 8: For quality control purposes only, please select "dark chocolate" below. [RANDOMIZE, SINGLE CHOICE]

1. White Chocolate
2. Milk chocolate
3. Dark chocolate
4. Truffles
5. None of the above [ANCHOR]

Template 9: The following question is for data quality control purposes only. Please select the item that is not an ice cream flavor below. [RANDOMIZE, SINGLE CHOICE]

1. Strawberry
2. Cookie Dough
3. Mint Chocolate Chips
4. Basketball
5. French Vanilla

Template 10: We very much appreciate your thoughtful responses to this survey so far. The following question has been added for data quality control purposes only. Please select 'soccer' from the list of sports below. [SINGLE CHOICE]

1. Basketball
2. Football
3. Hockey
4. Soccer
5. Tennis

Template 11: Please select "neutral" from the following list — this question is for data quality purposes only. [SINGLE CHOICE]

1. Very unimportant
2. Unimportant
3. Neutral
4. Important
5. Very important

Template 12: The following question is for data quality control purposes only. Which of the following items is not a sport? [RANDOMIZE, SINGLE CHOICE]

1. Tennis
2. Peanut Butter
3. Soccer
4. Basketball

Template 13: Please select the last response option below — this question is for data quality purposes only. [SINGLE CHOICE]

1. Strongly Disagree
2. Disagree
3. Neither agree or disagree
4. Agree
5. Strongly Agree

Template 14: We very much appreciate your thoughtful responses throughout the survey. The following question has been inserted as a quick and simple quality check before moving forward. Of the five response options below, please select "Very Likely" at the bottom to continue. [SINGLE CHOICE]

1. Very unlikely
2. Unlikely
3. Neither Likely nor Unlikely
4. Likely
5. Very likely