Questionnaire Design Boot Camp

Takeaways

1. The survey design process has a lot of steps that need to be completed BEFORE collecting data (as shown in Groves’ Survey Lifecycle diagram).

2. Aday’s “who+what+where+when+why” formula can help you to create clear, concise research questions.

Takeaways

1. Key considerations when choosing a mode:

• Interviewer administered vs self administered?

• Use of technology?

• Channel of presentation of questions?

• Topic of survey?

• Respondents?

Takeaways

1. The Response Process Model assumes people go through 5 stages in responding to a survey question (although it is not a clean and linear process).

2. Ideally, all your respondents will understand your question in the same way.

Takeaways

1. Facts are theoretically verifiable by external observation or records.

2. Recall can be an issue with factual questions: information may not be encoded, respondents may retrieve partial or generic information, or they may have forgotten.

3. Short, simple questions can help to improve recall.

Takeaways

1. Attitudinal questions can ask about opinions, feelings or predictions.

2. Context effects are always present – as a questionnaire designer, think about how to minimize their impact.

Takeaways

1. Both socially undesirable and socially desirable topics can be sensitive.

2. In general, self-administered modes get more accurate reports on sensitive questions.

3. To improve respondent comfort levels, consider:

   1. Assuring confidentiality

   2. Minimizing cognitive burden

   3. Using unfiltered questions

   4. Using loaded wording

Takeaways

1. Survey questions can be open ended, closed ended, or scaled.

2. Benefits of open ended questions: Allow respondents to be creative or bring up new topics; avoid bias from response options or scales; can be a check on data quality.

3. Watch out for primacy and recency bias.

Takeaways

1. Key considerations around scales:

• How many scale points? Odd (includes midpoint) or even (no midpoint) number of scale points?

• Bipolar (very dissatisfied to very satisfied) or unipolar (not at all satisfied to very satisfied)?

• Numeric (1-10) or verbal labels (Excellent, good, fair, poor)?

• Include “don’t know?” “Prefer not to answer?”

2. Some research recommends 7-point bipolar and 5-point unipolar scales

Takeaways

1. Key components of a questionnaire:

   a. Introduction: tell respondents who you are and what you’re doing.

   b. Screener: make sure you’re talking to the right people.

   c. Logical modules

   d. Thank you!

2. Start with easy questions; ask complex or sensitive questions late in a questionnaire.