LECTURE 7: QUESTIONNAIRE DESIGN

Youth Research Achieves Questionnaire Objectives

Toy-Opinion Research (YR) of Saint-Petersburg, conducts an omnibus survey of children every quarter. Typically, YR interviews 150 boys and girls between ages 6 and 8, along with 150 boys and girls between ages 9 and 12. Toy-Opinion uses mall intercepts of mothers to recruit for its one-on-one interviews, which last eight minutes!. The study obtains children’s views on favorite snack foods, television shows, commercials, radio, magazines, buzzwords, and movies.

Toy-Opinion intentionally keeps its questionnaire to eight minutes because of attention span limits of children. President Ivan Virlov notes that some clients attempt to meet all their research objectives with one study, instead of surveying, fine-tuning objectives, and re-surveying. In doing so, these clients overlook attention limits of young respondents when developing questionnaires.

“The questionnaires keep going through the approval process and people keep adding questions, ‘Well let’s ask this question, let’s add that question, and why don’t we talk about this also,” Virlov said. “And so you end up keeping children 25 minutes in a central location study and they get kind of itchy.” The response error increases and the quality of data suffers.

Virlov notes other lessons from interviewing children. When asking questions, interviewers should define the context to which the questions refers. “It involves getting them to focus on things, putting them in a situation so that they can identify with it,” Virlov said. “For example, when asking about their radio listening habits we said, ‘What about when you’re in Mom’s car, do you listen to radio?’ rather than, ‘How often do you listen to radio? More than once a day, once a day, more than once a week?’ Those are kind of big questions for little children.”

Questionnaires designed by Toy-Opinion to obtain children views on favorite snack foods, television shows, commercials, radio, magazines, buzzwords, and movies attempt to minimize response error.

Questionnaire Design Process

-Specify the Information Needed

-Specify the Type of Interviewing Method

-Determine the Content of Individual Questions

-Design the Question to Overcome the Respondent’s Inability and Unwillingness to Answer

-Decide the Question Structure

-Determine the Question Wording

-Arrange the Questions in Proper Order

-Identify the Form and Layout

-Reproduce the Questionnaire

-Eliminate Bugs by Pre-testing

Questionnaire Design Checklist

Step 1 Specify the Information Needed

1. Ensure that the information obtained fully addresses all the сcomponents of the problem. Review components of the problem and the approach, particularly the research questions, hypotheses, and characteristics that influence the research design.

2. Prepare a set of dummy tables.

3. Have a clear idea of the target population.

Step 2 Type of Interviewing Method

1. Review the type of interviewing method determined based on considerations, discussed on the last lecture

Step 3 Individual Question Content

1. Is the question necessary?

2. Are several questions needed instead of one to obtain the required information in an unambiguous manner?

3. Do not use double-barreled questions (Do you think that Coca Cola – tasty fresh soft drink?)

Step 4 Overcoming Inability and Unwillingness to Answer

1. Is the respondent informed?

2. If respondents are not likely to be informed, filter questions that measure familiarity, product use, and past experience should be asked before questions about the topics themselves.

3. Can the respondent remember?(Do you remember brand of T-shirt you had last summer?)

4. Avoid errors of telescoping and creation.

5. Questions which do not provide the respondent with cues can underestimate the actual occurrence of an event.

6. Can the respondent articulate?

7. Minimize the effort required of the respondents.

8. Is the context in which the questions are asked appropriate

9. Make the request for information seem legitimate.

10.If the information is sensitive:

a. Place sensitive topics at the end of the questionnaire.

b. Preface the question with a statement that the behavior of interest is common.

(for instance, if you are going to ask about debts on the credit card you can say, that situation with debts is very common)

c. Ask the question using the third-person technique.

d. Hide the question in a group of other questions which respondents are willing to answer.

e. Provide response categories rather than asking for specific figures.

f. Use randomized techniques, if appropriate.

Step 5 Choosing Question Structure

1. Open-ended questions are useful in exploratory research and as opening questions.

2. Use structured questions whenever possible.

3. In multiple-choice questions, the response alternatives should include the set of all possible choices and should be mutually exclusive.

4. In a dichotomous question, if a substantial proportion of the respondents can be expected to be neutral, include a neutral alternative.

5. Consider the use of the split ballot technique to reduce order bias in dichotomous and multiple-choice questions.

6. If the response alternatives are numerous, consider using more than one question to reduce the information processing demands on the respondents.

Step 6 Choosing Question Wording

1. Define the issue in terms of who, what, when, where, why, and way (the six Ws).

(What brand of shampoo do you use?)

2. Use ordinary words. Words should match the vocabulary level of the respondents

(Do you think that distribution of soft drink was organized adequately?)

3. Avoid ambiguous words: usually, normally, frequently, often, regularly, occasionally, sometimes, etc.

(How often do you visit department store:

-Never; - Seldom; - From time to time; - Often; - Regularly.)

4. Avoid leading questions that clue the respondent to what the answer should be.

(What do you think: should patriots buy import products? - Yes; - No; - Do not know.)

5. Avoid implicit alternatives that are not explicitly expressed in the options.

(Do you like to use airlines for short journeys?)

6. Avoid implicit assumptions.

(Do you have positive attitude to the balanced state budget?)

7. Respondent should not have to make generalizations or compute estimates.

(What are the grocery expenses in your family per capita?)

8. Use positive and negative statements.

Step 7 Determine the Order of Questions (Questions sequence)

1. The opening questions should be interesting, simple, and non-threatening.

2. Qualifying questions should serve as the questions at the end (info about demographic and personal characteristics).