Questionnaires – from Wikipedia

Questionnaire construction issues

• The research objectives and frame of reference should be defined beforehand, including the questionnaire's context of time, budget, manpower, intrusion and privacy.
• The nature of the expected responses should be defined and retained for interpretation of the responses, be it preferences (of products or services), facts, beliefs, feelings, descriptions of past behavior, or standards of action.
• Unneeded questions are an expense to the researcher and an unwelcome imposition on the respondents. All questions should contribute to the objective(s) of the research.

• The topics should fit the respondents’ frame of reference. Their background may affect their interpretation of the questions. Respondents should have enough information or expertise to answer the questions truthfully.
• The type of scale, index, or typology to be used shall be determined.
• The types of questions (closed, multiple-choice, open) should fit the statistical data analysis techniques available.
• Comparative and non-comparative scaling
• With comparative scaling, the items are directly compared with each other (example : Do you prefer Pepsi or Coke?). In non-comparative scaling each item is scaled independently of the others (Example : How do you feel about Coke?).

• Questions and prepared responses to choose from should be neutral as to intended outcome. A biased question or questionnaire encourages respondents to answer one way rather than another. Even questions without bias may leave respondents with expectations.
• The order or “natural” grouping of questions is often relevant. Prior previous questions may bias later questions.
• The wording should be kept simple: no technical or specialized words.
• The meaning should be clear. Ambiguous words, equivocal sentence structures and negatives may cause misunderstanding, possibly invalidating questionnaire results. Double negatives should be reworded as positives.
• If a survey question actually contains more than one issue, the researcher will not know which one the respondent is answering. Care should be taken to ask one question at a time.
• The list of possible responses should be inclusive. Respondents should not find themselves with no category that fits their situation. One solution is to use a final category for “other ________”.
• The possible responses should be mutually exclusive. Categories should not overlap. Respondents should not find themselves in more than one category, for example in both the “married” category and the “single” category - there may be need for a “not living with spouse” category.

• Writing style should be conversational, yet concise and accurate and appropriate to the target audience.
• Most people will not answer personal or intimate questions.
• “Loaded” questions evoke emotional responses and may skew results.
• Presentation of the questions on the page (or computer screen) and use of white space, colors, pictures, charts, or other graphics may affect respondent's interest or distract from the questions.
• Numbering of questions may be helpful.
• Questionnaires can be administered by research staff, by volunteers or self-administered by the respondents. Clear, detailed instructions are needed in either case, matching the needs of each audience.
• Methods of collection Postal – Low Cost, Anonymity, Not labour intensive Telephone – High Speed, Detailed questions, Rapport with respondents, High response rate Electronic - Low Cost, High Speed, Anonymity, Not labour intensive Personally Administered – Detailed questions, Rapport with respondents, High response rate


Types of questions

• Contingency questions - A question that is answered only if the respondent gives a particular response to a previous question. This avoids asking questions of people that do not apply to them (for example, asking men if they have ever been pregnant).
• Matrix questions - Identical response categories are assigned to multiple questions. The questions are placed one under the other, forming a matrix with response categories along the top and a list of questions down the side. This is an efficient use of page space and respondents’ time.
• Closed ended questions - Respondents’ answers are limited to a fixed set of responses. Most scales are closed ended. Other types of closed ended questions include:
• Yes/no questions - The respondent answers with a “yes” or a “no”.
• Multiple choice - The respondent has several option from which to choose.
• Scaled questions - Responses are graded on a continuum (example : rate the appearance of the product on a scale from 1 to 10, with 10 being the most preferred appearance). Examples of types of scales include the Likert scale, semantic differential scale, and rank-order scale.
• Open ended questions - No options or predefined categories are suggested. The respondent supplies their own answer without being constrained by a fixed set of possible responses. Examples of types of open ended questions include:
• Completely unstructured - For example, “What is your opinion of questionnaires?”
• Word association - Words are presented and the respondent mentions the first word that comes to mind.
• Sentence completion - Respondents complete an incomplete sentence. For example, “The most important consideration in my decision to buy a new house is . . .”
• Story completion - Respondents complete an incomplete story.
• Picture completion - Respondents fill in an empty conversation balloon.
• Thematic apperception test - Respondents explain a picture or make up a story about what they think is happening in the picture

Sample question presented using a five-point Likert item
An important distinction must be made between a Likert scale and a Likert item. The Likert scale is the sum of responses on several Likert items. Because Likert items are often accompanied by a visual analog scale (e.g., a horizontal line, on which a subject indicates his or her response by circling or checking tick-marks), the items are sometimes called scales themselves. This is the source of much confusion; it is better, therefore, to reserve the term Likert scale to apply to the summated scale, and Likert item to refer to an individual item.
A Likert item is simply a statement which the respondent is asked to evaluate according to any kind of subjective or objective criteria; generally the level of agreement or disagreement is measured. Often five ordered response levels are used, although many psychometricians advocate using seven or nine levels; a recent empirical study found that a 5- or 7- point scale may produce slightly higher mean scores relative to the highest possible attainable score, compared to those produced from a 10-point scale, and this difference was statistically significant. In terms of the other data characteristics, there was very little difference among the scale formats in terms of variation about the mean, skewness or kurtosis.
The format of a typical five-level Likert item is:
1. Strongly disagree
2. Disagree
3. Neither agree nor disagree
4. Agree
5. Strongly agree
Likert scaling is a bipolar scaling method, measuring either positive or negative response to a statement. Sometimes a four-point scale is used; this is a forced choice method since the middle option of "Neither agree nor disagree" is not available.
Likert scales may be subject to distortion from several causes. Respondents may avoid using extreme response categories (central tendency bias); agree with statements as presented (acquiescence bias); or try to portray themselves or their organization in a more favorable light (social desirability bias). Designing a scale with balanced keying (an equal number of positive and negative statements) can obviate the problem of acquiescence bias, since acquiescence on positively keyed items will balance acquiescence on negatively keyed items, but central tendency and social desirability are somewhat more problematic.

Semantic differential
Osgood's semantic differential was designed to measure the connotative meaning of concepts. The respondent is asked to choose where his or her position lies, on a scale between two bipolar adjectives (for example: "Adequate-Inadequate", "Good-Evil" or "Valuable-Worthless"). Semantic differentials can be used to describe not only persons, but also the connotative meaning of abstract concepts—a capacity used extensively in affect control theory.