Survey+resources

Helpful information on survey construction:

Survey questions are known as “items.”

Each item is composed of 1) a “stem” and 2) a “response format.”


 * 1) Stems.** The stem is the question or statement to which the subject provides a response. [Response formats provide a framework for subjects’ answers. These answers compose the data the investigator will analyze to answer the research question.]

Acceptable stems are short (ie, shorter than 20 words) and clearly stated in a simple sentence or statement. They should not contain modifiers such as “almost everyone” or “usually,” which can confuse respondents as in the following example:

“In the opinion of most people, the consumption of alcoholic beverages may be in some cases detrimental to the overall emotional and physical well-being of the consumer.” Strongly disagree–1, disagree–2, neutral–3, agree–4, strongly agree–5

A shorter, clearer stem statement is: “Drinking alcohol is harmful to my health.” Strongly disagree–1, disagree–2, neutral–3, agree–4, strongly agree–5.

Having definitive statements as stems allows the scale to better discriminate among respondents. Although stem statements should be unambiguously stated, they should neither prompt a particular answer from the respondent nor encourage dishonest responses. To avoid offending respondents and to encourage accurate and honest answers, items should be nonjudgmental and socially and culturally inoffensive. For example, replace “How often did you smoke cigarettes as a college student?” with “Have you ever smoked cigarettes?”

State item stems positively when ever possible. Negative item stems can confuse respondents and may fail to obtain their true opinion. For example, replace “Driving under the influence of alcohol is not socially responsible behavior” with “It is irresponsible to drive while drunk.”

Item stems should focus on only one variable. Asking subjects to select a response containing two answers is confusing and reduces the accuracy of the survey data. For example, replace “How many articles and book chapters did you read last week?” with “How many articles did you read last week?” and “How many book chapters did you read last week?”

For further issues about the wording of your questions, you can also consult the following website: http://www.socialresearchmethods.net/kb/quesword.php


 * 2) Response Formats.** Response formats may be open or closed.

An open response format allows subjects to answer a question in free text. For example, a survey assessing faculty computer skills might ask, “Please list the computer applications you use at least twice per week” and leave space for respondents to write their answers. An advantage of the open format is that subjects are not limited in their responses. However, answering open questions requires more time and effort from subjects than checking responses on a list. In this example, faculty may forget or fail to record every computer program they use or choose not to complete the survey at all. Thus, the response rate may be lower and the data less complete than that obtained using a closed format. In addition, free-text data can be difficult and time consuming to analyze.

Closed-response formats give subjects a structured way to answer items by requiring the subject to chose from a list of options. These structured responses greatly facilitate data analysis. A researcher might ask:

Please check the computer applications you use at least twice per week: Microsoft Word _, PowerPoint _, Excel_, Access_, Word Perfect_, Other_.

Including the “other” option allows subjects to provide an answer that the investigator may not have anticipated. For example, some respondents may use SPSS more than twice per week. If on the pilot testing of our survey, the investigator received a large number of SPSS answers in the “other” category, he/she would add SPSS to the list of options in the final survey. The investigator may choose from a number of formats for closed-response scales. Each scale has its strengths and weaknesses.

//Likert-type Scales.// The traditional Likert scale, with a statement as a stem followed by responses ranging from strongly disagree to strongly agree, is commonly used in surveys. This format is familiar to respondents, and it lends itself well to measuring constructs like attitudes.

The number of scale points for responses can vary, but five is usually satisfactory. Investigators should decide in advance whether to use an odd or even number of response points. An odd number of points permits respondents to adopt a neutral position, but an even number of points forces a commitment. Novice investigators may assume that increasing the number of points on a scale widens the range of responses and improves item discrimination, but this is usually not so. In a phenomenon called the “floor” or “ceiling” effect, subjects tend to choose responses that cluster at either the top or bottom of any scale. As a result of this clustering, the instrument may not capture a significant amount of the true variability in opinion among respondents. Increasing the number of points on a scale may worsen this phenomenon. For example, if 10 participants rate an STFM presentation on a 5-point scale (1=poor, 5=excellent), the ratings may range from 3 to 5. Given a 7-point scale, participants might rate the session from 5 to 7. The top 3 points in both scales capture all the variability. Investigators anticipating such a problem might experiment with different response formats such as:

How satisfied were you with the residents’ performance on the inpatient service? Very dissatisfied, dissatisfied, satisfied, very satisfied

//Rating Scales.// Rating scales, similar to Likert scales, allow degrees of expression of an underlying opinion. Item stems need not be statements, but they should be as neutral as possible to allow the scale to discriminate among respondents’ answers.

Our residents’ knowledge of cardiology is: 1-poor 2-minimally acceptable 3-acceptable 4-good

//Pictorial Scales.// Simple pictorial scales do not require respondents to be able to read, write, or count. This advantage makes them especially suitable for use with children.

How do you feel about visiting the dentist?

//Rank Lists.// A rank list of items can be useful for market research, but it should be short because respondents quickly become fatigued after too many choices. For example, a pizza restaurant might ask their customers:

Please indicate your preferred pizza toppings by ranking the following items 1 (most liked) to 5 (least liked) __1__ pepperoni __3__ sausage __2__ cheese __5__ tomatoes __4__ mushrooms

A problem with ranking lists is that they do not indicate relative degrees of liking or disliking. For example, does the above respondent like pepperoni twice as much as cheese or three times more than sausage? Also, what if the respondent’s favorite pizza topping is anchovy, which is not listed as a choice? Adding an “other” category to the list addresses this problem.

//Semantic Differential Scales//. Semantic differential scales consist of a pair of opposite responses, with several check points between them. These scales are excellent for measuring attitudes. For example:

When I think about my residents’ in-training exam scores, I feel:

Sad_._._._._._._ Happy Anxious _._._._._._._ Serene Bad_._._._._._._ Good Calm_._._._._._._ Excited

A challenge with semantic differential scales is finding pairs of true opposites. For example, is the opposite of angry “pleased,” “serene,” “calm,” or none of these?

The above is excerpted from Cindy Passmore, MA; Alison E. Dobbie, MD; Michael Parchman, MD; James Tysinger, PhD, “Guidelines for Constructing a Survey” //Family Medicine// (2002), 34(4):281-6. It was accessed (11/30/08) at: http://209.85.173.132/search?q=cache:pig9LZELPGkJ:www.stfm.org/fmhub/fm2002/apr02/rs1.pdf+survey+constructing&hl=en&ct=clnk&cd=4&gl=us&client=firefox-a

Another type of question/response not dealt with in the above article are //Dichotomous Questions:// When a question has two possible responses, we consider it **dichotomous**. Surveys often use dichotomous questions that ask for a Yes/No, True/False or Agree/Disagree response. There are a variety of ways to lay these questions out on a questionnaire: Do you believe that the death penalty is ever justified? _ Yes _ No Please enter you gender: _ Male _ Female This last portion is from: http://www.socialresearchmethods.net/kb/questype.php