New Professional Member Special>>> Save $15 and receive a SHRM tote bag
Many HR pros are surprised to learn that legal protection from retaliation isn’t always guaranteed for them.
Save $15 on a Professional Membership and Receive a FREE Tote Bag.
Get the HR education you need without travel expenses or time out of the office.
We don't just visit a city, we take it over. Join us in NOLA -- June 18 - 21, 2017.
Too often, when talking about the competency of Critical Evaluation, our thinking fast-forwards to processes—interpreting information, solving problems, making decisions. Although it is wise to reflect on evaluation processes, it is also important to take a step back and consider what goes on beforehand. Think about your research methodology. How will you collect the data that is to be evaluated? Especially when using a survey, you will want to ensure that the data you collect is credible. This is true whether you are early in your career and working on your first survey or an executive overseeing the survey data collection of others.
While surveys are a great way to collect quantitative (and sometimes qualitative) information, they need to be designed so that errors in measurement are reduced. A key objective is to anticipate any cognitive issues that may arise among respondents taking the survey. Writing clear, direct questions and providing all possible answer choices is critical.
The following brief questionnaire is illustrative. Read through the five questions below and see if you can detect the cognitive issues that a respondent may have when taking this survey.
ISSUE: This double-barreled question asks the respondent about two different subjects: confidence level and motivation.
AVOID IT: Break into two separate questions. Easy fix.
ISSUE: Some respondents may not understand the meaning of “autonomy.” Provide a definition of the word somewhere in the question stem or elsewhere in the survey. If you are unable to do so, add another answer choice (e.g., “Not sure” or "Not applicable).
AVOID IT: Make sure your questions are clear by testing them with someone from the target population. Or try the cognitive interview technique if you are familiar with it.
AVOID IT: To address the first issue, make sure answer choices do not overlap (i.e., the above options can be changed to 0-4, 5-9, 10-14, etc.). As for the second issue, use a reasonable time frame when asking a respondent to recall an event. If you’re not sure about what might be considered reasonable, seek input from an expert reviewer or someone from the target population.
ISSUE: Many respondents may not find the answer they want, or may want to select more than one option.
AVOID IT: An open-ended question may be more advantageous here. If you would prefer to keep it multiple choice, enlist the help of a mini focus group before you create the survey to gather all possible answers in the correct lingo.
The issues discussed above are not all-inclusive. But following these simple tips to avoid respondent response challenges will help you create effective, high-quality surveys that collect strong, credible data. This will lead to better interpretations of information, solutions to problems and decisions made—in other words, better critical evaluations.
Lindsay Northon is HR competencies specialist in the SHRM Research Department.
You have successfully saved this page as a bookmark.
Please confirm that you want to proceed with deleting bookmark.
You have successfully removed bookmark.
Please log in as a SHRM member before saving bookmarks.
Your session has expired. Please log in again before saving bookmarks.
Please purchase a SHRM membership before saving bookmarks.
An error has occurred
Recommended for you
Join SHRM's exclusive peer-to-peer social network
SHRM’s HR Vendor Directory contains over 3,200 companies