The Office of Institutional Effectiveness oversees the administration of national surveys to the Whittier College community for the use of informed decision-making.
Besides the Office of Institutional Effectiveness, many people and organizations on and off of our campus conduct surveys of Whittier students. The number of invitations to participate in on-line surveys has increased dramatically, to the point of survey fatigue.
If you are interested in surveying Whittier students, please consider the following:
Conduct a survey, focus group, interviews or observation
Whittier College regularly administers a large Survey Calendar (Moodle login required) to undergraduate students at various stages during their years at Whittier. You may also want to consider alternative methodologies for collecting relevant data such as focus groups, interviews, or observations. These qualitative approaches have important advantages and may be a better choice than a survey. All the data we collect can be used to assess a wide variety of questions relevant to academic and co-curricular development.
Surveys and many results are owned by Whittier College and are archived on our Moodle page. The data is provided only to members of the Whittier College community. All external requests must go through the Office of Institutional Effectiveness. The Office requires a minimum of two weeks’ notice for a data request.
Defining your research questions
Take the time to clearly define the research questions you want to address. Check in with peers and relevant decision-makers to elicit their thoughts on important questions to address. Review the literature on your topic to see how questions have been framed by other researchers, and what related issues or themes should be taken into consideration when exploring this topic.
Developing reasonable question items and timeline
It is easy to type surveys into web survey software or questions for an interview or focus group, however, that doesn’t mean that they are quality items. We recommend that you give yourself ample time—four to six months is not unreasonable—to develop questions that make sense to you, your team, and the participants. Question items should be “pre-tested” on a small number of individuals from the population of interest and revised on the basis of their feedback.
Avoid the temptation to include questions that may provide interesting but not particularly useful results. Also, consider whether some of the data you want is available through other sources such as institutional files. Any data collection, whether a survey, focus group or interview should begin with a statement that clearly explains:
- The purpose
- That participation is voluntary
- That the respondent can skip questions he or she would prefer not to answer
- Whether responses provided will be treated as anonymous or confidential data
- How information will be reported and used
When developing question items, here are some general guidelines to bear in mind:
- Questions should not be “leading” or contain jargon or technical terms that may not be understood by all respondents
- Response categories should reflect a comprehensive array of choices, including “not applicable,” “don’t know” and/or “other” where appropriate
- Limit the use of open-ended questions; as much as possible, position these at the end of the survey instrument
- Short list of questions generate more responses and minimize the imposition on the valuable resource of our students’ time
Survey methodologists specialize in the construction of survey questions and their response categories. Consider having someone with survey design expertise review your survey instrument. Links to Whittier resources are provided at the end of this document.
Surveys should usually not be administered during periods of peak workloads, or during vacations or holidays. In general, more surveys are run during the spring semester than during the fall, so it might be advantageous to considering fielding a survey in the fall. Check the Survey Calendar to minimize respondent survey fatigue and potential conflict with other surveys. Contact Melissa Christian at mchristi@whittier.edu to have your survey added to this calendar so that others can work around it.
IRB review may be necessary
If your survey can be considered “research,” it must be reviewed by the Human Subjects & Institutional Review Board (IRB) at Whittier.
Selecting the sample
It is not necessary to survey an entire population in order to have valid, generalizable results. A random sample will do the job while minimizing costs—including the costs of survey fatigue. If you need help determining an appropriate sample size for your project, you may contact the Office of Institutional Effectiveness for assistance.
Considering incentives
Many campus surveys offer respondents the chance to be entered into a raffle drawing for prizes. Gift cards and personal electronic devices are common prizes. The literature on survey methodology suggests that these kinds of incentives have a modest impact, increasing the response rate slightly.
Surveys that offer incentives to respondents must track respondent identities in some way. If identifying information (such as names, e-mail addresses, or student identification numbers) are kept with the survey responses and confidentiality is promised to respondents, the study needs a security protocol for keeping the data safe (see next item).
Collecting data: data security
Survey respondents must be notified if their responses will be anonymous, kept confidentially, or are entirely non-confidential. Furthermore, all FERPA requirements must be complied with.
Anonymous data do not include names, addresses, student identification numbers or any other personal information that would make it possible to associate a response with any given individual.
Data that are confidential contain information that may identify an individual respondent. These files must be stored with great attention to data security and access. If you plan to collect confidential data, contact IRP or Office of Institutional Effectiveness for more information on developing a security plan.
Analyzing and reporting the results
Before you begin your survey, develop a plan for analyzing the data and reporting the results. How will you use the collected data? With whom will your results be shared? In what format will results be shared – as visual presentations, written or electronic reports?
Consider how you might share your results with others on campus who are interested in related questions. By sharing your findings with the Whittier College community, you can not only enlighten the campus community about your work, but you may also be able to head-off a new data collection effort. Remember that survey results cannot be presented or published beyond Whittier College without IRB approval.
Available resources at Whittier College
Human Subjects & Institutional Review Board (IRB): This site contains information about Whittier’s IRB policies, a decision tree to help determine whether your research will require IRB review and approval, and forms to request approval for research.
Whittier’s Office of Institutional Effectiveness: provides support for scholarly inquiry and assessment, including assistance in survey design, collecting, coding and analyzing data.
Qualtrics Survey Software: Whittier has licensed Qualtrics for creating web-based surveys.
