Criteria for Picking a Study Topic

This essay is written for assessment specialists, TLT Roundtable members, faculty members, students, and others who know they need to do a study, but don't yet know what topic precisely to study. It's written for people who have too many idea about what they might study, and are looking for help in choosing among those options for the first topic to tackle.

To pick candidates for studies, consider criteria such as:
+         the direct or indirect educational importance of the target; this includes what people learn, who can learn (access), what it costs, how fulfilling or frustrating it is to be a faculty member, staff member, or student, etc. – it's just as easy (difficult) to study an important question as an unimportant question, so you ought to pick an important one!
+         the leverage that the findings might give you for improving services, perhaps helping you spot hidden problems or figure out which methods are working better;
+         the amount of time and money currently involved in the activity (because expensive activities usually offer more opportunity for saving time and money than do activities that are already inexpensive);
+         frequency of the activity across the institution, program, or course.  Activities that occur frequently are usually more important to study and improve than activities that occur only rarely.

It may also help to consider some different reasons that institutions do such studies, including:
+         accountability and program improvement (e.g., self-studies for accreditation),
+         creating large scale improvements in teaching and learning (e.g., using periodic evaluation to focus people’s attention on key issues of good practice; using evaluation to help ‘debug’ new initiatives);
+         budgeting (Many folks think about evaluation for budgetary purposes only in terms of documenting success to help prepare requests for funds.  But, if you're trying to raise money, evaluation can also be useful in describing stubborn problems and unmet needs.)

+         improving the ‘match’ between academic services (e.g., libraries, Internet services) and the instructional programs they support;
+         taking a look at controversial programs or practices

+         reducing the stresses that can lead to faculty, staff and student burnout.

Yet another set of questions that might help you rank possible targets for studies:
*          Downside: if no study were done, how risky would the institution’s continued “ignorance” be?
*          Upside: how important could potential findings be for improving access, learning outcomes and/or cost-effectiveness of educational programs?
*          Can a study be completed quickly enough to use the findings before they become obsolete?  (Get some expert help on this one; you may be able to get data more quickly than you think)

Should we evaluate a program that is just getting started?  Some folks assume that the time to study a new initiative is after it has stabilized.  Ironically at that point many people wish that they had begun their studies years earlier.  Studies that begin when (or before) an innovation begins have at least three roles: a) gathering data that can be used later to document whether or not the new program is making a difference, b) gathering data that can help the new program detect problems that might (if undetected) ruin it, c) gathering data that can help the new program raise money.   

For a different approach to selecting a topic, see "Finding a Great Evaluative Question: The Divining Rod of Emotion." Feeling fear and excitement are "gut" indicators that you're asking questions that are really worth the effort to answer, as the examples in this paper illustrate.