Shining a FlashlightÔ on Teaching, Learning, and Technology:

TO improve teaching and learning with technology, what should we study next, and how?

Agreed upon Stopping time for this task: _______

Name of Recorder/Reporter: ____________________

This task is designed to help a TLT Roundtable or similar group suggest what types of studies might best help improve teaching and learning with technology, and what sorts of support are needed in order to undertake the recommended studies.

As you do each step below, each participant should first answer its questions individually. Then, first in small groups and then as a whole, agree on answers for each question before moving onto the next question.

1. What studies are most important for your institution (or program) to work on first?

1. It’s not possible to study everything about everything simultaneously. (And many institutions are not studying much at all!) Yet such information can be extremely valuable in improving teaching and learning with technology while reducing some of the anxiety that surrounds the huge investments in infrastructure. Your TLT Roundtable can help the institution if it can make some recommendations about what to study and what kinds of support will be needed. The first question: which program(s), services, issues, infrastructure or competencies should your institution study first? (Brainstorm as many as ten possible study targets and then pick your top three candidates.
   
   To pick your candidates for studies, you might consider criteria such as:
   + the direct or indirect educational importance of the target;
   + the leverage that the findings might give you for improving services, perhaps helping your spot hidden problems or figure out which methods are working better;
   + the amount of time and money currently involved in the activity (because expensive activities usually offer more opportunity for saving time and money than do activities that are already inexpensive);
   + frequency of the activities across the institution. For example, you might decide to do a study of strategy a few faculty use for managing student e-mail because many faculty members currently feel overloaded by e-mail and the new strategy seems to offer a way to improve learning in a large number of courses while managing time spent on e-mail. Such a study would probably be more valuable than studying an experimental self-study module that only a few students use each year.
   
   
   It may also help to consider some different reasons that institutions do such studies, including:
   + accountability and program improvement (e.g., self-studies for accreditation),
   + creating large scale improvements in teaching and learning (e.g., using periodic evaluation to focus people’s attention on key issues of good practice; using evaluation to help ‘debug’ new initiatives);
   + budgeting (mapping problems and needs as part of building a case for the need for more money; finding ways to do some things more efficiently so that resources can be shifted toward other areas of need).
   + improving the ‘match’ between academic services (e.g., libraries, Internet services) and the curricular and academic programs they serve;
   + taking a sober look at controversial programs or practices.
   
   Yet another set of questions that might help you brainstorm about possible targets for studies:
   * Downside: if no study were done, how risky would the institution’s continued "ignorance" be?
   * Upside: how important could potential findings be for improving access, learning outcomes and/or cost-effectiveness of educational programs?
   * Can a study be completed quickly enough to use the findings before they become obsolete? (Get some expert help on this one; you may be able to get data more quickly than you think)
   * Don’t forget to consider studying things you haven’t done yet. Baseline data (data about the state of affairs before an innovation begins) can come in handy later when someone wants to know whether the improvement was really an improvement. You don’t want to have to say, "We have no idea because we don’t know how well or badly things were going before the innovation was put in place." Also such studies can be useful in shaping budget requests for the proposed improvement.
   
   Possible Targets for Studies (then put a check beside your top 2-3 priorities)
   *
   *
   *
   *
   *
   *
   *
   *
   *
   *
   *
   

• After the group discusses the possibilities, record here the 1-3 study topics on which your group will focus during the remainder of this task:
  >
  >
  >
  

II. Factors Affecting Who Designs the Study, and How It Is Designed

1. Users of your findings: Whom do you see as the users of the findings (e.g., if you were doing a study of the usefulness of library services and the Internet in supporting the curriculum, would you search for information that might influence the ways that students choose to learn? The way teachers choose to teach? The way librarians acquire print? The way webmasters craft Web sites? The ways the legislature allocates funds? Who are the audiences for your study and what do they need to know before they can act? ).
   
   

Users of your findings (you hope)	Specific actions by the users that you hope your findings will influence

2. Knowing where you’re starting and what you’re up against. What studies have been done about your targets already? What uses have been made of the data? If studies haven’t been done or the findings did not influence practice, what went wrong?




 





III. Support for Your Study (Infrastructure for Evaluation)


3. Support: Is appropriate support available? (By making sure such support is available, you’ll be helping your target studies and you will also be laying the groundwork for more such studies in the future.) For example:

• If your plan implies some coordination among studies being designed by different people, who will train the various designers? Help them decide whether to ask certain common questions so that they can share data? Who will maintain the database?
  
  
  
• What people or offices could help with the design of such studies? (e.g., helping make sure that questions on surveys are unambiguous, layout for surveys, etc.)
  
  
  
  
  
• Who could help with data entry, e.g., entry of answers via the Web, aid in creating scannable answer sheets, overseeing students who would enter data into your database from paper surveys?
  
  
  
  
• Who could provide assistance with analysis (e.g., statistical help)

• Who could provide funds?

• If people do studies of this sort, is it helping them win promotion? If that seems an issue, could the TLT Roundtable play a role in publicizing the importance of such studies for the institution and its students?

1. Which people and units will need to be supportive in order to make sure that these studies happen? Consider your answers to all the preceding questions in answering this one. There may be other groups to consider, too. For example, suppose the study involves sending a survey to all students. You want all students to see the survey as legitimate and be thoughtful in their answers. Is it important to get the early involvement of the student government or other student groups in helping you to design the survey and then to endorse it?