I. Getting Ready to Plan

What follows are some questions.  We tried to come up with the smallest number of questions whose answers would be of greatest importance for planning an institutional strategy for  uncovering important information -- information that could help improve the benefits (and control the costs) of educational uses of technology.

Who should answer the questions?  At many institutions an action team from the Teaching, Learning, and Technology Roundtable (TLTR) is a sensible group to put to work on this. At others some particular office may take the lead: Chief Academic Officer, Chief Information Officer, Teaching and Learning Center, the Library, Institutional Research,...  But a TLTR-like team, drawing on these and other centers of concern, would be best. That's because inquiries such as these can and should advance many agendas: more return for the technology dollar, the scholarship of teaching, better program outcomes, better distance learning, accreditation self-study, integration of the library and the academic program, ...

II. Two Complementary Strategies. One of These? or Both?

1. Bottom Up - culture of inquiry, scholarship of teaching: What priority should our institution place on increasing the number of people (faculty members, staff members, and/or students) who plan and carry out their own inquiries (to improve their own practice or for other reasons) during the next 12-24 months?  (We'd do that by providing widely available training, support, and assessment tools. -- see below)
2. Top Down - priority questions: What are the most important questions facing the whole institution or system, or facing individual departments, offices, programs or services -- questions where our institution, its departments, and offices need devise useful studies during the next 12-24 months? (We'd do that by sponsoring and assisting such studies -- see below).

1. Increasing the number of people doing studies

1. What is our institution's current capacity to do such studies? One way to assess capacity to do assessments is to fill out the "...do transformative assessment" self-study materials of EDUCAUSE's READY program, developed for EDUCAUSE by Steve Ehrmann of the Flashlight Program; these materials are free.  You should also take a look at the rubric developed by Gary Brown, Steve Ehrmann, Vicki Suter and others as part of EDUCAUSE's program.
2. If our institution had access to Flashlight tools and support last year, did those pioneers know about those options (e.g., tools, option to use Flashlight consulting for free or at reduced rates)? did they use that help? do they have suggestions about how to make Flashlight help more useful?
3. What sort of help did our assessment pioneers use to get started in their inquiries?  to carry out the work? to make sure the results were influential?  What resources generally exist to stimulate people here to consider doing a study? to help them do such a study (e.g., data entry? research design? statistics and other analytic help? publicity?)
4. Can we use at least some of those successful studies to publicize the importance of doing studies and the role Flashlight can play?

2. What questions are crucial to answer this year?

If your institution has a Teaching, Learning, and Technology Roundtable, this is an ideal question for them to address in the spring (for the coming academic year) or the fall.  "Are there particular studies that could have a big payoff for the institution as a whole or for one or more of its programs, offices or services?" Here are just a few of the many possibilities:

• To help develop a self-study for accreditation of a program or the institution, should we be studying and thereby increasing the leverage technology gives us to improve the quality of instruction? (Click here to go to a TLT/Flashlight resource page on accreditation and program self-study.)
• How is our Course Management System being used instructionally? what does that suggest about how we need to change faculty development, support, incentives, etc.?  What is the impact so far on workload and other costs: are there ways we can work smarter and better? (Click here to go to a TLT/Flashlight resource page on using data to get more value from Course Management Systems.)
• Some people say a distance or distributed learning program should be of a quality that is comparable to what we offer on campus. Others suggest that the program can and should be better than what historically we could have offered in a campus-bound format. In either case, are we getting the kind of data we need to assess and improve the quality of that program? Are we monitoring its support services?
• One of the most common instructional applications of computing at most institutions is using PowerPoint or a comparable product to aid lectures.  Sometimes the way slideshows are used makes lectures better, but sometimes the method of use can make the lecture worse. Can we help faculty members get feedback so that they can use the technology more successfully? (Click here to go to a TLT/Flashlight resource page on using data to improve the outcomes of using presentation software.)
• Our program (institution) is making a major investment in technology access (e.g., making sure all students have computers). What kind of study could help us make sure that investment pays off in better learning?
• Historically our library has been an important support for our academic program.  The Web's importance has grown dramatically as a gateway to intellectual resources around the world.  But we're uncertain about whether print and electronic resources are being used to the fullest to support learning? Can we uncover data that could help improve the match between what we teach, how we teach, and how information supports learning? 
• Are there aspects of your institution's mission or character that you hope technology will enhance (e.g., one college has identified "collaboration and community" as an institutional priority). Are there ways of using data not just to document but to improve technology's leverage in advancing this facet of the institution's life and program?

This list represents some of the areas where we think Flashlight can be of special help, but the most important thing is to find a question or questions that really matter.

Once you identify an issue, you can plan a single centralized study or try to encourage and support others to do studies that all tackle the same question. For example, if the institution is about to increase student access to computers, one option is to help several departments each to do its own study designed to chart and to improve the usefulness of those machines for their instructional programs.  Fostering a related set of studies runs the risk of duplication but helps assure that each study focuses on what really matters to that department or office.

Once some questions (study topics) of this sort have been identified, what staff, budget, and Flashlight help will be needed to answer them?

II. Planning the Year

Keep in mind that it's usually important both to answer important questions for the institution and its program while also helping more people and units to frame and answer their own questions. 

Another tip: institutions are way ahead of the game if they have at least one person who considers it a part of his or her job to do studies and/or to help others do studies.

In each of these areas, who needs to be doing what during the next month? the next three months? after that? 

Is there time to influence budget allocations for this year? What about next year?

If your institution is a TLT/Flashlight subscriber, a consultant has been assigned to work with you this year.  Feel free to ask his or her help in developing and carrying out this plan. Network members, for example, get two free days of consulting help a year, plus the option to buy more time (from any of our consultants) at half the rate we normally charge. Comprehensive Program members get two hours of consulting and Basic subscribers get one hour.