1. Ideas and Concepts for using data to improve faculty development/support
2. Cases - Examples of useful studies of faculty development/support
3. Flashlight tools and services
   
   Return to Faculty Development/Support Resource Page

I. Ideas and Concepts for Using Data to Improve Faculty Development/Support

If you're going to evaluate a program that helps faculty use technology to improve teaching and learning, a few of the following ideas might help you design your study.

1.1 Are the desired outcomes the same for all faculty?

For faculty development and support programs usually have two faces, each of which requires its own approach to evaluation (using data to improve the program's effectiveness)

• Uniform impact: to some degree, all participating faculty are being helped for the same purpose with the same kinds of expected outcomes, e.g., satisfaction, skill at using a course management system. Evaluation begins by developing a way to assess the impact or value-added, e.g., how good are they at using the CMS? or how much better have they become as a result of the training?
• Unique uses: to some degree, each faculty member is being helped to do different things (e.g., some faculty are improving presentation skills while others are learning how to facilitate online discussion and still others are working on topics of their own choosing). Evaluation begins with case studies of individual faculty to discover how that individual has benefited from the support.  Only after each case has been documented does the evaluation look for patterns across subjects.

If the program's emphasis is split half and half between these two emphases, then the evaluation effort  ought to be split half and half as well.  [For more on these ideas and their implications for assessment of impact and evaluation of process, subscribers can consult the Flashlight Evaluation Handbook.]

1.2 What's important to study?

Issues likely to be pertinent to evaluation of a faculty development or support program depend on its goals. Some are relatively generic; they could be featured in the evaluation of almost any program:

• in what ways do changes in teaching reflect the use of technology to advance some or all of the seven principles of good practice (or ideas of comparable general importance in evaluating quality of instruction)?
• does the scale of the program fit the scale of the needs? or is the program providing service to only a small fraction of the faculty who want and need it?
• how effective is the faculty support program relative to other programs with comparable resources and comparable needs?

Other issues will be specific to the program's intent and context.

• If, for example, the program uses student technology assistants, part of the inquiry might well focus on whether students and faculty are learning in distinctively valuable ways because of their work together
• If the program is focusing on 'broadcasting' low threshold applications and activities to large numbers of faculty, part of the evaluation would focus on what fraction of the messages are reaching various members of the faculty, and the issues that affect whether each message is heard and acted upon.

II. Case Studies

We're collecting successful formative evaluations of programs that help faculty use technology in their teaching. We're especially interested in uses of data that guide efforts to improve facultyl development or support, and/or help it become more cost-effective.  If you know of a study that might be linked here, please e-mail ehrmann@tltgroup.org )

Instructional Design at Washington State is helping improve distance learning courses while cutting costs

Some institutions see "instructional design" as an expense that may need to be cut in times of tight budgets.  But Tom Henderson, Gary Brown, and Carrie Meyers found in a series of studies at Washington State University that up-front instructional design both improved teaching-learning practices in courses and also helped control development and delivery costs.  These findings are helping define policy and practice at WSU,  one of the founding institutions of the Flashlight Program. Click here to see what they discovered. 

Faculty-Librarian Partnerships Effective in Developing Information Literacy in Minnesota, Dakotas

Project JSTOR was a three-year grant initiative from 1999-2002, supporting 35 public and private colleges and universities in Minnesota, North Dakota and South Dakota. Its goals: strengthen digital library use and scholarly research, particularly through the acquisition and use of the JSTOR digital library collection. Through the program, 20 colleges and universities became participating JSTOR members, joining a network of 15 other member institutions in the region.  The Flashlight Program conducted an external evaluation which included extensive interviews and surveys of program participants.

Perhaps the most significant finding was the power of Faculty-Librarian Instructional Partnership (FLIP) grants. Small grants helped at least one faculty member and a librarian at an institution to team up and improve a course's ability to develop information literacy among students. The grants seemed to help advance a new working relationship between faculty and librarians, while triggering substantial institutional increases in the use of the JSTOR collection and other online resources. Click here for links to a fuller description of Project JSTOR and the full text of the evaluation. 

At SE Missouri State, Investment in Faculty Development is Paying Off in Enrollment

One role of faculty support at SE Missouri State is to prepare faculty to teach online. David Starrett and Michael Rodgers studied who was being served by their institution's online courses. The University's investment in helping faculty use technology had been justified in large part by the hope that the resulting courses would serve students across the University's service area, students not close to campus. Their data indicated that online courses were serving precisely these students. Click here to see a summary of their study, and an e-mail address to get more data.

III. Flashlight Tools and Services

Relevant Flashlight tools and resources that come with TLT Group subscriptions include Flashlight Online, the Flashlight Faculty Inventory, the Flashlight Evaluation Handbook, and the Flashlight Cost Analysis Handbook. We also design special purpose assessment tools, and offer subscribers the opportunity to peer review one other's instruments and study designs, and then publish them to other subscribers.

Flashlight consultants also do external evaluations of faculty development and support programs, and provide suggestions and help for internal evaluations. If you'd like to chat about the possibilities, please send e-mail to online@tltgroup.org. We're especially interested in working with systems, consortia and institutional associations; we'd help develop rubrics and surveys to that the benefits, challenges, methods and costs of each program could be anlyzed in relation to other programs.