E-Newsletter for the Flashlight Program
For the Study and Improvement of
Educational Uses of Technology
SUMMARY OF THIS ISSUE
In this issue of F-LIGHT, the free Flashlight newsletter:
Goal#1 of F-LIGHT: Gather
and share examples of studies and of other evaluation-related activity
that are making a difference in their institutions. If you've
done something of this sort, whether or not you used Flashlight tools or
methods, we'd like to hear about it and have the opportunity to report on
E-mail is wonderful: please send the URL of this issue to everyone who needs this
information! For information about starting or ending a subscription, sending us
announcements, etc., see the bottom of this message.
institutions foster specific educational improvements not only through
traditional methods (e.g., faculty development, technology support,
budgets, mission statements, course evaluation procedures, ..) but also
through the use of a variety of studies (e.g., by faculty of their own
courses; by the institution of students, services, materials, and
Some of the possibilities of transformative assessment are
suggested by interviews with leaders at three institutions, the
University of Central Florida, Mount Royal College (Calgary, Canada),
and Washington State University.
interviews suggest several lessons about the transformative use of
assessment: 1) "transformation" may be a misnomer -- to the
people involved the issue is educational improvement and usually a
relatively specific improvement, 2) because technology is a tool, the
improvement theme itself may not even be explicitly technology-related, 3)
the assessment effort gains power when it is integrally related to other
means of advancing the improvement rather than being seen as an add-on, 4)
this integral relationship is the fruit of both top-down and bottom-up
efforts at change.
article was done as part of a collaborative project on transformative
assessment involving Flashlight, Educause, and the Coalition for Networked
the full article, click
There are plenty of great techniques for increasing response rates (and
the attached article cites a good book on the subject). Here I'd
like to focus on creating the justified expectation that responding
thoughtfully to a survey will really be worth the student's time. I've
come up with a dozen proposed principles of good practice, sorted into
- Creating a culture of inquiry at an institution so that students
have learned to expect that the next study they see will probably be
worth the time and thought it takes to respond;
- This next study has been designed to be produce findings and action
that are likely to justify the respondents' time; that doesn't imply
studies with preordained findings.
- The nature and value of the study is well explained to respondents.
For a description of these principles of good practice, click
Asking the Right Questions
At the recent TLT Group Summer Institute, held as part of Syllabus
2001 in Santa Clara, one of the five Flashlight sessions was a
leadership workshop. I got a bit of a surprise when I encouraged
participants to imagine the debate about whether to put time and money
into evaluation back at their institutions.
I started with a blank two-column chart on the screen - arguments
for, arguments again. I assumed we'd start with the
"pro" arguments and then see if there were
counter-arguments. My mistake! The first participant quickly
listed a dozen arguments against institutional efforts to study
educational uses of technology. As a group we added a few more and then
started to work on responses to these 'frequently made
objections.' I've summarized our
work on the this web page. If you have suggestions for how to extend
or alter this FMO, please let me know!
a generous grant from WebCT, Flashlight
is about to accelerate
its development of a study package to help institutions assess and improve
their use of course management systems. The study package will be
useful for all such products, commercial and home-grown. Initial
work will focus on a study package that can be used to improve faculty
development, course development services, and technology support.
continue to collect studies
that colleges, universities, corporations, schools and others may have done of
their uses of such Course Management Systems (CMSs),
especially studies at the departmental or institutional levels.
We've begun to assemble
and annotate the studies we've found. Not many so far: we need your
Here are the kinds of studies
and instruments in which we are most interested:
Studies and instruments
designed to document how an institution's system has been used, how it
hasn't been used, what kinds of support and training were successful or
unsuccessful, and what factors affect the system's use for different
focus groups, data collection by the WCMS's themselves, and other means of
Studies that produced
findings that were seen as useful by the institution (e.g., helped to
confirm or alter arrangements for training or support; helped confirm or
alter decisions about what WCMS to use).
that helped document whether the system was of educational benefit to
students, departments or the institution as a whole.
Studies of the obvious and
hidden costs (including time) of maintaining such systems.
If you have done such a study
could you please send us a copy or a URL? We'd
also like to talk with you about whether or how you might improve the study
If you know of such a study,
could you tell us how we can find it?
Please send your information
and suggestions to Helen Parke or Steve
The TLT Group also has other
resources helpful in choosing a Web Course Management System.
Institutions joining the
Flashlight Network recently include: Vanderbilt University, Washington and
Lee University, and Pomona College. What's
also exciting is that, thanks to generous support from the Ameritech/SBC
Faculty Development Technology Program, 94
institutions were subscribed to the Flashlight Tool Series! That's
a record for one consortium or system subscribing all at once and we gave them
an even more generous discount than usual.
For an almost-current list of the
approximately 350 institutions and projects around the world that are
Network members, Tool Series subscribers, or licensees of Flashlight
please visit our list
of participating institutions.
a Question about Educational Uses of Technology?
Sometimes you just need a bit of help - a contact, an idea, a reaction. We try to be as
helpful as we can, so drop us an e-mail and
let us know what's on your mind.
(including free demonstration accounts), the TLT Group, and F-LIGHT
Program for the Study and Improvement of Educational Uses of Technology
is part of the non-profit TLT Group, Inc., an affiliate of the American
Association for Higher Education.
your institution needs to get a better look at the Flashlight Current
Student Inventory, or at Flashlight Online (the Web-based system that lets
you use the CSI, among other utilities), the best way is for someone at
your institution to request a temporary, free demonstration account.
Send e-mail to Flashlight@tltgroup.org
with the header "Free Demo Account" to ask for details. One
account per institution, please.
The TLT Group publishes F-LIGHT every few weeks. You can see the name of the
author-editor at the bottom of this message; please feel free to send me mail about issues
of evaluation or research on teaching, learning and technology. Recent issues are posted
on our Web site.
Our thanks to Washington State University for their many ways of supporting
Flashlight, including providing the listproc for distribution of F-LIGHT.
We are also grateful to St. Edward's University and the Rochester Institute
of Technology for extensive support for Flashlight; to the founding
corporate sponsors of the TLT Group (Applied Theory, Blackboard, Compaq Computer
Corporation, Microsoft Corporation, the SCT Corporation, Student Online, and
WebCT); the TLT Group's other corporate sponsors; key public sector funders
of the TLT Group's work such as the Annenberg/CPB
Projects, the Andrew W. Mellon Foundation, the Fund for the Improvement
of Postsecondary Education (FIPSE), and the National Science Foundation.
If you know someone else who would like to be alerted to new issues of
F-LIGHT, please suggest
that they send e-mail to LISTPROC@LISTPROC.WSU.EDU with the one line message SUBSCRIBE F-LIGHT (the subscriber's first and last name)
To stop receiving the bulletin about F-LIGHT, please send e-mail to LISTPROC@LISTPROC.WSU.EDU with
the one line message SIGNOFF F-LIGHT
Stephen C. Ehrmann, Ph.D.
Director of the Flashlight Program and
Headquarters office hours: 10AM to 6PM Eastern
One Columbia Avenue, Takoma Park, Maryland 20912 USA
phone (301) 270-8312 fax: (301)270-8110