 |
|||||
     |
|||||
  |
F-LIGHT
E-Newsletter for the Flashlight Program
For the Study and Improvement of
Educational Uses of Technology
September 2001
SUMMARY OF THIS ISSUEIn this issue of F-LIGHT, the free Flashlight newsletter:
- Transformative Assessment: Three Case Studies
- Increasing Student Response Rates by Making Studies More Valuable To Them: A Dozen Principles of Good Practice
- Frequently-Made Objections (FMOs) to Evaluation and Some Responses
- WebCT starts the ball rolling: Selecting, Using, and Studying Web Course Management Systems
- New Flashlight subscribers - Total now over 200 inst'ns!
- Have a Question about Educational Uses of Technology?
- Learning More
About Flashlight( free demo accounts), The TLT Group and F-LIGHT
Goal#1 of F-LIGHT: Gather and share examples of studies and of other evaluation-related activity that are making a difference in their institutions. If you've done something of this sort, whether or not you used Flashlight tools or methods, we'd like to hear about it and have the opportunity to report on it here.
E-mail is wonderful: please send the URL of this issue to everyone who needs this
information! For information about starting or ending a subscription, sending us
announcements, etc., see the bottom of this message.
Transformative Assessment: Three Cases
Some institutions foster specific educational improvements not only through traditional methods (e.g., faculty development, technology support, budgets, mission statements, course evaluation procedures, ..) but also through the use of a variety of studies (e.g., by faculty of their own courses; by the institution of students, services, materials, and programs). Some of the possibilities of transformative assessment are suggested by interviews with leaders at three institutions, the University of Central Florida, Mount Royal College (Calgary, Canada), and Washington State University.The interviews suggest several lessons about the transformative use of assessment: 1) "transformation" may be a misnomer -- to the people involved the issue is educational improvement and usually a relatively specific improvement, 2) because technology is a tool, the improvement theme itself may not even be explicitly technology-related, 3) the assessment effort gains power when it is integrally related to other means of advancing the improvement rather than being seen as an add-on, 4) this integral relationship is the fruit of both top-down and bottom-up efforts at change.
This article was done as part of a collaborative project on transformative assessment involving Flashlight, Educause, and the Coalition for Networked Information.
For the full article, click here.
Increasing Student Response by Making Surveys More Valuable to Them: A Dozen Principles of Good Practice
There are plenty of great techniques for increasing response rates (and the attached article cites a good book on the subject). Here I'd like to focus on creating the justified expectation that responding thoughtfully to a survey will really be worth the student's time. I've come up with a dozen proposed principles of good practice, sorted into three groups:
- Creating a culture of inquiry at an institution so that students have learned to expect that the next study they see will probably be worth the time and thought it takes to respond;
- This next study has been designed to be produce findings and action that are likely to justify the respondents' time; that doesn't imply studies with preordained findings.
- The nature and value of the study is well explained to respondents.
For a description of these principles of good practice, click here.
Asking the Right Questions
Frequently-Made Objections (FMOs) to Evaluation and Some Responses
At the recent TLT Group Summer Institute, held as part of Syllabus 2001 in Santa Clara, one of the five Flashlight sessions was a leadership workshop. I got a bit of a surprise when I encouraged participants to imagine the debate about whether to put time and money into evaluation back at their institutions.
I started with a blank two-column chart on the screen - arguments for, arguments again. I assumed we'd start with the "pro" arguments and then see if there were counter-arguments. My mistake! The first participant quickly listed a dozen arguments against institutional efforts to study educational uses of technology. As a group we added a few more and then started to work on responses to these 'frequently made objections.' I've summarized our work on the this web page. If you have suggestions for how to extend or alter this FMO, please let me know!
WebCT Funds Flashlight to Develop Evaluation Tools to Assess and Improve the Use of All Course Management Systems: Case Studies Sought
Thanks to a generous grant from WebCT, Flashlight is about to accelerate its development of a study package to help institutions assess and improve their use of course management systems. The study package will be useful for all such products, commercial and home-grown. Initial work will focus on a study package that can be used to improve faculty development, course development services, and technology support.
We continue to collect studies that colleges, universities, corporations, schools and others may have done of their uses of such Course Management Systems (CMSs), especially studies at the departmental or institutional levels. We've begun to assemble and annotate the studies we've found. Not many so far: we need your help!
Here are the kinds of studies and instruments in which we are most interested:
-
Studies and instruments designed to document how an institution's system has been used, how it hasn't been used, what kinds of support and training were successful or unsuccessful, and what factors affect the system's use for different purposes.
-
Surveys, focus groups, data collection by the WCMS's themselves, and other means of investigation.
-
Studies that produced findings that were seen as useful by the institution (e.g., helped to confirm or alter arrangements for training or support; helped confirm or alter decisions about what WCMS to use).
-
Studies that helped document whether the system was of educational benefit to students, departments or the institution as a whole.
-
Studies of the obvious and hidden costs (including time) of maintaining such systems.
If you have done such a study could you please send us a copy or a URL? We'd also like to talk with you about whether or how you might improve the study design. If you know of such a study, could you tell us how we can find it?
P
lease send your information and suggestions to Helen Parke or Steve Ehrmann.P.S. The TLT Group also has other resources helpful in choosing a Web Course Management System.
New Flashlight Subscribers - A Record!
Institutions joining the Flashlight Network recently include: Vanderbilt University, Washington and Lee University, and Pomona College. What's also exciting is that, thanks to generous support from the Ameritech/SBC Faculty Development Technology Program, 94 institutions were subscribed to the Flashlight Tool Series! That's a record for one consortium or system subscribing all at once and we gave them an even more generous discount than usual.
For an almost-current list of the approximately 350 institutions and projects around the world that are Network members, Tool Series subscribers, or licensees of Flashlight tools, please visit our list of participating institutions.
Have a Question about Educational Uses of Technology?
Sometimes you just need a bit of help - a contact, an idea, a reaction. We try to be as helpful as we can, so drop us an e-mail and let us know what's on your mind.
About Flashlight (including free demonstration accounts), the TLT Group, and F-LIGHT
The Flashlight Program for the Study and Improvement of Educational Uses of Technology is part of the non-profit TLT Group, Inc., an affiliate of the American Association for Higher Education.
If your institution needs to get a better look at the Flashlight Current Student Inventory, or at Flashlight Online (the Web-based system that lets you use the CSI, among other utilities), the best way is for someone at your institution to request a temporary, free demonstration account. Send e-mail to Flashlight@tltgroup.org with the header "Free Demo Account" to ask for details. One account per institution, please.
The TLT Group publishes F-LIGHT every few weeks. You can see the name of the author-editor at the bottom of this message; please feel free to send me mail about issues of evaluation or research on teaching, learning and technology. Recent issues are posted on our Web site.
Our thanks to Washington State University for their many ways of supporting Flashlight, including providing the listproc for distribution of F-LIGHT. We are also grateful to St. Edward's University and the Rochester Institute of Technology for extensive support for Flashlight; to the founding corporate sponsors of the TLT Group (Applied Theory, Blackboard, Compaq Computer Corporation, Microsoft Corporation, the SCT Corporation, Student Online, and WebCT); the TLT Group's other corporate sponsors; key public sector funders of the TLT Group's work such as the Annenberg/CPB Projects, the Andrew W. Mellon Foundation, the Fund for the Improvement of Postsecondary Education (FIPSE), and the National Science Foundation.
If you know someone else who would like to be alerted to new issues of
F-LIGHT, please suggest
that they send e-mail to LISTPROC@LISTPROC.WSU.EDU with the one line message
To stop receiving the bulletin about F-LIGHT, please send e-mail to LISTPROC@LISTPROC.WSU.EDU with
the one line message
Stephen C. Ehrmann, Ph.D.
Director of the Flashlight Program and
Editor, F-LIGHT
TLT
Group
Headquarters office hours: 10AM to 6PM Eastern
Directions to:
One Columbia Avenue, Takoma Park, Maryland 20912 USA
phone (301) 270-8312 fax: (301)270-8110
e-mail: online@tltgroup.org