The Flashlight Program and Mount Royal College |
Robin
Etter Zúñiga, Flashlight Program of the TLT Group
Patricia
Derbyshire, Mount Royal College
(a
slightly different version of this article was published
in THE Journal)
Everything we do inside and outside the academy has been affected by this change. It shouldn’t be a surprise, then, that the faculty we speak with almost always have a story to share about how technology has changed the profession and their lives. Faculty simultaneously praise and scorn the World Wide Web for expanding student access to information resources. They are excited about the potential for students using multimedia for creative work, and frightened that their students are focusing on the visual to the near exclusion of other types of seeing, learning and knowing. We have a very strong sense that something significant is changing around us but, at the moment, most of us are still gesturing at shadows in the dark. How is the cyber revolution changing teaching and learning? How can we shape and mold this revolution so that it promotes the best practices in teaching and learning and avoid as many pitfalls as possible?
The Flashlight Program was started in 1994 to help institutions answer these questions. With a planning grant from the Fund for the Improvement of Postsecondary Education (FIPSE) of the United States Department of Education and additional funding from the Annenberg/CPB Project at the Corporation for Public Broadcasting, we set out to develop a set of tools educational institutions could use to evaluate the effectiveness of using technologies in the teaching and learning process. The program was born out of the realization that institutions are using technologies in many different ways, but have very similar questions about the way in which these technologies are affecting teaching and learning. It is also premised upon the realization that it is what we do with technology (using technology to improve teaching and learning) that really matters. In the course of our investigations, with special thanks to Trudy Banta from IUPUI, we realized that many of the questions that were being raised were easily linked to questions about whether teaching and learning were effective in terms of their ability to promote the Seven Principles for Good Practice in Undergraduate Education (Chickering and Gamson, 1987), and to what extent different types of technological tools (computers, the Internet) were promoting (or inhibiting) these practices.
Out of these efforts the first Flashlight “Tool Kit” (the Current Student Inventory) for evaluating how students are being affected by technology in learning, was born. This tool kit of around 500 items, now in a web-based format, also includes an interview/focus group protocol and an evaluation handbook with case studies. It was followed by a handbook on conducting cost analyses of the use of technology in instruction and will soon be followed by a Faculty Inventory, and somewhat farther into the future by an Online Student Services Inventory and an Information Literacy Inventory.
Numerous Flashlight inspired studies have been conducted at institutions throughout the United States and across the world. One such study was recently undertaken at the Mount Royal College in Alberta, Canada.
In September 1997, Mount Royal College began the first in an on-going series of studies examining its model for converting the entire institution into an effective user of learning technologies. The series would eventually merge into an extensive case study of initiatives outlined in the College’s Technology Integration Plan.
It was evident that the exploratory studies at Mount Royal had to take place at many sites, requiring different approaches in their execution and documentation. It was also evident that these studies were about much more than the technology interventions themselves. Situated within any technology innovation are nuances of how instruction must shift, how students learn differently, and how faculty learn to incorporate new pedagogical knowledge into a curriculum designed for a new medium.
Mount Royal’s administration and staff had been reviewing and discussing the integration of ideas presented in the AAHE bulletin the “Seven Principles for Good Practice in Undergraduate Education” (Chickering and Gamson, 1987) as part of this process. The Flashlight Program and “tool kits” were developed with an eye to this contemporary scholarship on teaching and learning; thus Flashlight provided a common language to carry out the Mount Royal studies. Using the Flashlight methods and tools, the persons involved in the evaluation process at Mount Royal were able not only to discuss the depth and breadth of the instructional technology, but also the depth and breadth of teaching, learning and faculty development as it emerged. This continues to be an important factor as they continue research and evaluation beyond the initial work of the Technology Integration in Teaching & Learning Environments (TITLE) project and as they examine the implications for administrative planning and policy.
Flashlight mirrors two emerging evaluation paradigms -- transformative and utilization-focused evaluation. Transformative evaluation (Mertens, 1997; Chelemsky, 1998) suggests that an evaluator be deliberate in balancing the organizational perspective with the beneficiaries, namely the departments, faculty and students, for whom the programs and activities are offered. A transformative approach asserts that the inclusion of the beneficiaries ensures critical new knowledge and that beneficiaries know more from personal experience about the quality and inadequacies of a program than anyone else, therefore learning what they know greatly improves the sensitivity of the evaluation.
Utilization-focused evaluation (Patton, 1997) supports many of the approaches that transformative evaluation embodies but additionally, urges the evaluator to consider the consequence of the design of a study from the outset as a function of how likely it will to be used for program improvement when a project is complete.
Flashlight offers the means to gather evidence for accountability and improvement. More importantly, it allows an institution the capacity to examine and acknowledge the resources that go into the program, the processes and learning activities in the development and execution of a new instructional innovation, and the outcomes of any given instructional technology from multiple perspectives.
The last significant factor in Mount Royal’s decision to choose Flashlight as a resource was that the Flashlight Current Student Inventory (CSI) questions were developed in the field. There was broad representation in the CSI development functionally (administrators, faculty, students, technical developers), and differing institutional profiles (two- and four-year colleges, universities, and distance learning programs). Items from the CSI had been tested with over 2000 students for validity. Case studies, profiles, expert reviews, and development of a large network of Flashlight member institutions are all critical pieces of the Flashlight methodology. For these reasons, Mount Royal decided that involvement in the Flashlight Project would provide high utility and a solid cost-benefit for the institution.
The TITLE project examined three teaching and learning situations: a CD-ROM resource, which augmented classroom-based teaching and learning, and two distance courses, which are offered to students on‑line. The TITLE project piloted the Flashlight CSI in five sections of Introductory English Composition with close to two hundred students who were using an on-line grammar resource, the word processor, and e-mail for peer editing.
Across these studies they began to see evidence-based practice of communications which ensured access and exchange between instructor and student and from student to student - allowing for not only rich and rapid feedback, but also the collective construction of new knowledge and inquiry. This in turn allowed for learning flexibility and heightened time-on-task, both correlates of effective learning.
Students were also providing valuable feedback on their unique uses of instructional technology in learning. They described what they saw as opportunities to extend and actively utilize new knowledge, approximate what they would experience in the workforce after graduation, and develop the ability to build upon and enhance the theoretical concepts that had been taught in a class lecture or provided for them in textbook form.
The
project also initially examined faculty integration and support
initiatives. The Learning Technology Integration Plan for faculty members
was the first generation program that provided hardware, software and
training to faculty members. The MRC Teaching and Learning Roundtable
focused on promoting communication and functioning to establish events,
showcases, and venues for discussion within the College community.
These studies quickly confirmed what worked and what created barriers to faculty integration of new technologies. Much of this information was known in an ad hoc manner throughout the College, however the TITLE project provided a forum to collect critical and constructive feedback for program improvement. Faculty members were able, from their frame of reference, to talk about their needs, the pace of the programs and workload issues.
Most recently, the College has offered a faculty development initiative entitled the PanCanadian Educational Technology Awards Program. Led by Norm Vaughan, the principal objective of the program is to provide directed opportunities for faculty to learn how to integrate learning technologies into their teaching practices. This program has skillfully utilized and incorporated findings from the TITLE project which transcend obvious issues of instructional technology integration.
The PanCanadian Educational Technology Awards Program addresses training needs that are responsive to faculty, with a concise understanding of the context for change within the institution and the strategic initiatives that support the shift. The program clearly acknowledges and situates available faculty resources - human, pedagogical, technical and assessment related for participants. The program provides moderate yet significant release time, financial incentives, and recognition of project work. Faculty indicate that the most gratifying and rewarding hallmark of the program was the recognition of the spirit and potential of an academic community which fosters inquiry, co-discovery, multiple support systems, and relevance of collegial experience.
Described as a coach, a facilitator, and an abundant resource, Norm Vaughan has also raised the bar on the program by building in a succession planning model called “Project Shadow and Mentor.” In this project the faculty-developer program itself becomes the site where technology integration can expand; with faculty champions providing orientation, critical direction and support to incoming faculty.
Flashlight continues to be a resource that helps Mount Royal College sustain a critical focus on what learning is, how we describe it, and what the essence of the teaching and learning activity is within each individual instructional technology project. Despite the variety and range of projects, there is a growing confidence in the ability of both the technology and the College to deliver on a model of enhanced learning for students. The college community shares a common knowledge and an expanding vocabulary with which to make meaningful observations about content and concepts in a powerful synthesis of scholarship and technology.
Over 200 educational institutions have joined Mount Royal College in their use of the Flashlight methods and tools. Like Mount Royal they have found that the teaching and learning with technology is a process of discovery. Numerous studies have been conducted on ways in which information technologies have affected the adoption of best practices in instruction. Washington State University has studied ways in which generative uses of the World Wide Web by students have improved educational performance. Jones International University designed an end of course evaluation using Flashlight questions. Indiana University Purdue University Indianapolis compared the costs and teaching and learning methods in composition courses in computer classrooms versus those in traditional face-to-face classrooms. A group of Nursing Schools is investigating ways in which online courses are facilitating (or hindering) teaching and learning practices, such as collaborative learning.
The Flashlight method and tools can be used in a variety of ways and for numerous reasons. An educational institution may want to use Flashlight to:
· guide improvement of courses and courses of study (e.g., majors, minors, freshman year skills development, writing across the curriculum) and strengthen the roles played by technology in such efforts;
· evaluate major grant-funded projects;
· improve technology-based services (e.g., libraries, computing services, telecommunications and Internet connectivity) and their leverage in educational improvement;
· support strategic thinking about the curriculum and technology services;
· prepare for accreditation;
· help faculty, departments or institutions compare their uses of technology and outcomes; and
· redesign student evaluations of faculty.
To find out more about the Flashlight Program and read about other studies that have been conducted using the Flashlight methods and tools visit www.tltgroup.org/programs/flashlight.html. You can also contact Patti Derbyshire about the Mount Royal study. The Flashlight Program is a program of The TLT Group, a non-profit that currently supports Mount Royal and about 150 other institutions around the world. For more information about the Flashlight Program or The TLT Group, please contact Robin Zúñiga.
learn about tltg || events & registration || programs || resources || listserv & forums || corporate sponsors || related links || home
