Flashlight Case Study

Training and Supporting Faculty to Assess Instructional Uses of Technology

Ann Haffer
Professor; Faculty Assessment Coordinator
and
Linda Downing
Manager, User Services
University Computing and Communications Services
California State University, Sacramento

California State University, Sacramento (CSUS) is in the second year of using Flashlight to help faculty assess their course and program outcomes and improve instruction. 

Getting Started with Flashlight: After CSUS joined the Flashlight Network, two members of the Assessment Committee of the University's Teaching, Learning and Technology Roundtable went to a workshop conducted by the TLT Group where they learned the Triad approach to assessment (identifying not just a technology and a desired outcome but also a specific use of technology that is intended to produce that outcome).  The committee then prepared instruction to introduce the triad Triad strategy for focusing an evaluation, and the use of the Flashlight Current Student Inventory (CSI) to the faculty on our campus.  These workshops were designed to help faculty develop assessment plans, whether or not technology posed particular issues in their courses.

Workshops for Faculty: Six workshops were offered during the 1999/2000 academic year.  The workshops were offered in two sessions. 

The first session included an introduction to the Triad process and the CSI.  Faculty were asked to identify a problem they wanted to solve and to describe a desired outcome if the problem were “fixed.” 

During the second session, the assessment committee, on a one-to-one or one-to-two basis, helped faculty work through a “paper and pencil” process to define their own triads.  After completing the Triad process the committee helped each faculty member to use the CSI to develop questions that could detect whether this triad really described what was happening in their courses. See http://www.csus.edu/tltr/assessment/ for materials used in the workshops. 

Benefits from the Workshops: Long before it was time to create the surveys, during the paper and pencil phase of designing their studies, many faculty members realized that some instructional activity or technology they were using in their courses was not working or even well developed.  They did not go to Flashlight Online to design surveys.  Instead, they stopped designing the assessment and started redesigning those course elements that they discovered needed fixing.  We believe that their discoveries have contributed to the improvement of teaching. 

For example, one faculty member’s selected problem was that a number of his students were not doing homework assignments.  In exploring the learning activities that should have prepared the students to complete the complex engineering assignments, the professor identified a number of reasons why the students were not completing the assignments.  He decided that the instruction was not complete enough for many students to grasp the concepts.  His solution was to design online instruction to help students review lecture materials and work through the problems.  His students then began successfully completing assignments. This is just one of many similar anecdotal experiences.

We also found that some faculty started to work on the Triad process, but didn't finish their designs in the second workshop session, and finally dropped the effort.  To reduce the chances of dropouts from the process, we have decided to increase our follow-up to keep in touch with interested faculty.  Because we believe that it helps to have someone coach faculty throughout the process, an online approach to supporting faculty members is being developed with help from the TLT Group's Flashlight Program.  When it is completed, faculty members will then be able to get help when they are ready to proceed.

Many faculty on our campus have Flashlight accounts and use the CSI in a variety of ways to help determine what is and what is not working in their courses and programs.  Flashlight surveys have been constructed and used to assess class assignments, courses, and programs.  For example, in Fall 2000 Communication Studies and Engineering faculty are using Flashlight to assess courses, the Online Liberal Studies program is using it for the second semester to assess Web courses, and the Distance and Distributed Education program is using it to discover faculty and student perceptions of teaching and learning in the distance environments used at CSUS.