F-LIGHT

E-Newsletter for the Flashlight Program

• Evaluating technology-supported classrooms in order to plan better facilities
• The importance of student feedback in the middle of the course, not just at the end
• Upcoming Assessment-related Online Workshops and Conferences
• Ehrmann's Web Log ('Blog)  - Internet II video, portals, and redefining general education.
• TLT/Flashlight subscription options; subscribing institutions
• Learning More About Flashlight( free demo accounts), The TLT Group and F-LIGHT (including how to start or stop an F-LIGHT subscription)

Evaluating Technology-Intensive Classrooms

Steve Ehrmann, Director, The Flashlight Program 

Institutions are spending hundreds of millions of dollars to train people to use the technology in their classrooms, to upgrade those classrooms and their buildings, and to design new generations of learning spaces. 

Much of that investment is being made “blind”: without a searching evaluation of the educational strengths and weaknesses of the current spaces.  In brief, it's important to discover where current facilities are making it difficult (or comfortable) for faculty and students to do what's valuable.  This can be more challenging than you might think because most faculty have used only a few types of learning facilities, as students and later as instructors. So they may have trouble imagining what they might want to do, and come to value, in a very different kind of space.  My first job was at The Evergreen State College in the mid 1970s. Evergreen is organized around learning communities, but its space, back then was quite traditional.  So a learning community of 80-100 students might be squashed into an uncomfortably small classroom and have to use other classrooms in order to break into smaller groups. Only recently have academic buildings been designed to support Evergreen's distinctive program. A new building there, for example, has a large room, with smaller breakout rooms nearby, along with facilities that make it easier for students to work in teams on projects, and store their project materials between team meetings.

New technology is empowering, that is, it increases the options for faculty and institutions. So how do we imagine new kinds of spaces for doing new kinds of things, when old spaces have so restricted our imaginations?

Here are a few criteria you might use to evaluate current facilities as part of a process of brainstorming new facilities. These examples include activities that are easy in many classrooms, as well as activities that are (today) easy only in a very small number of classrooms:

• How easy is it for the instructor to periodically split a large class into small groups, and then bring those groups back together, several times an hour (are the chairs bolted to the floor? do some groups need to leave the room in order to hear?)

• How easy is it for those groups of students to work on projects that involve spreading books and papers out?

• How easy is it for the instructor, or a student, to show all students what is on one student's computer screen?

• How easy is it for an instructor to include guest lecturers via audio in a way that the guest can easily discuss issues orally with several dozen students?

• How easy it for the instructor or a student to call up photos of students in this class, to make it easier to match names and faces? How easy is it for the instructor to look up academic information about students during the class discussion (e.g., the last homework the student submitted)?

The TLT Group has been developing a taxonomy of activities that are instructionally important but problematic in at least some educational facilities, along with illustrations of the kinds of physical and virtual spaces that makes each activity easier, more comfortable, and/or less expensive. 

The taxonomy can be used in surveys and/or as grist for planning team discussion. The list of activities and examples, which continues to evolve, was developed for TLT Group subscribing institutions but is now also publicly available.  Subscribing institutions can also get free evaluation surveys and a planning guide. The TLT Group also offers consulting help for planning physical and virtual facilities.

Asking the Right Questions at the Right Time

Drew Smith, The University of South Florida

Assessment is a two-way street.  As instructors, we've known for a long time that our students benefit from timely feedback to their attempts at demonstrating what they have learned.  But if feedback should be timely, why do we so often wait until the end of a semester before we ask our students about the quality of our teaching? 

One major step toward improvement of student feedback is the use of mid-course evaluations.  The driving force for these must usually be self-motivated on the part of the instructor, because many institutions seem to care only about the standardized end-of-semester evaluations.  And end-of-semester evaluations won't help fix a problem for the existing semester.  Mid-course evaluations, on the other hand, make it possible to catch problems before they ruin an entire semester's work.  Another advantage of mid-course evaluations over end-of-semester evaluations is that the student is less likely to mentally associate the evaluation activity with the student's own expectation of his or her final grade. 

If having your students evaluate your teaching in mid-course is the right time, what are the right questions?  I discovered the answer to this myself through a process of trial and error.  The first trial was during the 2003 Fall semester, using Flashlight Online to survey 22 library science graduate students.  Of the 10 questions I asked, 5 were fairly open-ended, while 5 could be answered by "yes" or "no".  Therein was the problem.  Give someone the option of answering "yes" or "no", and they'll take you up on it.  "No" doesn't usually tell you what was really wrong, and "yes" doesn't really tell you what you were doing right.

In the 2004 Spring semester, I repeated the survey in a different graduate-level class, replacing the yes/no questions with more elaborate questions.  One quick example:  I replaced "Do you feel that the instructor is knowledgeable concerning the subject matter?" with "Which areas of the course do you feel that the instructor appears to be most knowledgeable about, and which areas least knowledgeable about?".  The answers were far more enlightening!

Lessons to be learned:

1.  Don't wait until end of semester to find out how you're doing.

2.  Yes/no questions have little feedback value for purposes of improving your teaching.

3.  You won't get your survey instrument "perfect" the first time you use it.

Drew Smith

Instructor, School of Library and Information Science

College of Arts and Sciences, University of South Florida, Tampa

drewsmithusf@aol.com

Upcoming Assessment-Related Online Workshops and Conferences

Online workshops on Assessment and Dist. Learning

The TLT Group is considering whether to offer a series of online workshops for faculty and administrators. The general theme: how to use surveys and other data to improve distance/distributed courses. One would be aimed at individual courses, the others at program improvement. Two would describe a variety of models for gathering and using data to pinpoint how to improve learning, a third will focus on our benchmarking program in nursing, and the fourth will deal with cost analysis.

Want to get an e-mail if we do offer them? Equally important, do you want to give us advice on the timing, format and whether we should arrange for academic credit? If you'd like to learn more, please click here and then tell us what you think; responding will also allow you to get on the mailing/waiting list.

We've decided to offer our first online workshop on cost analysis, starting in October 2004.  Please fill our the survey if you'd like to hear about it, or if you'd like us to offer other kinds of online workshops.

Flashlight Online training - Subscribers Only!

We'll continue to webcast periodic training sessions for Flashlight Online users, administrators, and trainers. The next online training session is scheduled for July 12.   Click here for more information.   If you're not sure if your institution is a current subscriber, click here.

For details on this and other Flashlight and TLT Group events, both face to face and online, keep an eye on The TLT Group calendar.

TLT/Flashlight Subscription Programs and Subscribers

• The TLT/Flashlight Basic Collection includes site licenses to dozens of program development and assessment tools and resources, discounts to online workshops, and one hour of free consulting, while the 
• The Comprehensive Collection includes all that, plus unlimited institutional use of Flashlight Online, our powerful, web-based survey system that includes validated items and peer-reviewed templates for typical educational studies; Comprehensive subscribers get two hours of consulting. 
• Network membership includes all the Comprehensive benefits plus two free days of consulting/training and sharply reduced rates for additional days: the consulting can be used to aid planning, do external evaluations, train your staff, take part in projects, and a variety of other purposes. 

Benefits are added almost weekly. This web page links to recent notices we've sent to subscribers about updates and additions.

Over 130 institutions, systems, boards of regents, and multi-institution projects around the world now subscribe to these TLT/Flashlight services.  Their subscriptions make it possible for us ot create and distribute F-LIGHT and we thank them for that. Is your institution one of them? Check our list of participating institutions. Institutions subscribing, or resubscribing, since March 1 are:

• Auburn University

• Bethel College, Minnesota

• Brigham Young University

• Bucks County Community College

• Butler University College of Pharmacy

• California Lutheran University

• California State University - Fullerton

• California State University - Monterey Bay

• California State University - Northridge

• California State University - Sacramento

• Clark Atlanta University

• Cochise College

• COHERE (Canada)

• Colby-Sawyer College

• Dalhousie University

• Earlham College

• Embry-Riddle Aeronautical University

• Evergreen Valley College

• Fairfield University

• Florida International University

• Fox Valley Technical College

• George Washington University

• Gettysburg College

• Health Canada/Centre of Surveillance Coordination

• Hibbing Community College

• Houston Community College

• Howard University

• Husson College

• Iowa College Foundation

• Lewis University

• Linn-Benton Community College

• Loras College

• Louisiana Board of Regents

• Louisiana State University Baton Rouge

• Maryville College

• Miami-Dade College

• Middlesex County College

• Morton College

• Mount Royal College

• Niagara College of Applied Arts & Technology

• Nicolet Area Technical College

• Northwest Nazarene University

• Olivet Nazarene University

• Pace University

• Seton Hall University

• South Dakota State University

• Southern New Hampshire University

• Southern State Community College

• Stanford University

• SUNY New Paltz

• SUNY Stony Brook

• Tulane University

• University of Arizona, College of Nursing

• University of Central Florida

• University of Detroit Mercy

• University of District of Columbia

• University of Florida

• University of Hawaii - Manoa

• University of Kansas Medical Center

• University of Maryland University College

• University of Massachusetts - Amherst

• University of Massachusetts - Dartmouth

• University of Medicine & Dentistry of New Jersey

• University of Michigan - Ann Arbor

• University of Minnesota - Crookston

• University of Missouri - St. Louis

• University of North Carolina - Charlotte

• University of North Carolina-TLT Collaborative@Chapel Hill

• University of Notre Dame

• University of South Florida

• University of Tennessee - Knoxville

• University of Texas - Austin

• University of Vermont

• University System of Georgia Board of Regents

• Valencia Community College

• Vanderbilt University, School of Nursing

• Washington & Lee University

• Washington State University

• Winston-Salem State University

• Wright State University

Top of Page

Ehrmann's Web Log ('Blog)

Since my last log entry, I've been traveling quite a bit and working on more projects than I have space to talk about here. 

• The evaluation work on the use of Holocaust video has been fascinating for example; the technical obstacles of making such a large video archive available even to a few institutions over Internet II are formidable, but the faculty I've interviewed so far have engaged their students and altered their courses in a way that surprised even them. 

• The evaluation of the CNAV portal at Gettysburg College, also still underway, has given me quite a few new insights, including the importance of making it easier for students, faculty and staff to be able to see collections of student photographs, in order to identify students by face, not just by name.

• Our work with EDUCAUSE's National Learning Infrastructure (NLII) on their September workshop on Learning Space Design (don't even bother; it filled, and the waiting list filled, almost immediately; I'm sure we'll do another one, however) helped keep me moving forward on how to evaluate classrooms as well as virtual learning spaces (see above).

The topic where the stars really seem to have come together, however, is the relevance of technology to the reform of general education. I've been an admirer for some time of the work of the Association of American Colleges and Universities' Greater Expectations Project. I especially agree that general education should be defined as the desired, common outcomes of education for all students at an institution, and that these outcomes are extremely unlikely to be achieved by "Gen Ed" courses alone.

So, today, what kinds of communications skills should all graduates of your institution have? Should they be good at constructing academic arguments and resources that use different kinds of data (e.g., video? databases?) and have several points of entry and several narrative lines? If so, and students are creating web projects, who should be able to see those projects? who should give feedback? and how might those new audiences and sources of assessment change the motivation for learning?

That series of questions represents the knot I'm trying to untangle: how we should rethink the ends and means of general education in a technological era. We're working on this issue with the University of South Florida, and I'm also writing an article on this subject for the fall issue of AAC&U's magazine, Liberal Education.

Want to help? Drop me an e-mail at ehrmann@tltgroup.org

- Steve Ehrmann

The Flashlight Program for the Study and Improvement of Educational Uses of Technology is part of the non-profit TLT Group, Inc. Flashlight was created by Annenberg/CPB in 1993. The TLT Group is headquartered in Takoma Park, Maryland, just outside Washington DC, with additional staff in Texas, Richmond VA, and Pittsburgh; and Senior Associates around the world.

Our thanks to Washington State University for their many ways of supporting Flashlight, including developing and administering Flashlight Online and providing the listproc for distribution of F-LIGHT notices.  We are also grateful to St. Edward's University for extensive support for Flashlight; to the corporate sponsors of The TLT Group; and to funders whose dedication to higher education has aided the TLT Group's work, including Annenberg/CPB,  the birthplace of the Flashlight Program.

If your institution needs to get a better look at Flashlight Online, the best way is for someone at your institution to request a temporary, free demonstration account.  Send e-mail to Flashlight@tltgroup.org with the header "Free Demo Account" to ask for details. One account per institution, please.

If you know someone else who would like to be alerted to new issues of F-LIGHT, please suggest that they send e-mail to LISTPROC @ LISTPROC.WSU.EDU  with the one line message
   SUBSCRIBE F-LIGHT (the subscriber's first and last name)
Make sure that your e-mail is set to send plain text, not html or RTF.

Do the same for yourself if you have changed e-mail addresses.

To stop receiving the bulletin about F-LIGHT, please send e-mail to LISTPROC @ LISTPROC.WSU.EDU with the one line message
   SIGNOFF F-LIGHT
There should be no other text in the message (e.g., no signature file) and no subject header. If there are problems signing off, make sure your e-mail is set to send plain text, not html or RTF.

Top of Page