|
Return to Flashlight Evaluation Handbook list of
ideas for grant proposals and dissertations
In 1981 Carol Schneider (now president of
AAC&U), George Klemp,
and Susan Kastendiek did a study of exemplary college
teaching. What skills distinguished the superlative
faculty? This page summarizes the method they used. Someone
ought to apply these methods to identify skills of
superlative online teaching (the results might be similar).
A) Identify a pool of faculty who are widely
regarded as 'walk on water' teachers and another pool, whose
teaching is equally known, who are not regarded as stars.
The authors polled faculty, students and staff, asking each
such "reviewer" to examine a list of faculty in their
program, putting a single check beside individuals with
whose teaching they were familiar, and a second check if the
reviewer believed that particular instructor to be a
superlative teacher. The star group consisted of instructors
who were, among those who knew their work, regardless almost
universally as superlative. The controls were those
instructors whom virtually no one judged to be superlative.
B) Ask each individual in the two pools if
he or she is willing to be interviewed as part of a study of
excellence in teaching. (None of the instructors are told,
now or ever, which pool he or she is in.)
C) Ask each interviewee to tell three
personal stories of success in teaching and three stories
that ended in failure or frustration. The interviewers are
trained on how to help the interviewee tell stories that
revealed their thought processes as the critical incident
unfolds, from beginning to end.
D) Randomly choose half the "star"
interviews and half the "control" interviews and set them
aside. Closely study the remaining interviews, and use them
to create a competence model that distinguishes the "stars"
from the average and below-average performers: what does
each do and think about in comparable situations?
E) Create a code book with the kinds of things
interviewees say if they are using that competence. If, for
example, a competence was "belief that all students can
learn the material," what kinds of things might a person say
that would indicate that belief is shaping their actions?
F) Bring in a new set of people and train them to use the
codebook. Then have them use the code book to code the
interviews that no one has yet examined. Drop the
competences that don’t distinguish between the stars and the
controls in these interviews.
[The resulting 1981 report was called "The Balancing Act"
<http://bit.ly/Balancing-Act> and provided a fascinating picture of how faculty think. The
differences between stars and controls were dramatic. The
stars believed all their students were capable of mastering
the course and they were continually trying to figure out
how to make that happen, right up to the last day of the
course. The controls assumed the opposite - that many, even
most students, were simply not capable of succeeding in the
course. So their behavior was quite different, too.]
This approach, which has been used many times to develop
competence models, has promise for developing a model of one
or more kinds of excellence in teaching students online
(there may well be more than one way to be excellent,
depending on personal values and on situational issues -what
kind of course is it? what is the instructor's role?). The
findings could be used directly to design faculty
development materials. The findings may be useful for
developing criteria for hiring, if it turns out that
people's value and deeply held assumptions are an issue. If
that's the case, it may be more feasible to hire superlative
faculty (and faculty whose values and assumptions make them
easier to educate in these modes of teaching) than to try to
persuade faculty to change their paradigms. What do you
think?
Please send comments about this page
to
Steve Ehrmann.
____
1981 Schneider, Carol,
George O. Klemp, Jr. and Susan Kastendiek, "The Balancing
Act: Competencies of Effective Teachers and Mentors in
Degree Programs for Adults," Chicago: Office of Continuing
Education, University of Chicago.
Return to Flashlight Evaluation Handbook list of
ideas for grant proposals and dissertations
|
