|
|
|
Peer Review of Instruction Lynda Harding 2/2/2006 We’ve come a long way since John Cagle, now a senior faculty member himself, asked a senior colleague to attend his class and provide advice to help him improve his teaching skills. The colleague told him, “I could never sit in on someone’s class. That would be a violation of academic freedom!” Evaluation of courses, including peer review, is now a routine and required component of personnel reviews. Like many other forms of evaluation (including grading student work), course review has multiple purposes; purposes that sometimes seem to be in conflict. One purpose is improvement. As a beginning faculty member, John felt he needed feedback to improve his teaching skills, or even more specifically, the way he was teaching that particular course to a specific set of students. A second purpose of course evaluation is to provide evidence for personnel decisions. Many experts recommend that evaluation for improvement be kept totally distinct from evaluation for personnel decisions. However, the personnel process used very effectively on our campus blurs the distinction between evaluation for improvement (“formative assessment”) and evaluation for final decisions (“summative assessment”). Probationary plans set clear expectations for tenure. Faculty members are evaluated with respect to progress towards the plan goals. When expectations are not met, faculty members are given guidance to help them achieve the defined goals. At some point in the process, however, the faculty member receives a “final grade” in the form of a tenure/promotion decision. Course content is also becoming more public with increased awareness that while faculty members teach courses, students experience curricula. Providing a cohesive curriculum requires a common understanding of how the different courses fit together. An early step in the development of departmental assessment plans is the construction of a table that relates intended student learning outcomes to the educational experiences designed to help students achieve them. As teaching is a complex process, evaluating courses requires multiple sources of evidence. The Policy on Assessment of Teaching Effectiveness (APM 322) emphasizes student ratings of instruction and peer reviews of instructional processes and mentions peer reviews of syllabi and course materials. Often-overlooked sources of information on teaching effectiveness are review of graded samples of student work and instructor-originated assessments of the impact on student learning of newly implemented teaching strategies. These sources of information merit more weight in personnel processes than they currently receive. The details of the teaching evaluation process will vary with the purpose of the evaluation and the teaching methodology being used. These guidelines for peer review of teaching effectiveness recommend an overall process, including some principles to increase the effectiveness of peer review, and provide suggestions for areas that might be targeted in review of different types of courses. Peer Review Process. APM 322 calls for the faculty member carrying out the peer review to “discuss a plan for the evaluation, including the examination of relevant materials such as textbooks, syllabi, representative materials, and examinations.” While it may be more efficient to separate the review of less ephemeral materials such as syllabi, handouts, and examinations from review of the instructional process itself, the pre-observation conference provides a good opportunity to maximize the value of the peer review in improving instruction. It allows the reviewer to learn about the course, the session, the students, and the instructor’s approach. One aspect of the discussion of the “plan for the evaluation” is especially important – working with the instructor to narrow the target of the review. Peer review requires a large investment of faculty time, so it is especially important for the review to focus on important considerations. Like other people (including students), faculty members can only respond to a limited number of criticisms at one time. In addition, an instructor who participates in the development of the plan for evaluation is more likely to implement recommended changes than is one who did not. For faculty members other than those in the first year, the results of previous reviews are likely to suggest areas for improvement. The attached “Pre-Observation Conference” handout used in the campus peer coaching program provides a starting point for the development of forms for recording information at the conference. The peer coaching handouts also include descriptions of a range of techniques for conducting observations of traditional classes, some of which could be adapted for use in other types of classes, including online or service-learning classes. Even peer review intended for personnel decisions may be more effective if the observer works more as a neutral researcher gathering data and less as a judge. The evaluator and faculty member are expected to discuss the results of the visit prior to the submission of a written report to the department chair. This is an opportunity for the evaluator to exercise tact. Where possible, the data should be allowed to speak for themselves, and the primary focus should be on areas targeted in the pre-observation conference. While an important function of peer review is the examination of the level and scope of content, peers reviewing colleagues within their own disciplines need to be especially careful about assuming that their own approach to the content is the only or best approach. Our experiences with peer coaching suggest that it is especially difficult for an observer to remain neutral about the presentation of topics about which he or she cares deeply. Different instructors have different styles and strengths, and resulting differences in instructional approach may be perfectly appropriate. The reviewer must resist the temptation to be dogmatic.
Peer Review of Various Teaching Methods Faculty members are naturally more comfortable reviewing instruction that is similar to their own approach. The pre-observation conference can help the reviewer understand the instructional goals and approach of the faculty member under review. A more holistic view of instruction that avoids the tendency to equate teaching with lecturing can make other forms of teaching seem less foreign. Regardless of instructional mode, evaluation can consider components such as overall design of the course, the appropriateness of course learning outcomes and the assignments or strategies used to achieve the outcomes, the extent to which students are actively engaged in learning, the nature of feedback provided to students, and the course climate. It might make sense to frame evaluation around Chickering and Gamson’s Seven Principles for Good Practice in Undergraduate Education (available online at http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/7princip.htm). Attached is a list of sample effective teaching behaviors assembled from the literature on teaching and learning. The list is not intended to be directive or comprehensive, but is provided to help peer reviewers and those under review identify possible areas for review. As online instruction is becoming more common, and several faculty members have expressed concern over evaluating online courses. In our early experience with peer coaching in online courses, faculty members teaching online were especially eager to have the input of their peers and tended to be less defensive and more open to constructive criticism than were faculty members teaching face-to-face. To maintain equivalence with review of lecture courses, peer review of online courses should include review of several modules or lessons.
Sample Effective Teaching Behaviors
Content and organization of the instructional unit.
· Introduction grabs attention. · Lecture or instructional unit begins with a summary or organizational preview. · Objectives are clear and are met. · Structure is logical and apparent. · Important concepts are stressed. · Transitions are clear. · Time is well budgeted. · Examples and analogies are used effectively. · Explanations are clear and include all steps or points. · Terms are defined. · Verbal instructions or directions to students are clear. · Lecture or instructional unit has closure. · Students are provided with opportunities to engage with the material and process information.
Lecture delivery -- The instructor
· Observes the students to obtain feedback. · Varies voice and volume. · Maintains eye contact. · Uses movement. · Uses planned pauses for emphasis. · Uses body language (leaning forward, smiling) that encourages a positive class atmosphere. · Demonstrates enthusiasm.
Audiovisual aids
· Are legible. · Are used effectively. · Avoid distracting “flash”.
Questioning strategies
· Questions are focused and clearly worded. · Students are called upon in an unpredictable sequence. · In lecture, five to eight seconds are allowed for student responses. · In lecture, the question is asked before a name is given, so that all students have a chance to think about the answer. · Questions are at an appropriate cognitive level. · Where appropriate, all student responses are recorded without positive or negative statements by the instructor, separating brainstorming from the evaluation of responses. · Appropriate praise and tact are used in responding to students’ correct or incorrect responses. · Small group discussion is used to allow all students to participate in answering questions. · The instructor’s attention includes students regardless of gender, ethnicity, or physical location.
Groupwork strategies
· Activity lends itself to group process. · Purpose of task is communicated to students. · Task is well defined. · Instructions are clear. · Appropriate time is allotted · Time communicated to students. · Opportunity is provided for processing or closure.
Assignments
· Target course learning outcomes. · Are worth the time the students and instructor will invest in them. · Have clear instructions. · Are at the appropriate level cognitive level. · Are timed appropriately during the semester.
Experiential learning (including service learning)
· Students are prepared for the learning experience. · The learning experience relates directly to the course student learning outcomes. · The learning experience is appropriate in scope and complexity. · The time required is appropriate to achieve the student learning outcomes. · The learning experience is integrated into the course. · Community-based projects meet real community needs as determined by the community. · Students are given the opportunity to reflect upon and integrate their learning. · The instructor is flexible dealing with the potential loss of control and need for alternative teaching methods in a course with a strong experiential component.
Online instruction
· Students are given opportunities to create an interactive learning community. · Electronic content is current, appropriate, and cleared for copyright use. · Course materials are professional in appearance and error-free. · Goals and objectives are given for the course and for each unit within the course. · All activities are clearly related to the achievements of goals and objectives. · All content and activities are sequenced logically to support achievement of goals and goals are individually defined throughout course. · Evaluations of student progress are given consistently throughout the course and clearly state the goals and objectives for each. · Formative assessment is present in each learning unit. · The faculty member will clearly be able to determine student's progress toward goals and objectives using one or more of the following (quizzes, exams, assignments, communication tools, surveys). · Clear instructions on when and how students should interact with the material is present on all documents and folders. · Materials are organized logically and in sync with the syllabus. Supplemental materials are clearly marked and organized accordingly. · Course schedule and pace are listed in the syllabus and pace is defined on assignment and assessment level as well. · An orientation to the course with instructions for obtaining support and assistance is available both online and in the course syllabus. · The instructor has communicated a clear plan for feedback and response time. The instructor has set expectations for response time and feedback and communication tool etiquette.
Peer Review of Instruction Bibliography Print Resources Arreola, Raoul A, 1995. Developing a Comprehensive Faculty Evaluation System. Anker Publishing, Bolton, MA. The section on peer review (p. 64 – 70) warns of pitfalls and suggests precautions for personnel-related peer review. Bernstein, Daniel J., Jessica Johnson, and Karen Smith. 2000. An Examination of the Implementation of Peer Review of Teaching. 73-86 In Katherine E. Ryan (Ed.) Evaluating Teaching in Higher Education: A Vision for the Future. New Directions for Teaching and Learning 83. Jossey-Bass, San Franciso. Reports the mixed results of a peer review project. Some faculty teams improved their teaching (as measured by student achievement), while others did not (due, perhaps, to the structure of the reward system). If you’re interested in evaluation of teaching, this book is a good resource.
Braskamp, Larry A. and John C. Ory. 1994. Assessing Faculty Work: Enhancing Individual and Institutional Performance. Jossey-Bass, San Francisco. p, 200-209 “Observations and Videoconferences,” provides suggested procedures for conducting peer observations, including observation of collaborative learning.
Centra, John A. 1993. Critical Roles of Colleagues and Department Chairs, p. 115-134 In Reflective Faculty Evaluation: Enhancing Teaching and Determining Faculty Effectiveness. Jossey-Bass, San Francisco. Provides a brief review of a breadth of tools for formative and summative evaluation of teaching. Specific recommendations include the use of trained committees of about three colleagues to carry out RTP-related review of teaching; not overweighting classroom observations; and encouraging faculty members to work together to improve teaching.
Chickering, Arthur W. and Zelda F. Gamson, 1987. Seven principles for good practice in undergraduate education. American Association of Higher Education Bulletin pp.3-7
Chism, Nancy, 1999. Peer Review of Teaching: A Sourcebook. Anker Press. This is a handy book, well referenced and full of practical suggestions and guidelines.
DeZure, Deborah. 1999. Evaluating Teaching through Peer Classroom Observation, p. 70-96 In Peter Seldin (Ed.) Changing Practices in Evaluating Teaching: A Practical Guide to Improved Faculty Performance and Promotion-Tenure Decisions. Anker Publishing Co., Bolton, MA. This chapter provides an overview of the challenges of developing and implementing a peer review system and includes an extensive bibliography.
Howard, Jeffrey, 2001. Service-Learning Course Design Workbook. Michigan Journal of Community Service Learning. Ginsberg Center for Community Service and Learning at the University of Michigan. The source of the “best practices” listed for experiential and service learning.
Glickman, C. D., S. P. Gordon and J. M. Ross-Gordon. 1995. Supervision of Instruction: A Developmental Approach, 3rd ed. Allyn and Bacon, Boston. Includes diagrams of techniques for obtaining data during peer review. Hutchings, Pat. 1996. Making teaching community property: A menu for peer collaboration and peer review. American Association for Higher Education. With many brief examples of peer review processes from various campuses, and an extensive annotated bibliography, this is a highly accessible introduction to peer review and mentoring programs.
Keig, Larry and Michael D. Waggoner. 1994. Collaborative Peer Review. The Role of Faculty in Improving College Teaching. ASHE-ERIC Higher Education Report #2. George Washington School of Education and Human Development, Washington, D.C. The chapter on “Formative Peer Evaluation” provides useful guidelines for peer consultation and examples of peer evaluation processes at a number of institutions.
Murray, John P. 1995. Successful Faculty Development and Evaluation: The Complete Teaching Portfolio. ASHE-ERIC Higher Education Report 95 (8). George Washington University, Washington, DC. This is an overview of the use of faculty portfolios. On pages 61-70 is a useful review of the characteristics of good teaching.
Ryan, Katherine E. 2000. Evaluating Teaching in Higher Education: A Vision for the Future. New Directions for Teaching and Learning 83. Jossey-Bass, San Francisco.
Seldin, Peter. 1999. Changing Practices in Faculty Evaluation: A Practical Guide to Improved Faculty Performance and Promotion/Tenure Decisions. Anker Publishing Co., Bolton, MA.
Wilkerson, 1988. Classroom observation: the observer as collaborator, p. 95-98 In E.C. Wadsworth (Ed.), A Handbook for New Practitioners. Professional and Organizational Development Network in Higher Education. New Forums Press, Stillwater, OK. Intended for educational consultants in teaching and learning centers, a quick introduction to observing classes and providing effective feedback. Ethelynda Harding February, 2006
|
|
|
February 2 – 3-4 pm
EST References, Resources:
|
|
|
Your Questions, Suggestions, Comments If you have any questions or comments about this workshop, please contact Lisa Star at online@tltgroup.org Please send your questions or suggestions for improving our online workshops - including topics or leader/presenters that you would like us to include.
Send to Steve Gilbert at:
|
|
Joys and Sorrows
|
|
|
|
|
|
TLT-SWG Highly Moderated Listserver Since 1994 Faculty/Professional Development Program |
|
