Evaluating Many Assignments: A Flashlight CAT

Handbook and Other Materials l Asking the Right Questions (ARQ) l Training, Consulting, & External EvaluationFAQ

ARQ Workshop on this CAT l List of Flashlight CATs  l  Flashlight Evaluation Handbook Table of Contents

These materials are for use only by institutions that subscribe to The TLT Group, to participants in TLT Group workshops that feature this particular material, and to invited guests. The TLT Group is a non-profit whose existence is made possible by subscription and registration fees. if you or your institution are not yet among our subscribers, we invite you to join us, use these materials, help us continue to improve them, and, through your subscription, help us develop new materials!  If you have questions about your rights to use, adapt or share these materials, please ask us (info @ tltgroup.org).

This course research design (or Classroom Assessment Technique: CAT) is patterned after a study done some years ago by Jon Dorbolo of Oregon State University. As a young instructor, Jon taught "Introduction to Philosophy" for the first time. Every time he assigned a reading or other work, he asked students to fill out a form that asked about how difficult the work was, and how interesting it was.

The bad news: he discovered that almost every reading was judged by a majority of students to be boring.

Better news: it was not the case that a few students liked everything while a majority liked nothing. Instead each student had some readings they liked.

Best news: There was a pattern in their preferences.  One group of students liked readings A, C, F and so on, while another group liked B, D ... and a third group liked E, G .....   To Dorbolo the meaning of these patterns was obvious.  Students had different views of the meaning of life that were influencing their reactions to the readings.

That was the clue he needed. Dorbolo decided to redesign the course for the next term, allowing students to choose from seven different tracks of readings. One track, for example, was focused mainly on religious philosophy. A second track focused on utilitarianism and another on existentialism. Although each track was geared to a different set of student assumptions, all tracks were designed to teach similar skills of philosophical reasoning. Students spent some time in their own tracks, and some time discussing or debating issues with students in other tracks.

Lessons from this story:

  1. Finding out that students aren't (yet) very interested in the assignments can provide valuable clues for how to get them more interested, if you can make inferences about how there's a mismatch between what they want and what you offer.

  2. Technology has opened up some new options for teaching a course in which there are several groups of students, each of which wants something different.

Building your own study: We have created a couple of Flashlight Online templates that you could adapt ("Engaging Assignments').

  • Version A asks only about homework assignments, and asks students how much time they spent on that assignment. Handout; Preview of Survey.

  • Version B can be used to ask about any course activity (homework or in-class activities); it also asks students to describe one thing about that activity which influenced them to spend more time (or less time) on it. Handout; Preview of Survey

Once your data is in, take a look not just at the total responses to each reading or assignment, but also at who likes what. Are there patterns?  How much you take advantage of those patterns to create assignments and classwork more likely to engage all your students?


PS. If you'd like to see a course designed for students with different preferences, click here to explore some of Dorbolo's course materials.

Alias for this web page: http://bit.ly/Dorbolo 

PO Box 5643
Takoma Park, Maryland 20913
Phone
: 301.270.8312/Fax: 301.270.8110  

To talk about our work
or our organization
contact:  Sally Gilbert

Search TLT Group.org

Contact us | Partners | TLTRs | FridayLive! | Consulting | 7 Principles | LTAs | TLT-SWG | Archives | Site Map |