Evaluation Resource

Making the massive Statistics 200 course at Penn State more efficient has been the aim of a project whose evaluation is being carried out by Dawn Zimmaro. The project is supported by the Pew-funded Center for Academic Transformation headquartered at Rensselaer Polytechnic Institute (RPI).

Stat 200 begins with weekend and Monday reading of online and paper textbook material, proceeds to a "readiness" exam that students take first individually and then with their study group (the group needs to agree on an answer to each question). This kind of self-study, collaborative learning, and guided inquiry are key to the program's aim of improving outcomes relative to more traditional forms of instruction.

Then comes the single, mid-week lecture. Though the lectures are large (sometimes 200 students or more), care is taken to make the them interactive; for example, students might measure one another's head size and then move to a spot in the auditorium corresponding to that size, physically demonstrating a distribution. At the end of the week, students go to a computer lab to work in pairs or larger groups on complex tasks and assignments.

The evaluation plan puts primary emphasis on assessment of outcomes in statistical knowledge (pre/post tests), attitudes toward statistics (pre/post), and costs (reducing the cost of large classes is a primary goal of the Pew Center). Focus groups are the primary means of investigating the activities that determine those outcomes.

The project and its evaluation are still young but focus group findings are already proving useful. 

For example, focus groups uncovered the fact that a key element of the assessment-improvement cycle wasn't well understood by students and thus wasn't working. Interviewers discovered that students didn't realize that their study groups could (and often should) appeal low grades on assignments in order to receive partial credit if their reasoning was at least partially sound. From the faculty view this appeals process was supposed to be a key part of helping the students learn, and learn how to collaborate. But, because too many students didn't understand the purpose or rewards of the process, few used it. Steps are now being taken to make sure that students understand the process. 

The focus groups also indicated that half of the new lab assignments were not working as planned; lab assignments will be redesigned this summer.

Studies of program cost (a standard part of projects funded by the Pew Center) indicate that the project is making prudent use of local resources. Cost savings so far come mainly from the reduced use of teaching assistants. This particular savings did not threaten the department's support for its graduate students. Statistics is not a typical undergraduate major so grad students in statistics often have not taken a course like Stat200). Finding good TA's had been a burden, so this savings was a relief. In the longer term, the program expects to see other kinds of savings.

For more information on this unfolding study, contact Lisa Lenze, project manager for instructional design and evaluation on this project and see Pew Center's page on this project.

- This summary was written by Steve Ehrmann, based on a telephone conversation with Lisa Lenze.