Evaluation Of The North Carolina State University College of Engineering Mobile Computing Pilot Program

Joni Spurlin, Director of Assessment, College of Engineering

The goal of this project is to determine how the use of laptop computers and wireless connectivity can enhance the undergraduate academic experience in engineering.

This program assessment had five objectives; this brief article will discuss only two of them:

1. Evaluate the impact of teaching with wireless technology in a collaborative setting on student performance, specifically in the areas of writing and problem solving.
2. Evaluate the impact of teaching with wireless technology in a collaborative setting on faculty workload, pedagogy and amount of material delivered.

In year one (2001-2002) of the pilot program, 37 freshmen honor students in engineering participated.  Students in this pilot program were able to purchase a powerful computer (IBM ThinkPad A22m or T22) at a special price.  The students’ academic experiences were scheduled in blocks; that is, they were all scheduled in many of the same courses during their first academic year. The courses were developed by faculty and required the student and faculty to use the wireless laptops in the classroom.  For more information about the pilot program itself, see: http://www.eos.ncsu.edu/soc/pilot.php

Below is a brief explanation of the assessment methodology and a few of the findings from this pilot program.

Student performance was assessed in several courses where one section was taught with the wireless computer in the classroom and one section was taught with no expectation of student computer use during class.  Both sections were given the same final project or test and the results were compared.  Student backgrounds in the two sections were matched to help assure that, if there were differences in performance, they came from differences in teaching.

Students in the laptop sections demonstrated better problem solving ability. For example, in a computer-programming course, the instructor gave the same set of problems on the final exam to all sections of the course.  Later, each problem was judged using a problem-solving rubric [Editor’s Note: click for a definition of “rubric.”].  The scores on the final exam were also compared.  The laptop students scored higher on many dimensions of the problem-solving rubric and outperformed the comparison students on the final exam.  For a graphics course, the instructor gave the same final project to both a laptop and regular section.  The final project of developing a model was assessed using a similar problem-solving rubric.  The results from the assessment showed that the regular section students had more ability at paper sketches and visualization of the models, whereas the laptop section students had more ability at the preciseness of geometry and dimensioning of the model. 

Why did the laptop students score better?  Hardware,  by itself, has no impact on learning. But our study showed that faculty took advantage of the laptops to teach in more effective ways.  The impact of the technology’s availability on teaching was assessed through faculty logs, written summaries of their experiences and verbal discussion with the project managers. The faculty reported that having the laptop in the classroom allowed these courses to use more inquiry-guided learning approaches: lecture material alternated with lab/hands-on experiences.   Having lab-type experience tightly coupled with short lecture segments was the biggest benefit to students.  The students could practice what they were learning immediately, instead of waiting two days when they were in a lab section to practice what they learned in a prior lecture. 

For the graphics course, it is also thought that the presence of laptops in the laptop section and the lack of daily access of computers in the regular section may have led to more emphasis on sketching in the regular section.  Conversely, daily access to computers in the laptop section allowed the students to excel at the solid modeling tasks.  Because of the daily use of computers, the instructor in the laptop sections felt he was more demanding on the complexity and sophistication of models expected than in the regularly taught section.

The laptops enabled other differences in the learning process as well.  For example, the students were able to have software on their machines that they normally would have access to only in the labs.  (For example, for the calculus course, the students were able to have access to MAPLE software on their personal laptops.)

There were logistical and economic consequences, too.  The wireless connectivity allowed the faculty to teach in any room that had wireless capability, and therefore, the faculty were not tied down to a special lab with special equipment. 

Although we cannot be certain whether the improvement in student performance was caused by the technology or the fact that the instructor modified their teaching pedagogy because of the technology or because the lecture and lab experience was more closely aligned, this brief assessment shows that the wireless laptops program had a positive impact on teaching and on student learning and abilities.  In addition, the students had more access to software than other students, which may have also had an impact on student learning.

Consequences of our study: we have concluded that in a few years, most students will bring mobile computing devices to college, whether we require them or not, so the institution needs to prepare for the expectation of the students in this regard.   To ensure that our faculty are ready, we have initiated a faculty forum in which faculty are invited to discuss, present, and learn about teaching with this technology.  The forum is held at least two times per semester and has helped faculty discuss technical as well as pedagogical issues.

For more information, please feel free to contact:

Joni Spurlin, Ph.D., Director of Assessment, College of Engineering, North Carolina State University, Raleigh, NC. jespurli@eos.ncsu.edu;  or Kathy Mayberry, Coordinator of Student Owned Computing, College of Engineering Kathy_mayberry@ncsu.edu.