Assessment of a Handheld Computing Initiative
Betty L. Black and Marianne Niedzlek-Feaver
Department of Zoology, NC
Educational Goals: In Fall, 2000 we began a study to explore the use of handheld computers in two upper level Zoology courses: Developmental Anatomy and Evolution. Our goals were to utilize small computing devices to increase class participation, to introduce active learning exercises into our courses, and to facilitate "mobile learning" outside of the classroom.
Equipment: Having obtained funds for a pilot project, we purchased the Handspring Visor (a PDA using the Palm operating system) and the HP Jornada 820 (a mini-laptop marketed as "handheld" with 640 x 480 screen and a keyboard) in numbers sufficient for students to work in pairs (Developmental Anatomy) or groups of 3-5 (Evolution).
The Visors have been used for 3 years, during each Fall semester of Developmental Anatomy. Approximately half of the students in this course were allowed to check out a PDA for their exclusive use during the semester with the understanding that they would bring the device to class. The entire course website, including outlines of content, links to images with explanatory text, and study guides was stored within each Visor using iSilo software.
Discovering Limits: A few classroom exercises, involving labeling of images and filling in tables, were initially tried, but proved cumbersome to implement; the only way of providing the materials to the students was via "hotsynching" with a computer outside of the classroom. Students rarely used their PDA for note taking, although several accessed the stored course materials during lectures.
Because classroom use of the Visors was limited, our evaluation focused on use outside of class.
Focus Group Findings: A focus group held at the end of the 2000 semester elicited positive comments such as "screen size was good for reading" and "needed to study, then realized info was in my pocket". Negative comments centered around the need to scroll to view large images "don’t use for images larger than screen" and dislike of writing with a stylus "taking notes in class was frustrating".
Comparison of student exam averages indicated significantly higher achievement in the PDA-using group (78.4% ± 2.4 vs. 70.7% ± 3.8, P = 0.078), but this difference was not maintained in subsequent years.
Survey: A more extensive questionnaire was given to students in 2001 and 2002 of which a portion is summarized in tables 1-3. Results revealed that students did use the Visors to study course content and that this usage increased dramatically during the second year. In both years, students expressed a desire to use PDAs in other courses, with "definitely yes" increasing from 36% to 59%. Similarly, the fraction of students who would purchase a reasonably priced PDA for personal use rose from 7% in 2001 to 41% in 2002.
The Innovation: The Jornada
mini-laptop computers have been used both in the Spring
semester of Developmental Anatomy and in the Evolution course. Because our
pilot project included funds for
· In Evolution, the general approach was for student groups of 3-5 to examine different papers dealing with a general concept, or papers promoting different viewpoints of a current controversy. Students accessed articles wirelessly from the course website and recorded notes on Compact Flash cards (a storage media used with Jornadas). Flash cards were collected and used by the instructor to transfer student work to the course website.
· In Developmental Anatomy, students were assigned an article on an outside website or provided with a PDF formatted article on a Compact Flash card. Students discussed their assigned article with a partner during class, then either gave a short oral presentation or submitted an essay via a web-based form. Other classroom exercises in Developmental Anatomy included on-line quizzes taken during class with student responses immediately summarized and presented by the instructor to generate class discussion, and analysis and/or labeling of assigned images by students with subsequent oral presentations.
Evaluation of Jornada usage has been more extensive than for PDAs. In the first year of use, Evolution students were allowed to check out a Jornada for the semester. A questionnaire indicated that students found the mini-laptops less useful than expected outside of class. Focus groups further revealed that students had not realized the limitations of the Windows CE operating system (they expected the Jornadas to be as capable as a laptop computer) and were disappointed at the paucity of wireless access points outside of the classroom. In both courses, evaluation revealed that note taking on Jornadas was infrequent, but that students wanted to make notes on downloaded course web pages (a capability not supported by any available software). The questionnaire utilized in Evolution (table 4) further revealed that the most popular uses of the Jornada were accessing the Internet from campus and viewing graphics during class. There was some interest in using Jornadas for homework and in providing feedback to the "lecturer" during class, but activities such as transferring information to personal websites or sharing information with classmates (via "beaming" or Flash cards) were deemed less useful.
For the years in which Jornadas were used only in the classroom, student satisfaction with the technology was high (table 5). Data from the most recent Developmental Anatomy questionnaire (tables 6-8) indicates that students strongly prefer the Internet to access both articles and images during class. A digital means of accessing text was chosen by 67% of the students as compared to 33% who preferred either photocopies or a text book. For images, the division was even greater: 79% digital vs. 15% hard copy. When asked whether they would like to use Jornadas in other classes, 75% responded positively. Finally, student averages in Developmental Anatomy from the semester before and after Jornada use were compared. Student learning apparently improved with Jornada usage (or from the initiation of active learning exercises), since class average increased from 73.8% ± 3.5 to 82.7% ± 1.3, P = 0.068. A similar improvement in student performance was noted in Evolution as the portion of the course taught "actively" increased to 100% (table 5).
As a result of our 3-year assessment of handheld computing, we have made the decision to continue placing course materials in the Visor PDAs, but to extend Visor usage to the types of classroom exercises that have proved successful with Jornadas. To this end, we have obtained 802.11b wireless modules for use with Visors during Fall 2003. Based on positive results from active learning exercises, we will continue this approach in both Developmental Anatomy and Evolution. However, enhanced student performance may need to be balanced with student satisfaction regarding teaching style. Our questionnaires have revealed the interesting phenomena that while Evolution students performed best when the entire course was taught actively, they were less satisfied than when 40% of the course remained in lecture format (table 5). Similarly, when Developmental Anatomy students were asked which format they preferred, 54% chose 2 lectures with 1 "active learning" exercise per week, 21% chose 1 lecture with 2 exercises per week, and 25% preferred all lecture. No students wanted an exclusive active learning format. Follow up questions or focus groups will be added for our next evaluation to determine the reason for these attitudes.
Finally, due to the success of our pilot study and the increasing positive student reaction to small computing devices, our College has purchased Palm PDAs for use in the introductory "Topics in Agriculture and Life Sciences" course taken by all entering Freshmen. The College of Agriculture and Life Sciences plans to promote the incorporation of PDAs with Internet access into additional courses within the College, and we envision a future in which active learning will be facilitated by wireless computing devices throughout the University.
Additional information may be obtained from our website or by contacting the authors: betty_black@ncsu.edu and mnfeaver@unity.ncsu.edu.