Flashlight Case Study


Handbook and Other Materials l Asking the Right Questions (ARQ) l Training, Consulting, & External EvaluationFAQ


Student Satisfaction and Perceived Learning via a Course Management System

Cheryl Bielema, Ph.D., Instructional Development Specialist (2003)

Our institution adopted Blackboard as its web course management system in the fall semester of 2000, after several years using a homegrown web course tool. Our customized name for Blackboard is MyGateway. The faculty use MyGateway largely as part of a hybrid, or blended, course delivery strategy to enhance learning and help them manage class activities (communications, sharing of documents and grade reports).

The current report is part of a longitudinal study from the first semester of implementation.  The survey instrument is designed to collect data regarding computing capability and resources available to students, the degree of use made of MyGateway in select classes, and the perceptions of students regarding the use of MyGateway in select classes.  We sampled ten per cent of the courses using MyGateway. High use classes were distinguished from low use classes based on the instructor’s use of the course site.  This semester we measured daily instructor access of 12 key content and administrative areas.  Measurement was taken on September 18, 2002 .   The scores ranged from a high of 381 to a low of 1.  The mean number of day accesses was 46 (median 32).  Courses with instructor ranking above the mean were considered high use courses, and course with instructor rankings below the mean were considered low use courses.  We especially focused on the degree of use of MyGateway to make comparisons of learning activities and student satisfaction among classes that use a course management system more and those using it less.  Highlights from the survey follow:

  • 85% of the students indicated they own Pentium, Macintosh and “other” personal computers

  • 65% of students access MyGateway primarily off campus

  • 37% of students report some form of off-campus, broadband Internet access (DSL, cable modem, or work LAN)

Students accessing MyGateway from on campus were statistically more likely (p < .05) to:

  • Actively participate in the course (Question 12)

  • Work on assignments with other students (Question 13)

Students in classes with high use of MyGateway were statistically more likely (p < .05) to:

  • Review lecture notes for clarification (Question 9)

  • Discuss ideas with other students (Question 11)

  • Actively participate in the course (Question 12)

  • Work on assignments with other students (Question 13)

  • Complete assignments on time (Question 14)

  • Access other online materials related to course content (Question 15)

  • Spend more time studying for the course (Question 16)

Students in classes with high use of MyGateway were also statistically more likely (p < .05) to continue at UMSL, evidenced by their answers to these questions:  

  • Agree that they are very satisfied with the course (Question 32)

  • Take another course at UMSL (Question 36)

  • Complete their degree at UMSL (Question 37)  

Students in classes with high use of MyGateway were also statistically more likely (p < .05) to indicate frequent communication and interaction between faculty, content and themselves:

  • Agree that they were more able to seek clarification (Question 8)

  • Receive instructor comments on assignments quickly (Question 10)

  • Communicate with their instructor (Question 17)

  • Think more about course topics (Question 21)

  • Understand concepts and ideas in the course (Question 28)

Students in both high and low use classes were statistically likely (p < .05) to agree they would like to have MyGateway used in other courses.

Analysis of the data indicated several levels of responsibility for increasing the efficiency of the course management system and for ensuring greater satisfaction and success as students are involved in blended course models at UM-St. Louis. Orientation materials and technical support guidelines are available to both faculty and students; additional topics were identified for development by the MyGateway support team.  It was also recommended that faculty assume responsibility for orienting their students to MyGateway. Further, responses from these students indicated that faculty be encouraged to continue integrating MyGateway features and that future syllabi include expectations for student work online.

The full report can be found at http://www.umsl.edu/technology/mgwhelp/mgwinfo/mgwinfo.html, MyGateway Student Survey: FS2002. A copy of the instrument is included in the Appendix. 

(NEW!) Annual follow-ups have been done since this article was originally written; to see the Winter 2005 report, please click here.

For further information about the results, please contact Cheryl Bielema (314.516.7134), email bielema@umsl.edu or Robert Keel (314.516.6052), email rok@umsl.edu.

For Flashlight Online users interested in using, or modifying, the Bielema-Keel student survey is now under review for inclusion as a Flashlight template. It can already be used by all Flashlight Online authors. Use the "template" button to see it; It is near the bottom of the list of templates, ZS18160. If you don't know how to use a template to draft a survey, consulting the help system online or our tip sheet for using Flashlight Online.


PO Box 5643
Takoma Park, Maryland 20913
: 301.270.8312/Fax: 301.270.8110  

To talk about our work
or our organization
contact:  Sally Gilbert

Search TLT Group.org

Contact us | Partners | TLTRs | FridayLive! | Consulting | 7 Principles | LTAs | TLT-SWG | Archives | Site Map |