Institutional Studies of the Educational Uses of Web Course Management Systems
Steve Ehrmann
A Project of the Flashlight Network
Last updated October 4, 2004
The Flashlight Network is collecting studies by institutions of their uses of Web Course Management Systems (e.g., WebCT, Blackboard, Angel, FirstClass, Desire2Learn, and others) as part of our effort to develop a study package that technology support, faculty development, teaching-learning centers and other units can use to improve the educational benefits stemming from the use of such systems. (Click here to see our call for studies.)
|
Cheryl Bielema University of
Missouri-St. Louis |
Bielema and Keel surveyed students in two groups of courses. Faculty teaching the first group of courses had made frequent use of their course web sites, while faculty teaching the second group had only rarely visited their course web sites. Students reported that the first group of courses were superior on a variety of measures of quality, and also were more likely to predict that they would take more courses from, and graduate from, the University. | |
|
|
F-LIGHT article on study | The authors have done a series of studies on CMS use at Washington State University. Their data indicates that a) when instructional designers put courses online, faculty-student interaction, active learning, and other indicators of quality are likely to be higher than when faculty put their own courses online unaided, and b) savings in faculty time in development and teaching equal the costs of designer time. |
|
Norm Vaughan, Jim Zimmer Academic Development Centre Mount Royal College |
This is the first of four files from
some very nice work at Mount Royal. The purpose of this survey is to gather
information, from an instructor perspective, regarding the effect that the
web based course development tool - CourseInfo has on the teaching and learning
process. The information gathered in
this survey will be used in a research study which will help to inform the
selection of a Mount Royal College supported web development tool, and its
effective use in teaching contexts. Background skills, features rating forced
choice and free response |
|
|
Norm Vaughan, Jim Zimmer Academic Development Centre Mount Royal College |
Survey
used to gather information, from a student perspective, regarding the effect
that the web based course development tool - CourseInfo has on the teaching and learning process. Results
are to inform the selection of a supported web development tool, and its
effective use in teaching contexts. |
|
|
Norm Vaughan, Jim Zimmer Academic Development Centre Mount Royal College |
Students
described by demographics and computer skills and access. Research reports on
student training and support needs, student ratings of web links and course documents,
student behaviors, impact on learning, learner satisfaction. |
|
|
Norm Vaughan, Jim Zimmer Academic Development Centre Mount Royal College |
Research
results are presented in these areas: 1.
Training and support needs 2.
Most and least useful features of the course development tool 3.
Impact on teaching-learning methods/behaviours 4.
Perceived impact on student learning 5. Instructor satisfaction
|
|
|
University of
Texas, Austin |
This
study surveyed the reactions of a pilot group of faculty to features of the
Blackboard system. As the executive
summary says, "The
value of the study lies largely in its anecdotal responses rather than its
statistical analysis. Comments made by named and anonymous instructors
provided information about their specific needs for an online product. This
information has been and continues to be used to improve the functionality of
Blackboard course Websites and determine necessary levels of training and
support." |
|
|
Henryk
Marcinkiewicz, Ph.D., Director Center for
Teaching, Learning, & Faculty Development Ferris State
University IRC 204 1301 S. State
St. Big Rapids,
Michigan 49307 USA (231) 591-3826 (231)
591-2914 FAX |
This
is not a study. It is a rating scale used in planning for web-based course
management Currently
in press. Two
major divisions: 1) Teaching & Learning
2)
Technical Considerations The
rater scores each item as low, adequate, or exceptional. Example is “Ease of
composition” under instruction presentation. The scale is comprehensive in its coverage of points to consider in making decisions about WCMS. |
|
|
Cheryl Bielema University of
Missouri-St. Louis |
Also:
http://mygateway.umsl.edu [On log in page
choose Help The first item under Help for Faculty is My Gateway
Faculty & Student Survey:FS2000] |
Large-Scale
Adoption: I am characterizing our experience as a large-scale implementation of Blackboard. The final
stats indicate that 186 faculty members added content to one or more of their
course sites (308 total), and these courses comprised 5409 students. Requisite
Computer and Access: 68% of the students accessed Blackboard via modem, off
campus, while 23.6% used student or departmental computer labs. We
were impressed by the fact that 84% of the students indicated they owned a
computer; our faculty members mentioned lack of a computer as a major barrier
for students. Perceptions, perceptions! Learning
Impacts of Blackboard Use: The random selection of instructors yielded
an almost 50-50 split among those faculty utilizing many of the Bb features
and those who only used one or two: 57% greater use of features; 42% lesser use of features, as
"judged" by students indicating which of the Blackboard features
had been utilized in their course. UMSL
students indicated the following learning impacts as a result of their
web-assisted or -enhanced courses. Those more likely to "refer to course
syllabus" [with Blackboard] -- 83%; to "access other online materials"
-- 72%; to "actively participate in the course" -- 61.5%. The statistic we really appreciated seeing
(and are touting near-and-far): "would like Blackboard used in my other
courses" -- 82%. Most of the learning questions (derived
from Flashlight Inventory) are in a positive direction, and that's good! If
you have more specific questions after reading the evaluation report, please
ask me (Bielema). We are well underway with a second semester's training and
support of the Blackboard web course management system here. I'm ready to
assist in the web course management tool study you are launching. |
|
Kevin Oliver Virginia Tech |
This
1999 study of CourseInfo used data about how faculty were using the system to
guide the development of online faculty development resources. |
|
|
Jerry Drake and
Robert Holt, George Mason University |
This
article describes an early, successful use of WebCT to strengthen student
preparation for classroom discussion by using online quizzes. It's a
well-balanced investigation of an instructionally important use of a CMS. |
|
|
Herbert Lyon Black Hawk
College |
Comments
from one instructor on what he considers valuable about WebCT. Useful for
looking at the educational uses of a system, feature by feature. |
|
|
Charles Graham,
Kursat Cagiltay, Joni Craner, Byung-Ro Lim, Thomas M. Duffy Indiana
University |
The
report focuses on evaluation of four on-line courses. The purpose is to
provide feedback on strengths and areas for improvement of on-line courses.
Summary of general findings and recommendations from the course evaluations
are: Instructors
need familiarity with strategies successful for on-line teaching, therefore
publicize best practices. Learn
how to capitalize on strengths of asynchronous conferencing tools. Examine
course management strategies that do not compromise quality. Faculty
need access to development resources. |
|
|
Lynnae Rankine
(lead author) Online Learning
Resource Developer and WebCT Administrator Web Interactive
Study Environment (WISE) part of the Flexible Learning Unit University of
Western Sydney (UWS) Australia. Email
l.rankine@uws.edu.au Stephen Sheely WebShell
Coordinator, WISE. Deborah Veness Manager Flexible
Learning Unit University of
Western Sydney (UWS) Australia. |
This
paper is not an empirical study. Instead it describes how WebCT is supported
at the University of Western Sydney in Australia. The paper includes some useful insights into organizational
routines. |
|
|
Allan Schmidt Frank Keis |
WebCT
User & Training Survey Analysis. |
Good
questions for a scanning survey (these are items the model makes me think of,
not necessarily ones that are in the survey in these words) Describe what a
WCMS is and then ask whether the respondent has used one or more such systems
in the current term? ever? Have you made use of a local technical or
instructional development expert to help you, or to partner with, in
developing one or more courses? teaching assistant? Question(s) about the
server's speed and reliability, tech support when server has problems. Did
you have formal training such as a workshop in how to use this system? Questions
about what worked and didn't work during training to use the system (ask
about hands-on, about documentation, lecture (too much? too little? clear?) [We could also do something just about training that is longer
but I suspect we'll want a section of a scanning survey on this topic, too,
to get people's views of the training some months later). Integration of WCMS system to other
university data and systems? Items
for a more in-depth study. How much time did the faculty member spend doing
various development and teaching tasks? Questions
that may be of less value. Why are you not using the system? If we adopted
this system, for how many more courses would you use it? |
|
Chris Davis Baker College |
Chart
Bb Training BBTrainEval.xls (not
yet on Web) |
Miscellaneous
information collected to begin to examine what is needed to engage faculty in
on-line course development |
| Sandy Britain and Oleg Liber | Essay | This undated essay suggests how to use teaching-learning and organizational ideas to select a Course Management System. It provides some valuable insights for designing studies to detect whether systems are meeting those objectives. |
| (NEW!) Edutools at Western Cooperative for Educational Telecommunications | Web site with check list of features of various Course Management Systems | This system, upgraded from the Landonline web site in British Columbia, provides an extensive resource to help choose Course Management Systems by looking at their features. |