MyGateway and FS2000 Course Survey Results
MyGateway
(i.e., Blackboard™) was introduced on the University of Missouri-St. Louis
(UMSL) campus Fall Semester 2000. Approximately 150 instructors used the web
course management tools for 270 courses. Their level of use varied from posting
the syllabus and emailing students to utilizing the full range of features
available (i.e., course documents, discussion board, external links, group
pages, online quizzes, and student gradebook).
An
extensive training schedule was implemented during the annual Summer Institute
for faculty, as well as continuing orientation and teaching tips through the
semester. The MyGateway support team included an innovative faculty member
Robert Keel, Sociology lecturer, as coordinator, Mary Fowler, User Services
Manager, Cheryl Bielema, instructional designer and coordinator of South Campus
Faculty Resource Center, Kyle Collins and Kelly Crone-Willis, system
administrators, and Jennifer Spearman-Simms, North Campus Faculty Resource
Center. The total number of faculty participating in training was 175.
Evaluations
were completed by participating instructors, mid-semester, and by students in
randomly selected courses, near the end of the semester. This report summarizes
the results of these two evaluations. Additionally, implications for future
technology initiatives, support and training, are suggested for administrative
consideration.
Instructors
answered which of the features they were using; percentage of total work hours
per week they had devoted to MyGateway, and offered one successful teaching
technique. They were asked to list barriers and benefits to students with
online teaching and to provide suggestions for additional support. They
accessed the web-based survey via Flashlight Online™.
The percentages faculty members gave for the amount of time
devoted to using or teaching with MyGateway each week ranged from a high of 85
percent to a low of 2 percent. The mean percentage was 15 percent (i.e.,
averaging, 6 hours per week).
When asked whether this time was higher, lower, or above the
same as the time they spend teaching traditional courses, there were more
faculty members who indicated the time spent with online preparations and
teaching was "higher"
(44%), 29% responded "less
time," and 27%, the time was "about
the same." Several faculty members indicated they would be spending
less time in subsequent semesters.
Faculty members listed the following barriers and benefits
for students when online course activities and materials are required. The
barriers and benefits are ordered according to frequency.
Lack of computer and access from home (n=13)
Lack of basic computer skills
(n=10)
User name and password problems
(n=10)
Lack of experience with active
learning/self discipline (n=6)
Reluctance or unwillingness of
faculty to give up face-to-face activities in favor of online strategies (n=4)
Course materials always available
(n=20)
Opportunity to thoughtfully
address issues (n=7)
System is easy to use and
convenient (n=6)
Increased contact between
instructor and student (n=6)
Access enriched learning
materials (n=5)
Preparing students for workplace
after graduation (n=5)
[Opportunity to] learn at their
own pace (n=5)
Having lecture notes provides a
study guide, allows concentration on lectures, and generates great discussions
within on-campus class time (n=4)
Creating a learning community/study groups of students (n=4)
Respondents listed a variety of teaching techniques that
they had implemented with MyGateway.
Most often mentioned were electronic discussions that continued beyond the
classroom, posted announcements and class materials, and use of quizzes.
Details follow.
--"Discussion group is a
great way to get feedback on lectures. I often use the discussion to correct
misperceptions and to expand on creative ideas"
--"Threads allow several
conversations to go on at once. Students pick and choose the topics.... This
extends our classroom discussion"
--"Students posted a review
of an article...[received] feedback from other students.... I began to
understand how some of them process information"
--"I
feel like I'm more in touch with my students"
--"Improved
access to lecture notes including PowerPoint presentations"
--" Notes contain a variety
of links that allow students to explore concepts and ideas on their own"
--"Pages
that simultaneously document and demonstrate software"
--"Collection
and immediate sharing of current articles, events, and trends"
--"I use the quizzes as
reviews for in-class exams. I've noticed an increase in the grades on the first
test compared to years past"
--"Online quizzes appear to
be effective--helping the students stay on top of their reading in order to
prepare for the quizzes"
--"I
have established student presentation teams as email groups"
Support Ideas
The following are ways in which the MyGateway support group
might help faculty do more with online teaching and learning. The comments are
divided into several categories: Faculty Training and Development; Faculty
Support; Software/Hardware Improvements; and, Student Help.
Continue with update classes.
Suggested topics included "Instructional uses for discussion boards and
virtual chat," "New features of the next MyGateway
version," "Developing
quizzes," "Instruction strategies for totally online courses,"
"Use of student gradebook."
Session where successful users
demonstrate how they have used MyGateway in their classes, scheduled as
"brown bag lunches."
Resource pool of Internet-based
learning activities (e.g., Merlot. com)
Provide printed tutorials and/or
remind faculty where to find existing materials
Be available for questions at the
time they need help
Help with transporting work
already developed to future platforms
Help convert traditional
materials - lecture notes, handouts, exercises - to online
Make it easier to negotiate between the
pages, and between courses (a common suggestion)
Create rubric construction system for
grading student papers and submitting grades to the gradebook. Other
improvements: mechanisms for deducting points for late submissions; making
corrections to student papers; and the ability to que student papers for
sequential opening
More classrooms equipped with computers
Refine password and IDs for
students ("make them more sensible"); allow work/home email addresses
to be entered into MyGateway
Crash course for students before
semester begins
More PR to the students indicating how
this is a good thing
Representative
courses were selected for an end-of-semester evaluation to be administered in
the classroom. "Active" courses in the Blackboard system were first
categorized by college. (Active is defined as those web sites in which content
was added during the semester). The number of instructors to be involved was
based on that college’s participation rate in the initial semester. Individual
names were randomly selected from each college pool; the instructors were
contacted for participation via email. The specific courses to be surveyed were
identified in consultation with the instructor and by their “fit” within the
week’s evaluation schedule.
In
the initial contact, six of the instructors indicated they had not used
MyGateway during the semester. One of these was using the legacy web course
tool, Course Web Wizard. Another
instructor declined to participate, and another who originally agreed dropped
out when he was called away during the survey week. The remaining five
instructors did not respond to follow up emails and telephone calls.
The
final sample numbered 14 courses, comprised of 565 enrolled students. Eight of
the classes were surveyed face-to-face with hard copy instruments and scantron
sheets. The other six were invited to complete an online survey. The in-class
surveying yielded a higher response rate than the more “voluntary” online
version, (66 percent compared to 26 percent response rate, respectively).
Instructors using the online survey were asked to remind their students after a
week to complete the survey if not already done. No attempt was made to reach
the students who were absent the day of the survey completion in face-to-face
settings. Total respondents numbered
282.
Students
indicated the highest degree earned, if applicable. They were asked how many
credits they had completed at UMSL, as well. Half indicated no degree and are
classified as undergraduates for the table below. Almost a quarter (23%) had
previously earned an Associate's degree while 20.5 percent indicated they had a
BA or BS. Those having completed advanced degree programs comprised 3.2
percent.
Table
1. Highest Degree Earned
|
Degree
Earned |
Percent |
|
No
Degree |
51.0 |
|
AA/AAS |
23.0 |
|
BA/BS |
20.5 |
|
MS/MA/PhD/EdD |
3.2 |
Credits
earned at the time of the survey were sub-divided into 15 semester-hour
segments. Thirty-eight percent appeared to be in their first semester at UMSL,
indicating 0-15 credits; 16 percent had earned 16-30 credits; 18 percent, 31-45
credits, while 7 percent checked 46-60 semester hours. Those accumulating 60+
credits comprised almost 20 percent of the respondents.
Technology
requisites are a computer and Internet access. Students were asked to check
primary computer access and capabilities they had available. Types of Computer,
Primary Access Location, and UMSL Dial-Up Traffic are the three tables that
follow.
Table
2. Types of Computers Owned by Students
|
Computer
Types |
Percent |
|
PC,
Pentium or newer |
49.6 |
|
PC,
486 or older |
20.5 |
|
Macintosh |
2.5 |
|
Other |
7.8 |
|
Don't
Know |
2.8 |
|
No
personal computer |
16.6 |
Nearly
half (49.6%) of the respondents reported owning a computer, Pentium or newer,
20 percent owned a 486 or older PC, 2.5 percent had a Macintosh. An additional
8 percent indicated they didn’t know what type of computer they owned. Sixteen
(16) percent did not report owning a computer.
Table
3. Primary Access Location
|
Internet
Access Location |
Percent |
|
Off
campus, via modem |
68.3 |
|
Student
or departmental computer labs |
23.6 |
|
Student
Housing |
3.7 |
|
Other
UMSL computers |
3.3 |
|
WebTV |
1.1 |
Students
indicated their primary access location, off-campus via a modem (68%), computer
or departmental labs (23.6%), student housing (3.7%), and WebTV (1.1%). If a
student’s primary access was from the computer labs, they were questioned about
wait time in getting to a computer. Their responses indicated limited wait
times, with 62.3% typically waiting 0-5 minutes, 26.5% having waited 6-15
minutes.
Table
4. UMSL Dial-up Traffic
|
Percentage
of Time Dial-up is Busy |
Percent
Responding |
|
0-10 % |
70.1 |
|
11-25% |
17.7 |
|
26-50% |
6.1 |
|
51-75% |
4.3 |
|
76-100% |
1.8 |
If
a student’s primary access was via dial-up and modem, they were asked about the
time they encountered a busy UMSL dial up. Reports were favorable, in that 70
percent experiencing busy dial-up 0-10% of the time, and an additional 17.7%
reporting busy dial-up 11-25% of the time.
Satisfaction
with individual Internet access speed was a third aspect to be questioned.
Nearly 75% indicated they were satisfied with speed of access, while the other
26% indicated dissatisfaction. Important to future educational applications
using multimedia is the fact that 86% of the students responded that their
personal computers were “multimedia capable.” Access speed may be adversely affected
by the bigger, better and more diverse applications.
Students indicated whether or not
they had used the variety of features available on MyGateway, as did the
instructors in the "Mid-semester MyGateway Assessment." Students in
the sample utilized the features in about the same proportion as the
instructors had reported mid-semester. There were several notable exceptions.
The areas, quizzes, course documents, specifically readings, student drop box,
student web pages and virtual chat, appeared to be greater draw for the
students than the instructors.
Comparisons
between student and instructor responses regarding the specific features
utilized show greater use from the students’ perspective. This finding might
have several explanations. First, the students were instructed to skip
questions that did not apply. The students whose courses used only a few of the
features may have skipped the remainder, rather than mark each one “no.”
Secondly, the instructors responding to the mid-semester assessment were not
the same as those instructors whose classes were selected for the students'
survey. That faculty group may not have utilized the features to the same
extent as the randomly selected group.
In
addition, students may have utilized the capabilities of MyGateway independently of their instructor. This unintended
consequence of students’ MyGateway
usage was first noted when the coordinator took courses off-line that didn’t
have a specific faculty member named as instructor. Students began calling the
help desk, wondering why their courses disappeared and describing how they were
using the course web site to communicate with classmates and access web
materials.
Table
5. Utilization of MyGateway Features
|
Features |
Percent Students IndictingUse |
Percent Faculty IndicatingUse |
|
Announcements |
86 |
86 |
|
Assignments |
81 |
72 |
|
Email |
75 |
61 |
|
External Links |
66 |
56 |
|
Course
Documents/Lecture Notes |
74 |
54 |
|
Student Gradebook |
76 |
51 |
|
Course
Documents/Readings |
76 |
40 |
|
Discussion Board |
49 |
40 |
|
Quizzes |
62 |
24.5 |
|
Student Drop Box |
50 |
16 |
|
Group Work Pages |
21 |
12 |
|
Student Web Pages |
46 |
12 |
|
Virtual Chat |
27 |
.08 |
A
series of statements sought to identify any learning differences between
web-assisted courses and traditional courses, not utilizing MyGateway. The
areas where respondents indicated greater differences included “referring to a
syllabus" (83.9% agree; 16.1% disagree) “receiving quick feedback on
quizzes and exams" (79.2% agree; 20.9% disagree), "accessing online materials"
(72.4% agree; 27.6% disagree), “communicating with instructor” (70.5% agree;
28.5% disagree), “helpfulness of quizzes” (68.2% agree; 31.8% disagree), and
“receiving instructor comments quickly” (67.9% agree; 32.0% disagree),.
Areas
where there was less difference between face-to-face courses and web-assisted
courses appeared in “work with fellow students,” (27% agree, 73% disagree),
“greater study time devoted,” (54% agree, 46% disagree), and about “discussions
about concepts improved learning,” (45% agree, 55% disagree). These results may
indicate that the instructors did not have specific learning objectives related
to online collaborative assignments, reading and research completed online, or
threaded discussion requirements.
Table
6. Learning with MyGateway
|
Using
MyGateway, I was more likely to: |
AGREE |
DISAGREE |
|
Seek
clarification |
62.1% |
37.9% |
|
Review
lecture notes |
54.5% |
44.5% |
|
Receive
instructor comments on assignments quickly |
67.9% |
32.0% |
|
Discuss
ideas with other students |
38.3% |
61.6% |
|
Actively
participate in the course |
61.5% |
38.5% |
|
Work
on assignments with other students |
26.8% |
73/2% |
|
Access
other online materials related to this course |
72.4% |
27.6% |
|
Spend
more time studying for this course |
54.3% |
45.7% |
|
Communicate
with my instructor |
70.5% |
28.5% |
|
Receive
feedback on quizzes and exams quickly |
79.2% |
20.9% |
|
Refer
to course syllabus |
83.9% |
16.1% |
|
Online
quizzes were helpful |
68.2% |
31.8% |
|
Online
discussions encouraged more thinking about the course concepts |
44.8% |
55.2% |
|
Online
environment contributed to my learning |
64.9% |
35.1% |
|
I
would like to have MyGateway used in other courses |
82.6% |
17.3% |
In
summing up the online learning experience, students were asked if the online
learning environment had contributed to his/her learning. Sixty-five (65)
percent of the respondents agreed, while 35 percent disagreed. An even stronger
advocacy for online teaching and learning was revealed by 82 percent of the
students who indicated they would like MyGateway
used in other courses.
Respondents
were asked to assess their rate of participation and effort in using MyGateway compared to participation and
effort in traditional courses. In each case, a majority indicated that their
participation and effort was about the same. Fifty percent marked about the
same participation, and 54 percent indicated about the same effort. However, 35
percent indicated “greater to much
greater” participation, and 33.5% indicated “greater to much greater” effort.
Perception
of faculty enthusiasm was positive, as students indicated whether or not “the
instructor was excited about using MyGateway
– 81% agreed or strongly agreed. Students themselves indicated satisfaction
with MyGateway by a slightly higher
percentage – 82% were satisfied or greatly satisfied, while 18% were
dissatisfied.
Students
described more specific experiences in using MyGateway. They responded to
open-ended questions: "what are the
benefits of using MyGateway," and "how
can the use of MyGateway be improved for the course?"
The
responses of 175 students who listed personal benefits of using MyGateway were
categorized according to major themes. The following benefits are listed in
order of frequency.
Access to
assignments and grades anytime (32%)
Always know
what is going on in class (16%)
Availability
of online quizzes helped me learn content (10.5%)
Accessibility
of email to faculty anytime (8.7%)
MyGateway use
correlated to greater involvement and learning for me (7.7%)
Quicker
feedback on tests and exams (6.6%)
Discussion and
increased interactions with other students (6.6%)
Easier to work
and study at home (4.2%)
MyGateway is
easy to use (3.5%)
Useful links
(3.1%)
For
this listing, the responses from 164 students were categorized according to
major themes. The following suggestions for improvement and/or problem areas
are listed in order of frequency.
Had no
problems; no suggestions for improvement (13.6%)
More courses
should use; require faculty to use (10.5%)
Use more or
all of MyGateway capabilities (8.7%)
Initial start
up problems [e.g., passwords] (4.8%)
Not
"compelled" to use MyGateway (2.8%)
System not
always dependable; improve speed of connection (2.8%)
Faculty
members and students were on the fast track for adoption of MyGateway as a
course management tool and a means to enhance teaching and learning activities
at UMSL. A conservative estimate is that 150 instructors used the tool in
approximately 270 courses during Fall Semester 2000. Enthusiastic support for
MyGateway came from the students surveyed, in that 82 percent indicated they
would like MyGateway used in other courses. What implications are there for the
future? We would like to suggest these service areas for administrative
consideration.
The
need for faculty training will continue. Orientation to using MyGateway will be
needed for new users, while techniques for improving upon specific features
within the tool are being requested (e.g., small group team building; use of
student grade book for all course requirements). Creating online, interactive
learning modules and mentoring by successful online instructors might shorten
the learning curve for some faculty members and be more accessible to our
adjunct faculty. Faculty orientation to UMSL's teaching and technology
resources, policies and procedures might be included as one of the learning
modules.
Additionally,
effective online learning and teaching are needed areas of research.
Collaborating in research projects might be a logical, next step for the
support group.
Students
may increasingly be required to complete course requirements online. They will
need advice and help to obtain the requisite computer and Internet provider
service. Recommendations could be developed as an online supplement for faculty
to use with students. A crash course in use of MyGateway for students might be
routinely scheduled during the first two weeks of a semester, since that period
generates more problems.
Increasingly,
remotely located students enrolled in UMSL's online academic programs will need
online support services, too. Registration and financial aid offices, and the
bookstore could become involved in the online environment.
We
can leverage the knowledge and expertise of our faculty and staff by seeking
out potential collaborators and partners. There are more grants requiring
matching funds and partnerships than ever. A pool of money that could be tapped
for promising technology integration projects or access to skilled grant
writers would add to our abilities to develop these leveraged partnerships.
Questions
about the survey findings may be directed to Dr. Cheryl Bielema, Instructional
Development Specialist, 127 South Classroom Bldg., 8001 Natural Bridge Rd.,
University of Missouri, St. Louis 63121-4499. PH: 314-516-7134; EM:
bielema@umsl.edu. The evaluation team included Robert Keel, Blackboard
coordinator, Don Boehnker Director, Faculty Instructional Services, Larry
Pickett, Assistant Director, User Services, and Cheryl Bielema.
Information
Technology Services
January
18, 2001