Comprehensive Evaluation of MyGateway Use by Faculty
and Students
Report of WS 2005
The MyGateway evaluation team
included Cheryl Bielema, Larry Irons,
For further information about the results,
please contact
Comprehensive
Evaluation of MyGateway Use by Faculty and Students
WS 2005 Executive Summary
The
following information details findings from a survey that collected data on
student assessments of MyGateway use in classes during Winter Semester
2005. The survey instrument is designed
to collect data regarding the computing resources available to students, the
degree of use made of MyGateway in select classes, and the perceptions of
students regarding the use of MyGateway in classes. We sampled 5% of the courses using MyGateway
(54), and included an additional targeted sample of courses using ITV, web-only,
or video-based instruction (75). High use classes were distinguished from
low use classes based on the
instructor’s use of the course site as well as student activity in the course
site. A total of 1107 students completed
the survey This represents 9.7% of the total number of students (11,416) taking
regular classes at UM-St. Louis during the winter semester 2005, and a survey
response rate of 31% for the sample. We
focused on the degree of use of MyGateway within classes to help us understand
how students in classes that use it more compare to those in classes that use
it less.
The
MyGateway survey has been done four times since the Fall 2001 semester. Each survey indicated that student satisfaction
and perceived benefits from MyGateway increase in courses where the system is
used more. Student self-reports in each of these surveys indicate an increased
engagement in courses (defined as spending more time on the course) in high use
classes. More significantly, each
iteration of the survey found that student intent to take additional classes
and complete their degree at UMSL is increased by use of MyGateway.
Survey
highlights:
- Sixty-nine
percent (69%) of students indicate they own a Pentium class or Macintosh
personal computer. Only 4.4% of the
students report not owning a personal computer.
- Nearly
71% of students access MyGateway primarily from off campus.
- Sixty-seven
percent (67%) of students report some form of off-campus, broadband Internet
access (DSL, cable modem, or work LAN).
This is up from 51% in FS 2003.
- Twenty-five
percent (25%) of students report spending three or more hours online each
week for the course surveyed.
- Nearly
65% of the students indicated they accessed MyGateway two or more days a
week for their course work.
- Learning
Activities--students
in classes with high use of MyGateway were more likely (p < .0004) to:
o
seek clarification when they did not understand something
(Question 9)
o
review lecture notes for clarification (Question 10)
o
receive instructor feedback quickly (Question 11)
o
spend more time studying (Question 15)
o
do better in their class (learn more) (Question 19)
o
find the use of online quizzes helpful (Question 24)
o
find access to course grades helped them stay more engaged
(Question 25)
o
report online discussions made them think more about course topics
(Question 26)
o
complete course assignments (Question 27)
- Student
Satisfaction--students
in classes with high use of MyGateway were statistically more
likely (p < .0004) to:
o
agree that they are very satisfied with the course (Question 20)
o
take another course at UMSL because of MyGateway (Question 30)
o
complete their degree at UMSL because of MyGateway(Question 31)
o
agree that they were more able to seek clarification (Question 8)
- Students in both high and low use classes agreed that they would like to
see MyGateway used in all of their classes (mean score: 2.31
and 2.25, respectively).
- Communication--students in classes with high
use of MyGateway were statistically more likely (p < .0004) to:
- communicate
with their instructor (Question 16)
- communicate
with other students in the course (Question 12)
- work in
online groups (Question 13)
- work in
face-to-face groups (Question 14)
- Benefits
to students in face-to-face, hybrid (mixture of class sessions and
online activity), or online courses
were similar to the previous year’s list.
- Immediate
access to grades and other course information and tracking individual
progress (N=406)
- Ability
to communicate with professors and classmates (N=172)
- Convenient
and expedient means to complete course work (N=118)
- Personal
organization (N=43)
- Time
and travel to campus saved (N=52)
- Connection
to campus strengthened (N=26)
- The most frequent response to a question about problems
experienced was “No Problem” (N=335). Problems noted by students included:
o Log-in and slow loading of MyGateway (N=55)
o Teachers not keeping course site up-to-date or making full use of the features (N=42)
o Inconsistent use of MGW and posting of documents (e.g., without dates or reference to course topics/assignments) makes for confusion (N=32)
o Navigating MGW is time-consuming (N=12)
Overall Impact of Online Materials and Learning Activities
Faculty and students continue to make significant strides
incorporating MyGateway into the learning process. The use of MyGateway is leading to changes in
how students and instructors define their respective roles in the
classroom. Student perceptions, attitudes, and behaviors documented in this
survey, and the consistency of these findings over the previous four years, highlight
MyGateway’s contribution to furthering strategic campus goals:
·
Recruitment
·
Retention
·
Student engagement
·
Community building
Figure A: Courses Using MyGateway
The number of courses using MyGateway has increased each year.
However, the number of courses making extensive use of MyGateway has not
increased at the same pace as those categorized as "low-use" courses. Although
our survey analysis indicates a significant correlation between MyGateway use and
student satisfaction, engagement, and retention, our efforts to encourage
faculty to explore the full potential of Mygateway needs to be extended.
Technology in the
Classroom
The following recommendations, drawn from student responses,
are offered as a means of continuing to insure student acceptance of MyGateway
and to promote effective use of campus online resources.
·
Deliberate use of MyGateway by instructors – integrating
the online features of the system into course requirements to avoid perception
of "add-on" work
·
Providing students with information on how to access campus
computing resources when they experience problems
·
Answering criticisms when MyGateway is perceived as not being
appropriate for certain courses
·
Blending Internet-based, asynchronous learning activities
with face-to-face experiences as a means of increasing student engagement
Behavior and
Perception
·
Students frequently commented that knowing their grades and
current standing is a benefit of using MyGateway
·
MyGateway is increasingly characterized as a learning aid,
making education accessible and providing “flexibility”
·
MyGateway enables a sense of belonging with the campus,
keeping up with events, programs and news
·
A small fraction of students indicated they don’t like to use
MyGateway
Implications and recommendations drawn from the data continue
to guide technology planning and support provided by Information Technology
Services and UM-St. Louis academic administration. Future support and training will focus on
those things that will help improve our students’ learning outcomes and
satisfaction in blended learning classes.
Additionally, insights drawn from the data can help UM-St. Louis
continue to refine the blend of traditional face-to-face classroom experience
with online resources.
Comprehensive Evaluation of MyGateway Use by Faculty
and Students
Report
of WS 2005
Purpose
The ongoing evaluation of technology-enhanced credit courses,
making use of the course management tool, MyGateway, documents the experiences
of two key stakeholder groups: enrolled students and teaching faculty. The questions and subsequent analyses here
identify essential support strategies for faculty who use technology to improve
learning experiences for students.
Objectives include:
·
Documentation of improvements in UMSL students' computing
capabilities and sophistication,
·
Documentation of the use of MyGateway features by selected
faculty in WS 2005, and,
·
Demonstration of indicators of optimal teaching and learning
interactions between and among faculty and students.
Methodology
A
stratified random sample (5 percent of "active" courses within
MyGateway) was selected according to levels of faculty use of the course
management tool MyGateway [total courses=1,080]. The number of courses randomly selected=54;
those not participating=4. An additional targeted sample
consisted of the total population of MyGateway courses involving ITV and
Internet classes (N=75). This semester
we measured daily instructor access of 12 key content and administrative areas
along with student activity. Measurement
was taken on March 8, 2005. Courses with an instructor ranking above
the mean were considered high use courses, and courses with an
instructor ranking below the mean were considered low use courses. Faculty members were contacted for
participation and scheduled for in-class completion of the questionnaires
during the evaluation period, April 4-May 13, 2005.
Completed
surveys by undergraduate and graduate students numbered 1107 (31% response
rate) [approximate enrollment in sample=3597, including 988 in the random
sample and 2,609 in the targeted sample].
Low use courses numbered 54 (50 participated); high use
courses numbered 75 (all participated).
Students responding in high use courses numbered 779, while 328
students in low use courses responded.
The response rate was substantially lower for the online surveys 493
(19%) than for face-to-face surveying 614 (62%).
Questions
in the MyGateway Student Survey focused on:
- Computing and Internet access
- Engagement in learning
activities in MyGateway courses
- Student satisfaction with
courses using MyGateway
- Communication between students,
and between students and teachers
Computers and Access
for Students
Technology requisites
are a computer and Internet access.
Students were asked to check primary computer access and capabilities
they had available. Types of Computer,
Primary Access Location, and Type of Internet Connection are the three tables
that follow.
The survey
has been done four times between Fall 2001 and Winter 2005. Over each iteration
an increasing percentage of the respondents have reported owning a newer
computer (e.g., Pentium). Over 69% in
Winter 2005 compares to just over 65% in the prior semester. Those owning a Macintosh have not varied much
from semester to semester -- within the range of 2.4%-4.4%. Only 4.4% of the students indicated they did
not own a personal computer.
The percent
of students not owning a personal computer has shown a trend down from semester
to semester. The range is a high of
16.5% in the first semester of surveying, FS 2000, to 4.4% in the current
survey.
Figure 1: Types of Computers Owned by Students
Students in
the High Use and Low Use courses had very similar access to the
Internet. Over time, there has been a steady increase in access from home and
work, with a slightly lower percentage using on campus computer labs.
Figure 2:
Primary Access Location
If a student’s primary
access was from off campus, they were asked specifics about the type of
Internet connection: dial-up telephone via modem, dial up with DSL, cable
modem, or other. See Table 3 for
connection details of both groups.
Broadband
connections to the Internet (e.g., DSL and cable modems) have increased
overall, with DSL outpacing cable modems in both groups. Slightly over half of this student sample has
broadband capabilities (67%), compared to 50.7% in the student sample one year
ago.
Figure 3: Type of Internet Connection to UMSL Servers
This semester, we
asked students to comment on how much time they spent working online for the
course in which they were surveyed.
Students in High Use courses indicate that they spend more time
online for their courses.
Time
spent online (q8) (WS05 marks the introduction of this question)
|
Percent of Students in Low Use Courses
|
Percent of Students in High Use Courses
|
Percent of all Students
|
Less than 1 hour per week
|
67.1%
|
43.2%
|
51%
|
1-3 hours per week
|
21.1%%
|
24.8%
|
23.6%
|
3-6 hours per week
|
9.0%
|
23.8%
|
19%
|
6-10 hours per week
|
1.7%
|
5.5%
|
4.3%
|
More than 10 hours per week
|
.7%
|
2.7%
|
2%
|
No answer
|
.3%
|
0%
|
.1%
|
Table 1: Time Spent
Online for Surveyed Courses
We also
asked students about their perceptions of their use of MyGateway for the class
in question. Students in High Use
courses are nearly twice as likely as students in Low use course to report accessing MyGateway three or more days per
week.
Frequency
of Mygateway Use (Q5) (WS05 marks the
introduction of this question)
|
Percent of Students in Low Use Courses
|
Percent of Students in High Use Courses
|
Percent of all Students
|
Everyday
|
6.2%
|
16.9%
|
13.7%
|
3-4 days a week
|
24%
|
43.6%
|
37.8/%
|
1-2 days a week
|
33.8%
|
29.1%
|
30.5%
|
Less than once a week
|
35.4%
|
10.4%
|
17.8%
|
No answer
|
.6%
|
0%
|
.2%
|
Table 2: Student Perceptions of MyGateway Use
Learning With MyGateway
We examine the degree to which level of use (high or low) of MyGateway predicts student perceptions over the following dependent variables: Learning Activity (questions 9, 10, 11, 15, 19, 24-27), Student Satisfaction (questions 20, 28-31), Student/Teacher Communication (questions 11, 16, 17), Student/Student Communication (questions 12-14), and Perceived Learning Outcome (questions 19, 24). [A copy of the survey instrument is attached as Appendix A beginning on page 19 of this report]
Level of Use and MyGateway Student Evaluations
The
survey items allow us to analyze differences between students in classes with
low use of MyGateway and students in classes with high use. Our working hypothesis is that increased
exposure to classes using MyGateway results in higher degrees of learning
activity, satisfaction, and student/teacher communication.
Every
Learning Activity item measured by the MyGateway survey indicates that students
in low use classes are more likely (p < .0004) to disagree
with positive statements about learning activities in MyGateway classes. Thus, students in classes with high use
of MyGateway were more likely to seek clarification when they did not
understand something (Question 9), review lecture notes for clarification
(Question 10), receive instructor feedback quickly (Question 11), spend more
time studying (Question 15), do better in their class (learn more) (Question
19), find the use of online quizzes helpful (Question 24), find access to
course grades helped them stay more engaged (Question 25), suggest online
discussions made them think more about course topics (Question 26), and
complete course assignments (Question 27). The responses to this cluster of questions
documents the significant role of effective use of MyGateway in promoting
student engagement in their courses.
Use Of MyGateway In Relation To Learning Activity
|
|
high/low |
N |
Mean |
Std. Deviation |
Std. Error Mean |
|
q9 |
L |
311 |
4.19 |
2.478 |
.141 |
|
H |
732 |
3.38 |
2.214 |
.082 |
|
|
q10 |
L |
287 |
4.63 |
2.636 |
.156 |
|
H |
698 |
3.48 |
2.356 |
.089 |
|
|
q11 |
L |
305 |
4.49 |
2.727 |
.156 |
|
H |
739 |
3.17 |
2.334 |
.086 |
|
|
q15 |
L |
302 |
5.04 |
2.382 |
.137 |
|
H |
723 |
3.86 |
2.227 |
.083 |
|
|
q19 |
L |
308 |
4.31 |
2.421 |
.138 |
|
H |
742 |
3.04 |
1.996 |
.073 |
|
|
q24 |
L |
280 |
4.72 |
2.730 |
.163 |
|
H |
660 |
3.74 |
2.612 |
.102 |
|
|
q25 |
L |
314 |
3.22 |
2.626 |
.148 |
|
H |
754 |
2.26 |
1.871 |
.068 |
|
|
q26 |
L |
285 |
5.01 |
2.713 |
.161 |
|
H |
714 |
3.67 |
2.528 |
.095 |
|
|
q27 |
L |
308 |
3.59 |
2.309 |
.132 |
|
H |
742 |
2.83 |
2.002 |
.074 |
Table 3: Group Statistics-Learning Activity
Independent Samples Test
|
Question |
t |
df |
Sig. (2-tailed) |
Mean Difference |
Std. Error Difference |
|
q9 |
5.186 |
1041 |
.000 |
.806 |
.155 |
|
q10 |
6.716 |
983 |
.000 |
1.149 |
.171 |
|
q11 |
7.891 |
1042 |
.000 |
1.319 |
.167 |
|
q15 |
7.561 |
1023 |
.000 |
1.178 |
.156 |
|
q19 |
8.803 |
1048 |
.000 |
1.271 |
.144 |
|
q24 |
5.228 |
938 |
.000 |
.987 |
.189 |
|
q25 |
6.742 |
1066 |
.000 |
.960 |
.142 |
|
q26 |
7.389 |
997 |
.000 |
1.337 |
.181 |
|
q27 |
5.376 |
1048 |
.000 |
.764 |
.142 |
Table 4: Learning
Activity Items (independent samples t-test)
Use of MyGateway in Relation to Satisfaction
Four
items (questions 20, 28, 30, 31) related to student satisfaction with courses
using MyGateway indicate that students in low use classes are more
likely (p < .0004) to disagree with positive statements about their
satisfaction with MyGateway classes. Responses to question 29 are uniformly
skewed towards agreement with the statement (“I would like to have MyGateway
used in other courses”) regardless of degree of use. This finding follows the trend
we noted in our analysis of last year’s survey.
In all the previous surveys, student satisfaction was predicted by use
level as well.
|
Question |
high/low |
N |
Mean |
Std. Deviation |
Std. Error Mean |
|
q20 |
L |
311 |
4.17 |
2.459 |
.139 |
|
H |
745 |
2.84 |
2.007 |
.074 |
|
|
q28 |
L |
305 |
5.07 |
2.141 |
.123 |
|
H |
738 |
4.49 |
2.279 |
.084 |
|
|
q29 |
L |
314 |
2.25 |
1.842 |
.104 |
|
H |
751 |
2.31 |
1.820 |
.066 |
|
|
q30 |
L |
307 |
3.77 |
2.446 |
.140 |
|
H |
735 |
3.16 |
2.238 |
.083 |
|
|
q31 |
L |
287 |
3.82 |
2.392 |
.141 |
|
H |
692 |
3.24 |
2.199 |
.084 |
Table
5: Group Statistics-Satisfaction Items
|
Question |
t |
df |
Sig. (2-tailed) |
Mean Difference |
Std. Error Difference |
|
q20 |
9.205 |
1054 |
.000 |
1.336 |
.145 |
|
q28 |
3.767 |
1041 |
.000 |
.574 |
.152 |
|
q29 |
-.419 |
1063 |
.675 |
-.051 |
.123 |
|
q30 |
3.885 |
1040 |
.000 |
.607 |
.156 |
|
q31 |
3.657 |
977 |
.000 |
.580 |
.158 |
Table 6: Satisfaction
Items (independent samples t-test)
Students in
high use courses were more likely to agree with question 20 (“I am very
satisfied with this course because it used MyGateway”). This finding confirms previous responses to
the question. For question 28 (“It takes
more effort to complete a course that uses online resources like MyGateway” we
see a continuation of a trend first noted in the Fall 2003 survey with students
in low use classes being more likely
to disagree with the negative statement.
For questions 30 and 31 however, a pattern from the fall 2002 survey
continues; students in high use
courses were more likely to agree with question 30 (“I am more likely to take
another course at UMSL because it uses MyGateway) and with question 31 (“I am more likely to complete my degree at UMSL because of
the advantages of using MyGateway in my courses”). In other words, students in high use
courses are motivated by their degree of use of MyGateway to take other courses
and complete their degree program.
Responses
to question 29 (“I
would like to have MyGateway used in all my courses”) do not signify a
distinction between the two groups. Students both high and low use classes indicate a preference for
having MyGateway used in all of their classes (means: 2.25 and 2.31).
Use of MyGateway in Relation to Student/Teacher Communication
|
|
high/low |
N |
Mean |
Std. Deviation |
Std. Error Mean |
|
q11 |
L |
305 |
4.49 |
2.727 |
.156 |
|
H |
739 |
3.17 |
2.334 |
.086 |
|
|
q16 |
L |
315 |
3.54 |
2.370 |
.134 |
|
H |
740 |
2.81 |
1.999 |
.073 |
|
|
q17 |
L |
300 |
4.34 |
2.759 |
.159 |
|
H |
712 |
3.10 |
2.407 |
.090 |
Table 7: Group Statistics-- Student/Teacher Communication
|
Question |
t |
df |
Sig. (2-tailed) |
Mean Difference |
Std. Error Difference |
|
q11 |
7.891 |
1042 |
.000 |
1.319 |
.167 |
|
q16 |
5.141 |
1053 |
.000 |
.732 |
.142 |
|
q17 |
7.161 |
1010 |
.000 |
1.240 |
.173 |
Table 8: Student/Teacher
Communication Items (independent samples t-test)
The
three items related to student/teacher communication in courses using MyGateway
indicate that students in low use classes are more likely (p < .0004)
to disagree with positive statements about their degree of communication
with faculty in MyGateway classes. Thus, students in classes with high use
of MyGateway were more likely to agree that they were able to receive
instructor comments on assignments quickly (Question 11), communicate with
their instructor (Question 16), and (a continuation of a trend from Fall 2003)
receive feedback on quizzes and exams quickly (Question 16); the latter
point is indicative of the increasing use of online assessments in MyGateway.
Use of
MyGateway in Relation to Student/Student Communication
|
|
high/low |
N |
Mean |
Std. Deviation |
Std. Error Mean |
|
q12 |
L |
308 |
4.54 |
2.492 |
.142 |
|
H |
736 |
3.32 |
2.318 |
.085 |
|
|
q13 |
L |
286 |
6.03 |
2.175 |
.129 |
|
H |
688 |
4.26 |
2.655 |
.101 |
|
|
q14 |
L |
284 |
6.12 |
2.045 |
.121 |
|
H |
648 |
5.28 |
2.328 |
.091 |
Table 9: Group Statistics- Student/Student Communication
|
Question |
t |
df |
Sig. (2-tailed) |
Mean Difference |
Std. Error Difference |
|
q12 |
7.626 |
1042 |
.000 |
1.227 |
.161 |
|
q13 |
9.994 |
972 |
.000 |
1.775 |
.178 |
|
q14 |
5.239 |
930 |
.000 |
.837 |
.160 |
Table 10:
Student/Student Communication Items (independent samples t-test)
The three items related to student/student
communication in courses using MyGateway indicate that students in low use
classes are more likely (p < .0004) to disagree with positive
statements about their degree of communication with faculty in MyGateway
classes. Thus, students in classes with high use of MyGateway agreed that
they were more likely to communicate with other students in the course
(Question 12), work in online groups (Question 13), and work in face-to-face
groups (Question 14).
Use of MyGateway in Relation to Perceived
Learning Outcomes
|
|
high/low |
N |
Mean |
Std. Deviation |
Std. Error Mean |
|
q19 |
L |
308 |
4.31 |
2.421 |
.138 |
|
H |
742 |
3.04 |
1.996 |
.073 |
|
|
q24 |
L |
280 |
4.72 |
2.730 |
.163 |
|
H |
660 |
3.74 |
2.612 |
.102 |
Table 11: Group Statistics- Perceived Learning
Outcome
|
Question |
t |
df |
Sig. (2-tailed) |
Mean Difference |
Std. Error Difference |
|
q19 |
8.803 |
1048 |
.000 |
1.271 |
.144 |
|
q24 |
5.228 |
938 |
.000 |
.987 |
.189 |
Table 12: Perceived
Learning Outcome by Use (independent samples t-test)
Two items in the survey are designed to measure
student perceptions of learning outcomes.
Questions (19, and 24) indicate that students in low use courses are more
likely (p < .0004) to disagree with positive statements about their
learning outcomes, with students in high use courses more likely to agree with those same statements. Thus,
students in high use courses were more likely to agree with question 19 “Using
MyGateway in this class helped me do better (learn more)” and question 24 “The
use of online quizzes and tutorials in MyGateway helped me learn course
content” (a continuing pattern first seen in Fall 2003).
Discussion
The
data presented above indicate that increased use of MyGateway results in higher
estimates of learning activity, higher degrees of satisfaction, more
student/teacher communication, more student/student communication, and a
perception of increased learning outcomes.
Additionally, the data suggests that increased use of online assessments
and other activities contributes positively to student perceptions of
satisfaction and learning outcomes. The
data from the current MyGateway survey lend support to the conclusions reached
in the survey analysis of 2001, 2002, and 2003; the more students are
exposed to MyGateway the higher their estimation of its positive effects, and
the more likely they are to continue (and complete) their education at UM-St.
Louis. In fact, what we document
with this analysis is that the positive effects are not simply consistent based
on MyGateway use, but increase as the use of MyGateway increases in the
classroom and throughout the campus. High use of MyGateway results in:
- Increased reports of learning activities
(student engagement)
- Higher levels of student satisfaction with their
courses
- More Student communication with their
instructors
- More communication between students
- Higher levels of perceived learning outcomes
Benefits of Using
MyGateway
Seven-hundred
and fourteen (444 students from High use
classes, and 270 students from Low use
classes) students provided written comments concerning their perceptions of
MyGateway use in class and on campus.
Students described specific experiences using MyGateway in response to
open-ended questions: “what are the benefits of using MyGateway,” and “did you
have any problems using MyGateway?” The
research team established categories of response based on a thematic analysis
to isolate similarities and differences in the perceived benefits and problems
reported by the students. Manual counts
of comments were done for each category.
Responses applicable to more than one category were counted in each.
Figure 4: Benefits of
using MyGateway
Access to
grade information provides students with critical information that allows them
to track their status in a particular course.
This extends a trend from the Fall 2003 survey where “knowing grades”
ranked at the top of the perceived benefits list. Students are aware of the value of MyGateway
in keeping them informed as to their progress in a course. This information can allow students to modify
work habits in order to maximize their performance in their courses.
Benefits, in their
own words
·
I stay more involved with all of my classes and it
helps me know exactly where my grades are.
·
Being able to view grades helps a lot to motivate
me...I can easily chart my progress through the grade book.
·
I think it is necessary to have MyGateway because it
plays a huge role in keeping students connected with their courses.
·
I can send emails to the teacher and the other students
very easily and the discussion boards foster open dialogue.
·
I have really benefited from using MyGateway. I work
and have 3 busy children. I feel this is a wonderful way to continue my
education and take care of my family.
·
[MyGateway] is a centered place to access documents and
grades for all my courses.
Problems in Using My Gateway
The most
frequent response to the question, “Did you have any problems using MyGateway?”
was “No Problem” (High use N=222; Low use N=113). Students listed fewer problems than
benefits. Problems noted among this
sample of students included:
Figure 5: Problems
using MyGateway
Problems, in their own
words
- Only problem I have using MyGateway
is not all teachers use it correctly. Most post assignments and all
information [in the same place] instead of the appropriate location.
- Feel MyGateway is under used
by most teachers. Would like more online class work.
- ...Ex., one uses communication
features; another uses it for lecture notes and assignments; while, a
third doesn’t use MyGateway at all.
- [Internet Service Provider or
the MyGateway system] has gone down in the middle of taking online
quizzes.
- Busy signals when dialing in
(sometimes have to dial 5+ times)
- I think that the instructor
should be informed to use MyGateway grade book...it helps you know where
you stand.
Ideas for Improvement
Students
provided suggestions for improvement as they commented about the benefits and
problems they had experienced.
Improvement ideas, overall, centered on increasing use of MyGateway by instructors (n= 42) for both
groups. Another theme voiced by students
in both High and Low use classes was “inconsistent or inappropriate use of
MyGateway” by their instructors (N=32).
Consistent use of MyGateway features, how best to organize course
documents, and the essential nature of the grade book – all were mentioned by
multiple students. In addition:
- Have assignments we can actually do [online], practice
test or quizzes.
- I think the instructor should be informed to use
MyGateway grade book...
- The MyGateway system is beneficial to me if the faculty
use it consistently. It get on my nerves when they say they are going to
post something...and it takes them a week to do it.
Several students
commented that in one way or another MyGateway
helped organize their learning efforts.
Perhaps a MyGateway users’ guide should include advice for helping
students and faculty organize materials and assess priorities. Following are representative comments:
·
School matters remain separate and more organized.
·
Keeping track of assignments/ announcements’ changes
in the syllabus.
·
Accessing chapter tests...helps you understand the
course information.
Several
students suggested that a real plus to them was that MyGateway increased feeling of connection to other students, their
teachers, and campus activities/resources in general (N=26) – this was
particularly important to online and remotely-located students where physical
access to campus was limited or impossible.
- Easy access to what is happening at UMSL. Being an
evening student I don’t know how I would survive on campus without it.
- It helps to connect people.
The following are statements as to how use of MyGateway
supported learning for
respondents. The most often mentioned
learning supports included 24x7 accesses to course materials, use of groups,
discussion board, file exchange, and grade book. The importance of determining standard
instructional practices related to use of the course management system was
stated repeatedly in comments of students.
- I can check course announcements, syllabus,
assignments, professor email, students’ email, and grades from anywhere
with Internet including work and home and while traveling.
- This class set up group discussion board for our
projects and it has helped us stay connected.
- The features I use most are email, group file exchange
and retrieving information. I found that it was a great help for me in all
my courses.
- MyGateway allowed me to stay on track if I missed a
class.
- This was the most MyGateway-involved class I’ve taken,
and it really enhanced class participation.
Impact of Online Materials and Learning Activities
Faculty and students at UM-St. Louis have made significant
strides toward incorporating MyGateway into the learning process. The use of MyGateway is leading to changes in
how students and instructors define their respective roles in the classroom,
and online learning components offered via MyGateway are exerting a positive
influence on students’ perceptions.
Following are several implications drawn from the current data.
Academics: Perception and Behavior
·
Knowing their grades and current standing leads students to
modify their study habits—the ready availability of course related materials on
MyGateway allow students to follow through on these changes at their own pace.
·
MyGateway is increasingly characterized as a learning aid,
making education accessible, convenient, and flexible. It enhances communication and helps students
track their individual progress.
·
MyGateway creates a sense of belonging across the
campus. Not only are students keeping up
with events, programs and news, but they are coming to view themselves as part
of a “learning community.”
·
A few students consider class attendance “optional” with the
availability of back-up class materials on MyGateway. We would expect this trend to continue as
instructors offer key resources online.
This signals the beginning of a shift in what students and faculty
perceive to be the relevance of face-to-face interaction, and offers an
opportunity for instructors to rethink the structure of classroom interaction.
Technology in the Classroom
·
Deliberate and consistent use of MyGateway and better
communication concerning how MyGateway supports learning would allay
perceptions that MyGateway is an add-on to the course requirements; creating
more work without learning pay-offs.
Students articulated their concern in terms of ‘time loss’ as, “Mandatory use was frustrating due to time
constraints” and “[a problem is] the
time I feel it takes away from the learning time.”
·
Successfully accessing online materials and campus computing
resources – using computers at convenient times, logging on, staying connected,
and downloading files -- all are important to the learning process. There are still students without the
necessary computing capabilities, hardware, or personal computing skills,
although the number of such students is steadily decreasing.
·
There are students who are critical of MyGateway and say that
professors are too dependent on MyGateway, too lazy to teach. A representative comment is “Some courses relied too heavily on
information posted instead of teaching the material.” There are other students who state a
preference for face-to-face instruction or don’t think the course, as designed,
works well online.
·
Classrooms have changed.
Internet-based, asynchronous learning activities to be completed outside
the classroom are becoming the norm.
Students self-report more time
spent on course materials and activities when MyGateway is used more intensely and often in their
classes, for the past several semesters.
Recommendations
Analysis of the data has exposed several levels of responsibility for increasing the efficiency of the course management system and for ensuring greater student satisfaction and success. Orientation materials and technical support guidelines must be updated to reflect the evolving nature of the Internet and newly available technologies, and be made available to both faculty and students (e.g., wireless zones and instructions for configuring laptops).
In addition, the positive effects of using MyGateway, as increasingly substantiated by our students, might be the topic for curricular discussions in Departments and Colleges at this University. The unique requirements of disciplines could be factored into the specific recommendations for MyGateway integration into courses.
Student Support
Services Suggested
Knowing the technological requirements of a course, as
well as how to obtain assistance when problems occur, are important to
students’ success. Preparing additional
help documents and involving instructors in distributing a basic introduction
to MyGateway or pointing their students to web-based assistance seem
advisable. How to use wireless
connections around campus was a need appearing for the first time in this
study.
The current data reveals that students are expected to access, and print, a variety of documents for use in their courses. Many dial-up users are experiencing slow or interrupted access to MyGateway course sites This seems to be particularly problematic for dial-up users accessing large files, or who have limited periods of inactivity from their private Internet service providers. Instructors need to be cognizant of not only the educational value of materials and activities in an on-line environment, but also the technological requirements that enable all students to participate actively in the on-line environment.
There were suggestions for system improvements from survey
respondents:
·
Quick links to the most frequently used features (e.g., grade
book)
·
Ability to attach more than one file to a Discussion Board
posting
·
Option to receive automated email notification when
instructor posts an announcement in MyGateway.
Although the MyGateway system is currently in the process of
being upgraded and enhanced for the Fall 2005 semester, it would be relevant to
provide students, faculty, and staff a specific mechanism for providing usability
feedback throughout the year.
Accessing
MyGateway and other technical resources have become increasingly vital to
student success and satisfaction with UM-St. Louis. Issues such low bandwidth, Internet
connection stability, inconvenient or no computer access, and technical
expertise are creating a “have-have not” disparity among our students. It will become increasingly important for
administrators to deal with equalizing the access among students. Faculty must factor in the lowest common
denominator for online instructional components. Further, to insure students are adequately
prepared to take advantage of the possibilities offered by an increased level
of online learning activities, key information and skills must be available:
- Computer literacy courses to
insure student ability to participate fully in the online learning
environment,
- Publication of minimum computer
configurations to enable access to online campus resources,
- More detailed course
descriptions in the campus bulletin and course schedule to inform students
of required technology and skills necessary for online learning
activities.
Instructional
Improvements Suggested
Students
continue to request most or all MyGateway features be used in all their
courses. Depending on the subject matter,
integrating more of the MyGateway components would help increase
instructor-student interaction and increased interaction among students,
according to this and previous student surveys.
Instructional
improvements gleaned from the survey analysis include the following:
- Syllabi and assignments should
be available on the individual course web sites. Further, the syllabus ought to include
specific expectations for students’ online participation, guidelines to
how an instructor intends to use MyGateway, as well as details for
accessing technical assistance. A
syllabus template is posted for faculty review on the Center for Teaching
and Learning web page, http://www.umsl.edu/ctl.
- Descriptions for effectively
and efficiently using discussion boards, quizzes, and virtual chats seem
advisable to revise and/or publicize, in light of students’ questions
about their use.
·
Departments might set standards for the course management
system: what features will be consistently utilized and create student support
documents that specify minimum computing capabilities and overall technology
learning objectives. An excellent
example is the Department of Communication Online Course Handbook. See http://www.umsl.edu/divisions/artscience/communication/files/onhand.pdf. Specific topics for departmental handbooks
might include: expectations for online work, suggestions for how-to-learn
effectively online, and descriptions of the supplemental materials and online
resources available for individual students and new faculty wishing to
incorporate technology and the Internet in their teaching.
·
Providing basic online student services to facilitate
communication, advising, enrollment and business transactions with the
university are in progress. Gathering
input as to what, where, and how additional support services could be developed
with existing technology resources is the logical next step in meeting the
challenge of no new or decreased funds.
Answers to the ideal
blending of face to face and online learning question will impact the
selection of technology. The next
generation of technologies, such as hand-held computing devices and cellular
internet connectivity, has already become a reality for many. Technology review teams could include faculty
and student representatives, in addition to the technicians. Pilot studies contributing to the
“scholarship of teaching and learning” are worthwhile investigative activities
for both technology administrators and academicians.
Campus Goals
Student
perceptions, attitudes, and behaviors documented in this survey, and the
consistency of these findings over the previous four years, highlights
MyGateway’s contribution to the strategic campus goals of:
·
Recruitment
·
Retention
·
Student engagement
·
Community building
Implications and recommendations drawn from the data will
continue to guide technology planning and support provided by Information
Technology Services and UM-St. Louis academic administration. Future support and training will focus on
those things that will help improve our students’ learning outcomes and
satisfaction. In addition, fuller
distance delivery of some programs will enhance our institution’s ability to
meet the needs of our broad constituency.
Appendix A
University of Missouri-St.
Louis
WS 2005 MyGateway Course
Survey
University of Missouri-St. Louis
MyGateway WS2005 Course Survey
Course name and
number:_____________________________
Name of
Instructor__________________________________
Please complete the survey based on your experiences with MyGateway, the web course management
tool. Most of the questions ask for one response (based on your priority or
primary experience). Your responses are anonymous.
Your participation is completely voluntary, and you are free to refuse to
answer any questions. UM-St. Louis Information Technology Services thanks you
for your participation. Your input will be useful in planning student support
and developing new technologies.
1. My primary location for accessing the
Internet for educational purposes is (mark
only one)
- On
Campus (labs, library, MSC, and/or classrooms)
- Student
Housing (Dorms, Meadows,
- At
Home
- At
Work
- Other
2. What kind of computer do you use at home?
- 486
or older PC
- Pentium
or newer PC
- Mac
- Other
- None
3. How long do you usually wait for a computer at
UMSL computer labs?
- 0-5
minutes
- 6-15
minutes
- 16-30
minutes
- More
than 30 minutes
- I
don’t use UMSL computer labs
4.
If
you connect to the Internet from off-campus, what type of Internet connection
do you have?
A.
Dial-up
modem
B.
DSL
C.
Cable
Modem
D.
Work
LAN
5.
Which of the following statements best characterizes the way you used
(accessed) MyGateway in this class?
A. I used (accessed) MyGateway every day for
this class
B. I used (accessed) MyGateway 3-4 times a week for
this class
C. I used (accessed) MyGateway 1-2 times a week
for this class
D. I used (accessed) MyGateway once a week or
less for this class
6.
Which of the following statements best characterizes the way your
instructor used MyGateway in this class (for posting grades, announcements,
documents, and/or communicating with the class)?
A. My instructor used MyGateway just about every
day
B. My instructor used MyGateway 3-4 days a week
C. My instructor used MyGateway 1-2 days a week
D. My instructor used MyGateway less than once a
week
7. What are
the most common features of MyGateway used in this class (check all that
apply)?
A. Announcements and
email
B. Discussion Forums
C. Groups
D. Text Documents
(Syllabus), PowerPoint Presentations, and Web pages (lecture notes, readings,
etc.)
E. Multimedia
resources (movies, images, and sounds)
F. Quizzes, Exams and
Tests
G. Grade Book
H. Calendar and Tasks
I. Chat and the
virtual classroom
J. Centra (real-time
internet and audio connection, and class presentation recording)
8.
How much time did you spend online each week for this course?
Less
than 1 hour 1-3 hours 3-6 hours 6-10 hours More
than 10 hours
A B C D E
9. Using MyGateway in this class
made me more likely to seek clarification when I did not understand something.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
10. Using MyGateway in this class
made me more likely to review lecture notes to gain clarification.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
11. Using MyGateway in this class
made it more likely for me to receive instructor comments on assignments
quickly.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree Apply
12. Using MyGateway in this class
made me more likely to communicate with other students taking this course.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
13. Using MyGateway in this class
made me more likely to work in online groups.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
14.
Using MyGateway in this class made it more likely for me to work in
face-to-face groups.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
15. Using MyGateway in this class
made it more likely I would spend time studying.
A
B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
16. Using MyGateway in this class
made it more likely for me to communicate with my instructor.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
17.
Using MyGateway in this class made me more likely to receive feedback on
quizzes and exams quickly.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
18. The instructor encouraged us to
use MyGateway in this course.
A B C D E F G H
Strongly Strongly Does not
Agree Disagree
Apply
19. Using MyGateway in this class
helped me do better (learn more).
A B C D E F G H
Strongly Strongly Does not
Agree Disagree
Apply
20. I am very satisfied with this
course because it used MyGateway.
A B C D E F G H
Strongly Strongly Does not
Agree Disagree Apply
21. Please provide any additional
comments you might have on the way MyGateway was used in this course:
Now we would like you to respond to the following questions and
statements concerning MyGateway use here at UMSL, in general:
22.
How many courses are you taking this semester?
A:
1 B: 2 C: 3 D: 4 E:
5 F: 6 G: 7 or more
23.
How many of the courses you are taking this semester use MyGateway?
A:
1 B: 2 C: 3 D: 4 E:
5 F: 6 G: 7 or more
24.
The use of online quizzes and tutorials in MyGateway help me learn
course content.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
25.
Being able to access course grades via MyGateway helps me stay engaged
with my course work.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
26. The online discussions in
MyGateway make me think more about course topics.
Strongly Strongly Does not
Agree Disagree
Apply
27. Using MyGateway makes it more
likely for me to complete course assignments.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
28. It takes more effort to complete
a course that uses online resources like MyGateway.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
29.
I would like to have MyGateway used in all my courses.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
30. I am more likely to take another
course at UMSL because it uses MyGateway.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree
Apply
31. I am more likely to complete my
degree at UMSL because of the advantages of using MyGateway in my courses.
A B C D E F G
H
Strongly Strongly Does not
Agree Disagree Apply
We are interested in your specific experiences and suggestions for
improving the use of "MyGateway"
at UM-St. Louis. Please answer Questions 32 and 33 with as much detail as you
want. Use additional paper if needed.
32. The personal benefits to me in
using MyGateway are:
33. Did you have any problems using
MyGateway? (If you had problems, please
describe them briefly).
Thank you
for your responses!
MYGATEWAY Support
Information
Technology Services