Quality, Efficiency, and Course Design

Tom Henderson, Gary Brown , and Carrie Myers
Center for Teaching, Learning, and Technology
Washington State University
April 6, 2003

Two recent studies at Washington State University explored the impact of investing in up-front instructional design of online courses on the cost of their subsequent development and teaching, and on the quality of the resulting courses. The findings suggest that instructional design pays off in both ways.

EFFICIENCY THROUGH DESIGN - Study #1

During the fall 1999 term, the Washington State University (WSU) Distance Degree Program (DDP) delivered ten distance courses via the World Wide Web.  All but one of the courses used a web-based course management system developed at WSU called the Speakeasy Studio and Cafe. 

The Center for Teaching, Learning, and Technology (CTLT) along with the DDP began an activity-based costing (ABC) analysis of the costs of the courses using the Flashlight Cost Model (Ehrmann & Milam, 1999).  Just as the analysis was beginning the Washington State Higher Education Coordinating Board asked WSU to participate in the Technology Costing Methodology (TCM) developed by the Western Cooperative for Educational Telecommunications (WCET) and the National Center for Higher Education Management Systems or NCHEMS (Jones, 2001).  Several white papers on the TCM can be found on the WCET website at: http://wcet.info/projects/tcm/papers.asp).

The courses were developed by a multidisciplinary team consisting of the faculty member (or in some cases a faculty developer), DDP staff course developers, and staff from the WSU CTLT.  Four major activities were used; design, development, delivery, and assessment.  The average costs of each activity over the ten courses are summarized in Table 1.  Table 1 also includes the average total hours of work by the team to develop a course and the average number of students per course.

Table 1 - Average Course Cost by Activity and Average Hours and Students per Course

Key Finding #1

One of the key findings of this study was the relation of the resources spent designing a course to the costs developing and delivering the course.  Development costs were moderately inversely correlated to design costs (-.469) and cost to deliver a course was also moderately inversely correlated to design costs (-.502).  Our experience since this study has confirmed the inverse relation between design and the costs to develop, deliver, and maintain courses; the investment in design saves time and costs at later stages.

QUALITY THROUGH DESIGN - Study #2

The CTLT at WSU developed an online, formative assessment tool called GAPs for Goals, Activities, and Processes using CTLSilhouette which is the online survey system used to host Flashlight Online.  GAPs first surveys instructors asking what their goals are for a course and how they plan to assess those goals.  Students are then surveyed asking them what their course goals are, which methods of assessing their work they perceive to be most effective, and a series of questions designed to assess the learning processes taking place in the course.  Several of the questions on learning processes are based on Chickering and Gamson's Seven Principles of Good Practice in Undergraduate Education (1987).  Courses participating in GAPs during the Spring and Fall 2001 semesters were fairly diverse consisting of DDP courses that had gone through a design process, DDP courses that did not go through a design process, WSU Freshman Seminar courses (these courses met face-to-face but used a threaded discussion list fairly extensively during the course), other WSU courses (some used the web-based course management very extensively during the semester, some used it to augment their courses), and non-WSU courses. 

A regression analysis was used to analyze the answers to questions relating to the seven principles by the different groups.  The results of the regression analysis are summarized in Table 2.

KEY FINDING #2

The analysis of the GAPs data shows that courses that go through a formal, multidisciplinary design process score significantly better on questions relating to course processes.  

  Note: Several mini-reports of analyses on GAPs data can be found at: http://www.ctlt.wsu.edu/GAPS_Research_Links.asp.  The research relating to the above study is at: http://www.ctlt.wsu.edu/GAPs_SR7.asp. 

SUMMARY

The studies cited above indicate that a multidisciplinary design team's investment in course design is offset by reduced time and cost to develop, deliver, and maintain the course.  The investment in design pays off in improved course processes and course quality.  At WSU we are still in the process of studying and trying to improve this process.

William Massy has an interesting new book called Honoring the Trust: Quality and Cost Containment in Higher Education (Massy, 2003).  In an interview about the book by Tim Goral of University Business Massy says: "Quality and cost are two sides of the same coin."  (Goral, 2003).

REFERENCES

Boyd, Don. (2002, October). State spending for higher education in the coming decade.  Retrieved from the Internet 4/5/2003 http://www.nchems.org/State_Spending.doc. page 4.

Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education—Special Section pamphlet. The Wingspread Journal, 9 (2), 1-11.

Ehrmann, Stephen C., Joseph Lovrinic, and John H Milam, Jr. (1999). Modeling resource use in teaching and learning with technology, Version 1.0, The TLT Group, Washington, D.C.

Goral, Tim (2003. March). Quality is job one.  University Business.  Retrieved from the Internet 4/5/2003 .  http://www.universitybusiness.com/page.cfm?id=179.

Jones, Dennis. (2001). Technology costing methodology handbook - Version 1.0, WCET/WICHE, Boulder, CO.  Retrieved from the Internet 5/2/2003 . http://wcet.info/projects/tcm/proj-products.asp#Handbook

Massy, William F. (2003). Honoring the trust: quality and cost containment in higher education.  Anker, Bolton , Mass.

Rose, Gene and Wyatt Bill. (2003, Feb. 4).  State budget gaps growing at alarming rate according to new NCSL national fiscal report.  Retrieved from the Internet 4/4/2003 . http://www.ncsl.org/programs/press/2003/pr030204.htm.

BIBLIOGRAPHY

A reading list on activity-based costing in higher education can be found at the Cost of Networked Learning site at Sheffield-Hallam University 's web site.  The list is several years old now but it is very extensive.  http://www.shu.ac.uk/cnl/.