Using Student Feedback to Get More Value from ePortfolios

Handbook and Other Materials l Asking the Right Questions (ARQ) l Training, Consulting, & External EvaluationFAQ

1. Select Activities l 2. What other ingredients needed?  l 3. Monitor Activities l 4. Debug Activities l 5. Diagnose Barriers to Participation l 6. Control Costs l SummaryAttachment: List of Activities l
Part II: Using Student Feedback to Improve ePortfolio Activities l Flashlight Evaluation Handbook Table of Contents

These materials are for use only by institutions that subscribe to The TLT Group, to participants in TLT Group workshops that feature this particular material, and to invited guests. The TLT Group is a non-profit whose existence is made possible by subscription and registration fees. if you or your institution are not yet among our subscribers, we invite you to join us, use these materials, help us continue to improve them, and, through your subscription, help us develop new materials!  If you have questions about your rights to use, adapt or share these materials, please ask us (info @ tltgroup.org).

Revised November 2, 2008

"Formative Evaluation"

If you know what we mean by 'formative section', you can skip to the next section.  "Formative evaluation" of ePortfolio use is an intentional inquiry designed to produce insights that will enable authors and other stakeholders to get more value from the ePortfolio-support activity. In contrast, "summative evaluation" is information to help stakeholders decide what the benefits and costs of that activity have been: "how are we doing?" and "how did we do" (summative) in contrast to "how can we do better?" (formative)

For example, imagine a professor using ePortfolios to encourage students to think more consciously about what they have been learning in order to deepen that learning and plan for future learning. The faculty member calls this activity "reflection." 

  • A formative question might be to ask students to describe their own definition of reflection and to describe how reflection has influenced their own thinking about what to study next. 
  • Another formative question might be to ask students if they've had trouble uploading their reflections into the ePortfolio.

Student answers to each of those questions could help the faculty member decide what to do next in the course, or how to help individual students to do better in the course.

Student Surveys for Formative Evaluation - Feedback for the Instructor

Step 1 for a program is to encourage and help instructors use student feedback to figure out how to fine tune their use of ePortfolios in a course.  (See below for some ideas for questions.) Once there's some base of instructors who have found it useful to get student feedback, you may be ready to go to the next stage: a collaboration among leaders of the ePortfolio initiative with individual faculty, designed to help both groups get valuable feedback.

Student Surveys for Formative Evaluation - Feedback for the Instructor AND for the ePortfolio Initiative

Overview: We suggest a pluralistic approach to formative evaluation: one that gives participating faculty and administrators to ability to a) contribute questions to the total inquiry, b) ask students only those questions relevant to what that student has actually been doing with ePortfolios, c) give people reports on only on those questions that are most useful for them. This approach uses a new kind of feedback system called a "matrix survey."  

For this discussion, let's consider just three of the major activities for which ePortfolios are sometimes used and for which student feedback could be informative.

  • Deepen learning via reflection (e.g., reflection on how the work itself, sometimes in combination with other artifacts, provides evidence of capability; reflection on development of a capability)

  • Deepen learning, and relationships, by getting more kinds of people to assess the studentís learning.

  • Selecting work (and assessments), as evidence of personal capability, for job and grad school applications.

Maybe no one course at a university would use an ePortfolio to support all three of those activities (and there are, obviously, others in addition to these three). In fact most courses only use ePortfolios for a couple such activities. 

So can you create a formative evaluation survey that serves both the individual instructors and also the staff who support many such instructors.  One answer: use a matrix survey. 

What's a Matrix Survey?

Think of the following grid (matrix) as a responses from faculty who are describing courses in which they use ePortfolios and for which they would like feedback from their students. Column B contains an X in each row where faculty have reported that ePortfolios are used for reflection. Column C is checked for each course where faculty reported that ePortfolios were used as a vehicle to receive feedback from peers, professionals in the field, or others ('multiple sources'). And Column D is checked if faculty want student feedback on their efforts to use ePortfolios to apply for jobs or further schooling.  With a matrix survey created with Flashlight Online 2.0, students in each class can receive a different response form, with a different mix of questions.   Column A represents a group of questions that could be asked of all courses, regardless of how ePortfolios are used there.

  A B C D
Course1 x x    
Course2 x   x  
Course3 x   x  
Course4 x     x
Course5 x x    
Course6 x x   x
Course7 x      
Course8 x x   x
Course9 x   x  
Course10 x   x  
Course11 x x   x

 

A Prototype Matrix Survey on ePortfolio Use by Students

We have created a small proof of concept survey with four groups of questions. Matrix surveys created with Flashlight Online 2.0 have other advantages for this kind of survey; for example, it's quite possible that ePortfolios are called by different names in different courses. When interested faculty fill out a form to request a student survey for a course, we can also asked them what they call ePortfolios in that course.

To show you how different response forms can be for the same survey:

  • Here's the Flashlight response form for one respondent pool, an English class that calls its portfolio "TaskStream", where the students get the core group of questions, the reflection group of questions, and job/school application group of questions.
  • Here's the Flashlight response form for another class, a Civil Engineering class that calls its portfolio "iWebFolio," where students see the core questions and the questions on feedback from multiple sources.

Reports: The faculty member would be sent a report about student feedback from that class. 

But all these student responses are also flowing into a single database (even semester after semester, if you like, so that trends can easily be analyzed). (Matrix surveys are analogous to a statewide election; people in different locations may get ballots  -- surveys -- with different candidates -- questions -- on them, but the responses all flow to one place for centralized analysis.)

So the people running the ePortfolio initiative will be to analyze student responses, activity by activity, across courses (and across institutions if this initiative is a collaborative developed by several institutions).  "How are we doing at using ePortfolios to foster reflection" (in the courses where that's being tried) and what are the barriers we should be trying to lower?"  "In classes where both job applications and reflection are goals, do the two goals reinforce each other, or interfere with each other?"

Return to home page for ePortfolio Evaluation, Flashlight Evaluation Handbook

PO Box 5643
Takoma Park, Maryland 20913
Phone
: 301.270.8312/Fax: 301.270.8110  

To talk about our work
or our organization
contact:  Sally Gilbert

Search TLT Group.org

Contact us | Partners | TLTRs | FridayLive! | Consulting | 7 Principles | LTAs | TLT-SWG | Archives | Site Map |