|
Rubric home
page
Flashlight Online 2.0 may be the most powerful,
sophisticated tool available for
creating rubrics, analyzing cumulative
responses to different mixes of criteria used in different
settings, sharing
rubrics, using rubrics to
gather assessments
of a project or performance
from many types of judges, and finally
using those assessments for several different dimensions of
educational improvement. ePortfolio users
- this may be just what you need!
-
Creating rubrics: Use Flashlight Online 2.0
to create attractive, detailed rubrics, e.g., for
assessing teaching and learning...
 |
...for for program evaluation:
 |
NOTE: Flashlight Online includes two rubric builders,
giving it great flexibility. For example, a rubric can
be shown in an expanded or collapsed form. The rubrics
above are expanded.
-
A matrix
approach to rubrics. Imagine an
accreditation self-study in an engineering department that is
evaluating how different courses are
contributing to student learning goals (A-F), cumulatively.
The faculty have a rubric for assessing student
capabilities in each area. Only 3-4 of those
rubrics might be
relevant to any particular assignment or project,
however. No problem.
The following table represents a set of those
assignments, showing the criteria used to assess each
one. By using Flashlight Online, the faculty can collect
all these responses in a single, shared database.
And they can study the results at any time over the
years as more responses flow in. For example, the
faculty can study how ability A is developing as
the students move from course 101 to 303. If
there are problems teaching ability D, that should show
up in the rubric-based judgments made of assignments in
course 101, 204 and 304.
| Course in which assignment was made |
A |
B |
C |
D |
E |
F |
| Course 101-assignment #3 |
X |
|
X |
X |
|
|
| Course 101-assignment #5 |
X |
X |
|
|
|
|
| Course 202 - assignment #1 |
|
X |
X |
|
X |
|
| Course 204-assignment #5 |
|
|
X |
X |
|
X |
| Course 303-assignment #2 |
X |
|
|
X |
X |
|
| Course 404 (capstone)-capstone
project |
|
|
X |
|
X |
X |
-
Sharing Rubrics: Flashlight Online users
can build on rubrics originally created by other users. Any author can enter a rubric into the
system and then choose to make that rubric visible to
other Flashlight Online authors for copying and
modification. Or collaborate with one or more partners
in creating your rubric. In the coming months, we intend
to greatly enlarge our collection of rubrics in the
system, and make them available to all interested
Flashlight Online users.
-
Multiple
sources of assessment: Flashlight Online is
a survey tool that can create rubrics. So it's easy for
a faculty member, a student or an evaluator to send a
rubric (including a link to the project being assessed)
to a variety of judges who then each use the rubric to
rate the work. Flashlight Online then enables the author
to summarize those ratings, either as a single pool, or
breaking the ratings down by the type of judge who
produced them. The images below, created with
Microsoft Excel by researchers at Washington State University,
demonstrates one way to display
the pooled judgments of a panel of judges who all used the
same rubrics.
 
-
For varied use by many different stakeholders.
Researchers at Washington State University
have also begun to demonstrate how rubric-based judgments can
be used for multiple purposes, including
-
Providing formative feedback to
individual students (comparing their own project
ratings with average ratings given other projects in
the same class)
-
Testing whether different types of
assessors (e.g., professionals in the field v.
faculty in the field) using the same rubric to
assess the same student work produce the same
ratings (see the figure immediately above; they
didn't).
-
Taking advantage of Flashlight
Online's
matrix capabilities to create forms with
different mixes of rubrics for different classes,
and then pool data from across courses for each
rubric.
-
Or do the same thing with program
evaluation or review - ask departments to create
reports, with rubrics as a guide. Give departments
each a subset of rubrics that are appropriate for
their disciplines. The department could use the
rubrics to solicit feedback from its visiting
committee or other panel of experts. Each rubric
(e.g., for assessing the creative performance of
undergraduates) could be used by many but not
necessarily all, academic programs. To prepare
reports for regional accreditors or internal
reviews, the institution could analyze rubric-based
feedback across all departments.
-
A department could use the same
matrix strategy for using rubrics to analyze the
development of student skills as the students
progress through the curriculum.
Interested in learning more about this
strategy for using rubrics? Take a look at
this blog from Washington State University. They refer
to this multi-source, multi-user approach to assessment as a
'harvesting gradebook' and the tool they use is the Skylight
engine that powers Flashlight Online.
Interesting in trying out
Flashlight Online?
Perhaps your
institution already has a site license. If not, and
you'd like to learn about the system, email us at
flashlight<at> tltgroup.org.
|
