How often you hear words like these
from faculty, staff and students,
"Before we decide what to
do next, let's see if we can first unearth some additional
useful facts, so we'll be more able to make a successful
decision."
that's one indicator of a healthy culture of evidence.
Another: it should be easy for
those faculty, staff and students to tell you about success
stories of colleagues, past and present, where doing a
little research led to a triumph, or avoidance of disaster,
or just saving of time and avoiding stress. In other words
the folklore of the institution is supporting the idea that
pausing to do a little research is often a wise thing to do.
Another: most of the time people would look forward to being
surveyed or interviewed, because the last times that
happened, the results more than justified the thought and
time they put into their responses.
In our view, a culture of evidence is, among other
things, a key ingredient to teaching students the true value
of research. Equally important, a culture of evidence helps
the institution and its programs adapt to changing times and
tight resources.*
What features of an institution's facilities,
organization, policies and practices support and maintain
a culture of evidence? Among those
elements:
- Training in how to 'ask the right question,' especially peer-to-peer training in
how to ask important, answerable questions about one's
own practices. [Subscribers: The TLT Group can
help you craft and offer such workshops. For example,
for faculty learning how to ask questions of their
students that can help the faculty member figure out how
to teach better, see this collection of workshop materials.]
- Networks for sharing problems, methods and
findings: staff and faculty need to frequently
interact with peers, inside and outside the institution,
who are also using evidence to improve practice. They
can share ideas, techniques, findings and occasionally work
as a coalition to get more support for this work (by
'support' we refer to all the other bullets in this
list.)
- Tools for doing studies: A large and growing
fraction of staff have access to, and know how to wisely
use, tools of inquiry such as Flashlight Online.
Perhaps the most relevant feature of Flashlight Online:
it makes it easy for people to find surveys created by
others, and then either use them "as is" or adapt them.
So it can support the development of a culture of
evidence, while it's also providing useful tools for
individuals.
- Organized institutional data collection that
can be tapped by individual investigators and tailored
(e.g., faculty can add their own items to student course
evaluation surveys).
- Expert assistance can be, and is, used by
them to help with such studies (e.g., statistical help;
help in designing good surveys; help in gathering other
forms of specialized data)
- A shared sense of track record in this area -
examples of studies that helped improve learning, save
time or money, and the like are widely known. Potential
respondents to surveys, interview requests, etc. know
from past experience that this is usually a productive
way to use their time.
- Hiring and retention of staff: when faculty
and staff are hired and promoted, attention is paid to
their track records of using evidence to improve
practice and outcomes.
- Gradually developing among students a sense
of individual and collective responsibility for
education, so that responding to surveys and other
research seems both a way to gain control over their
environment and the fulfillment of a responsibility to
others
- Institutional priorities for improvement,
including the use of evidence to guide and accelerate
those improvements (e.g., an institution that has a
multi-year plan to improve distance learning, or
learning communities, or international study; and then
does institutional studies to support this effort while
also supporting studies on the topic by individual
faculty and staff with regard to their own contributions
to the effort).
- Priorities/budgets/persistence: A shared
sense that the administration values such practices of
inquiry and consistently invests in them
- Related policies and practices: does the
Institutional Review Board (IRB) have quick turnaround
on requests to gather data? do union contracts support
or interfere with efforts by faculty and staff to use
data to improve teaching?
- Support and rewards from outside the institution
for these practices and their results (e.g.,
regional accreditors who value such practices and reward
institutions that are especially good at them).
In 2003, The TLT Group, Washington State University, the
Coalition for Networked Information, and EDUCAUSE's National
Learning Infrastructure Initiative (now EDUCAUSE Learning
Initiative) collaborated in a project on "Transformative
Assessment."
This rubric, developed by Gary Brown of WSU as part of
this collaboration, is a useful tool for assessing
institutional progress toward using data to improve
practice. We hope to expand and improve this rubric soon.
This 2001 article on the efforts of three institutions
to create a culture of inquiry was another contribution to
that effort.
*Some people use the phrase
'culture of assessment.' As we define terms, a
culture of assessment makes assessment of learning an
everyday activity. A culture of evidence supports not
only assessment but also other inquiry by faculty, staff and
students intended to help them gather facts in order to make
choices.
Note to TLT
Group subscribing institutions: one way to use your TLT
Group consulting time: our consultants can help you assess the factors that support
(or are currently failing to support) a culture of evidence
at your institution, and then suggest steps that could help
you move forward. This kind of assistance might be
especially timely in the years before your institution's
next accreditation or program review.
