|
Creating a Culture of Evidence (Introduction)
l
Strategies for Increasing Use of Your Other TLT/Flashlight Subscriber Benefits
Flashlight Evaluation Handbook Table of Contents
One of Flashlight Online's unique strengths
is how easy it is for authors to show and share their
surveys with others, so that colleagues can build on their
work. The more people
at your institution who use
Flashlight Online, the greater an asset the system becomes. Your
subscription provides a variety of guides, cases, and
workshop materials. Here are
some additional ideas for widening the use of Flashlight Online. Please send more ideas like this to flashlight@tltgroup.org.
-
Create a support site for your
authors on your institution's web:
-
One of Flashlight Online
2.0's unique
strengths is the ability to create shared workspaces
((folders) where authors can store and share surveys and data.
Create a folder on a topic and then give relevant authors
access to those folders (e.g., forms for an upcoming
accreditation self-study; surveys for classroom assessment
in mathematics). If there are model forms, templates, item
banks, or rubrics of use on that topic, you can store them
there and then everyone with access to that folder can copy
and adapt those resources. If you like, you can also
give access to Flashlight Online authors
at other institutions -- Flashlight Online is just one big
system shared by authors at over 110 institutions around the
world. Looking for users at other institutions studying the
same issue you are (e.g., studies of distance learning?
needs assessment forms? course evaluation?); send e-mail to
flashlight@tltgroup.org if you'd like help finding
collaborators.
-
Encourage your authors to attend our free
monthly online users meetings,
"Second Tuesday."
These sessions are especially relevant for Flashlight Online
administrators and for people with responsibility for
evaluation, assessment, scholarship of teaching and learning,
and related topics.
-
Encourage your authors to become
individual members of the TLT Group. It's free
because your institution is a subscriber, and, among other
benefits, they'll get emails about our upcoming
Flashlight-related workshops (many of which are free for
everyone at your institution).
-
In workshops and web
materials, integrate 'learning to use
the tool' with 'learning to solve your problems with the
tool.' Your authors are much more likely to create a second survey if the findings from their first
survey were useful and energizing. Our
"Asking the Right Questions" workshop materials
exemplify this approach; each workshop is organized around
an important question that a novice author can use
Flashlight Online to answer. What other workshop materials
should we develop?
-
NEW! In workshops
and web materials, recognize that people have legitimate
objections to assessment and evaluation. We have a
web page of such frequently made
objections, and a
model feedback form that incoming participants in a
workshop can use to report on such worries and lay the
groundwork for discussion and debate.
-
NEW! In
workshops, recognize that discussions can be derailed when
participants don't realize that they are using conflicting
definitions for such basic terms as "assessment" or
"teaching." To help people become more aware of these
confusors,
subscribers can use a
feedback form such as this, before the session, to
discover where such misunderstandings are most likely.
-
Create your own listserv(s).
Charles Ansorge has done that, and also the faculty survey
mentioned in the preceding bullet; he's at the University of
Nebraska.
-
Work with a critical
mass of colleagues who care about research, evaluation and
assessment: At any
institution, there are folks who care whether the
institutional culture encourages and supports the use of
evidence to make decisions: improving teaching, improving
services, improving the work of institutional committees,
and, of course, teaching students to use surveys in
productive, effective ways. Meet regularly with those folks
and talk together about how you can strengthen those
practices. Click here to see how several institutions have gradually
succeeded in getting a substantial fraction of faculty and
staff using evaluation to help make decisions and improve
practice. Among the ideas you'll find there: using
Flashlight Online to gather evidence for institutional
committees; requiring assessment (and supporting it) as a
requirement in faculty mini-grants; using Flashlight to
support accreditation self-studies; using peer-to-peer
training strategies; developing libraries of locally useful
survey, rubrics, forms and other templates that many people
can share; strategies for getting people to come to training
sessions; and more.
-
Use Flashlight Online for surveys to
collect teaching/learning ideas from faculty (Click here for
details and a survey template.) Or do a needs assessment
survey. (Send e-mail to flashlight@tltgroup.org to ask for
ideas about such a survey.) Make sure the survey
itself has a note indicating that it was made with
Flashlight Online - this creates more awareness of the
system among faculty.
-
Use Flashlight Online to engage
faculty, staff and students in governance questions of
immediate concern to them. Valencia Community
College has pioneered this use of Flashlight Online,
regularly polling faculty for their opinions on new policy
questions, and polling students on course policies where
student opinions and engagement matter.
Click here to learn more.
-
Regular workshops on
using Flashlight Online
to solve problems: As part of our monthly online
sessions for Flashlight users and administrators, we usually
include a segment on workshop content and strategies.
-
Offer very brief
workshops (5-20 minutes long) as agenda items in
departmental faculty meetings or in brownbag lunches.
(For examples, see "Asking the Right Questions" workshop
materials.)
-
One way to help legitimate and spread use of
Flashlight Online as a tool for faculty to improve their
courses is to have faculty run the workshops (thanks to
Johnson C. Smith University for this idea!)
-
If
your institution is a Network
subscriber, we can design
a workshop for or with you, on a topic of your choice,
and offer it on campus or online. Popular topics:
evaluating your (mini)grant; using student feedback to
improve your course.
-
Tiers of
Support: The TLT Group and Washington State
University provide many kinds of support direct to
Flashlight Online users (e.g., free online training in using
the system; "Second Tuesday" user group meetings). But we suggest developing a tiered system of
support that begins at your own institution.
-
The first line of help should be at your institution
- you may want to build a team, including people who are
good at evaluation design. The first time around
you'll almost always need to turn to level '2' below but
after awhile, we hope that most questions can be
answered at level '1. This connection also is a
way for the local administrator to learn more about what
users are doing, and what they need. Flashlight Online
2.0 is going to offer you the option of building in your
own local 'help links' that will be seen only by your
own users.
-
For questions that can't be answered
locally, you should contact us
(flashlight@tltgroup.org or 301-270-8312) and
describe the problem. We suggest that you send in the
request personally, rather than the
person with the problem, so that you can
learn about the solution and help others later on who
have the same issue.
-
Be prepared to
deal with Frequently Made Objections. Most people at
institutions don't do studies and may not believe they're
worth the effort. If you understand their objections, it's
the first step.
-
Applause/Publication:
-
One way to help develop
institution-wide commitment to using data to improve
practice is to applaud those who have done it. How
about a special lunch to honor people whose inquiries were
especially productive or that had exceptionally high
benefits of payoff to effort?
-
Another strategy: if there's a survey
that one person has prepared and others are using, a survey
you think at least a few people at other institutions would
like to use,
submit it for peer review. Most such surveys pass review
and become approved Flashlight Online templates, available
as part of the system to all users in the world.
-
What other strategies also can help? Send
your ideas to ehrmann@tltgroup.org for inclusion in this list!
|
