5 ways CLE providers can use assessment
I was recently privileged to speak at the Association for Canadian Legal Education Directors (ACLED) annual meeting in Toronto. I was a long-time member of this group and it was an absolute delight to see so many familiar faces. Our time together was spent discussing ways to incorporate assessment into continuing legal education (CLE) programs and resources.
Assessment is integral to learning
From my decade in CLE, I can say that assessment is not a popular topic. After an undergraduate degree, a law degree, and bar examinations, I can appreciate that most lawyers are overjoyed at the prospect of never being tested again. The thought of CLE providers looking to assessment—in the absence of any mandatory requirement—is bound to raise a few eyebrows.
One of the things we’ve learned, however, is that many lawyers are truly invested in their education. They don’t just want the easy credit for attending; they want to be better lawyers. We have also learned that assessment (in many forms, not just old-school scantron sheets) is integral to learning.
As Benedict Carey (Author of “How We Learn: The Surprising Truth About When, Where, and Why It Happens”) says “[t]esting might be the key to studying, rather than the other way around. As it turns out, a test is not only a measurement tool. It’s a way of enriching and altering memory.”
5 Tips for CLE providers
When we started thinking about ways assessment principles could be used by CLE providers, we identified several tips. Here are just a few…
Tip 1: Help lawyers recognize their learning needs
Quizzes are a current social media trend: they are short, interactive, and provide immediate insight. Wouldn’t it be great to see them used for more than finding out to which Hogwart’s house you belong? (Ravenclaw) Or which Disney princess you are most like? (you’ll never know)
Identify a CLE program or resource from which you want lawyers to benefit. Develop test questions based on its learning outcomes. From this, create and share a short quiz, inviting lawyers to test their knowledge in the area. Those who do more poorly than expected may recognize a learning opportunity. Include a call to action to attend your upcoming program or engage the desired resource.
Tip 2: Pre-test to optimize learning
A short pre-test can do more than help someone recognize a general learning need. It can:
- Act as a helpful diagnostic, matching lawyers to the program or resource most suited to their level of experience. Even when providers try to distinguish programs as being fundamental, intermediate, or advanced, these terms can mean different things to different people. A pre-test can help ensure that someone is signed up for the right program—ensuring they don’t waste their time or have a frustrating experience.
- Help presenters tailor their material. With most CLE programs, presenters have little insight into the audience’s existing level of knowledge or expertise. Some presenters will use polling or audience response questions at the beginning of a presentation to gain some insights, but not all presenters are adept at tailoring their material “on the fly.” If attendees complete a short pre-test before a program, the results may help the presenter focus the content for maximum benefit.
- Prime participants to learn. New research confirms that pretesting influences the subsequent learning of related information (Little & Bjork, 2016). Pretesting helps “prime” learners to absorb what is most important. It can also expose false impressions—things learners think they know but don’t. They can also help adjust a learner’s expectations before they start the learning activity.
Consider including a pre-test at the beginning of a book chapter or online course. A pre-test can also be used in lieu of pre-reading for a CLE program. Because these pre-tests are usually short and interactive, they can be more engaging than reading a typical text.
Tip 3: Improve the quality of audience response questions
Many presenters (both live and online) use audience response or polling questions to enliven their presentations and add interactivity. This can be a great way to engage participants, gather data, and assess learning. Frequently, however, the questions are suboptimal. There are accepted practices for developing good selected-response questions.
As CLE providers develop their expertise in assessment, they become better able to identify what kinds of questions are most valuable, as well as how to craft highly-effective items.
Tip 4: Make assessment the program focus
In my view, one of the most transformational learning experiences in CLE is the learning-by-doing approach to advocacy. While different providers may take different approaches, generally this training involves participants practicing advocacy skills (providing a sample of their performance) and getting feedback (based on recommended practices or standards). In other words, this training is highly assessment-focused.
Lawyers have many opportunities to read, watch videos, and learn in other ways. In contrast, there are few opportunities for lawyers to be observed, assessed, and given feedback by a qualified observer. Assessment-focused training doesn’t need to be the exclusive purview of advocacy. Consider what other types of programs might benefit from an assessment-focused approach.
Tip 5: Assess what learning occurred
What if you knew what lawyers were actually learning in your programs? (Not just what you hoped they were learning.)
Many CLE providers still rely on training reaction surveys (also known as “smile sheets”) to evaluate their programs. These results provide helpful information about the participants’ immediate perceptions of the quality and usefulness of the program. These “level one” assessments do not, however, provide adequate data to assess whether the participants learned anything in the program.
Engaging in a pre-test as a baseline followed by a post-test at the end of a program can help identify whether learning has occurred. This can be tremendously helpful in evaluating and improving programs—and can give confidence to learners. It can also help flag future educational opportunities.
Stay in touch
For more information (and more tips!), contact us for a copy of our paper “What CLE Providers Need to Know About Assessment.” If you have thoughts on incorporating assessment into a continuing legal education activity, please let us know. We are always excited to discuss and share. Connect with us.
Jennifer
References
Carey, B. (2015). How we learn: The surprising truth about when, where, and why it happens. New York: Random House.
Little, J. L., & Bjork, E. L. (2016). Multiple-choice pretesting potentiates learning of related information. Retrieved from https://link.springer.com/article/10.3758/s13421-016-0621-z