Why you don’t know what you don’t know (and what to do about it)
When I was working in bar admission, I conducted intake surveys of recent law school graduates who were entering our program. Among other things, I had entrants anonymously rate their confidence levels in several important areas. Notably, most of these graduates self-assessed as being excellent writers. In fact, almost no candidate ever considered they might be average or below.
Spoiler alert: Not all law school graduates are excellent writers. This was demonstrated in our program year-after-year. Despite the exceedingly high confidence rates, this was an area in which more than a few candidates struggled. How can this be?
You don’t know what you don’t know
This is more than an adage. It is a reality with a profound effect on our competence and our continuing professional development.
In the field of psychology, we refer to a concept known as the Dunning-Kruger effect: a cognitive bias in which people erroneously assess their ability to be better than it is. The term was coined in 1999 by then-Cornell psychologists David Dunning and Justin Kruger.
David Dunning and Justin Kruger tested Cornell University graduates on logical reasoning, grammar, and humour. Those who scored in the bottom quartile “grossly overestimated their ability” on all tests. In one test, those in the bottom 10% believed they would rank in the top third. Why? Their ability in these areas was so poor they lacked the ability to assess what good looks like.
The bottom line: Relying on people to accurately self-assess can be risky.
Why not knowing matters
Lawyers have an obligation of competence. We do not re-test lawyers throughout their careers to ensure their continued competence; we trust them to self-assess. And therein lies the issue.
It isn’t that lawyers aren’t highly ethical. The majority of those I have encountered are. It isn’t that lawyers don’t care about competence. In my experience, lawyers are some of the most driven professionals imaginable—striving for excellence over basic competence any day. It isn’t that lawyers don’t engage in continuing legal education—they do.
The challenge is that they don’t know what they don’t know. As a result, they can’t correctly diagnose their learning needs. Without this vital step, there is no assurance that their continuing legal education targets what they need to learn to be safe, effective, or successful.
How the problem is made worse
As Dunning and Kruger concluded (p. 1121), “[t]he skills that engender competence in a particular domain are often the very same skills necessary to evaluate competence in that domain.” It follows, then, that lawyers may be better at self-assessing in areas in which they have existing knowledge and expertise.
For example, imagine a lawyer who has taken a family law course in law school, periodically reads up on family law developments, and talks to colleagues who practice family law. When a family law issue arises that is beyond the lawyer’s grasp, the lawyer may recognize that fact. This recognized knowledge gap acts as a motivator for the lawyer to learn about that area—or to refer the issue to someone more knowledgeable.
In contrast, imagine a lawyer who encounters an issue in which they have no frame of reference. Will they even recognize that the issue?
Today, there are many areas in which lawyers have no formal training that are critical to their safe, effective, and sustainable practice. Consider, for example, issues related to emotional intelligence, cultural competence, and technological awareness. If they do not have a solid grounding in what it means to be emotionally intelligent, culturally competent, or technologically-savvy, they might think these are competencies they possess.
The problem may be compounded if lawyers confuse exposure with proficiency. I cannot count the number of new law graduates and veteran lawyers alike I have heard say something along of:
“But, of course, I am an excellent [project manager, communicator, user of technology, people person, etc.] – I do [an activity that requires such skills] every day.”
Fact: Repetitively doing an activity poorly doesn’t make you good at it.
The challenge with high performers
Thinking you are good at something, doesn’t make it so. But just because you think are good at something, doesn’t mean you aren’t.
The Dunning and Kruger research also looked at high performers. In their Cornell study, those who performed in the top 25% significantly underestimated their performance. (The high-performing graduates estimated their performance to be in the 70th percentile, while their test results were in the 89th percentile.) A high-performer may assume that something that comes naturally to them, for example, comes naturally to everyone else.
In the words of Dunning and Kruger, “the miscalibration of the incompetent stems from an error about the self” (an internal illusion of superiority) and “the miscalibration of the highly competent stems from an error about others” (an external misperception of others).
So, what? Does it matter if someone is better than they realize? Yes.
While the consequence isn’t as dire as putting the public interest at risk, this can still be problematic. Why? Those who underestimate their proficiency may be missing out on opportunities to leverage their areas of competitive advantage. They may also waste resources engaging in learning activities that aren’t helpful. This isn’t really a problem, except for lawyers who have limited time, energy, or money to spend on professional development. (So… all of them.)
How to know what you don’t know
How is a lawyer expected to overcome the challenge of not knowing what they don’t know? Here are three things to consider:
- Develop some grounding in a range of areas—not just the narrow areas you think are important. Remember that taking a course or program can serve two purpose: (1) to help a learner develop proficiency in an area, or (2) to help a learner recognize when an area is over their head or beyond their interest. Sometimes your goal is the latter, not the former. Exposure to content and best practices in diverse areas may increase the odds a lawyer will recognize a knowledge or skill gap. For example:
- Follow general legal updates (not just those in your practice area);
- Take at least one program a year in an area beyond your current practice; and
- Look to resources in professional disciplines outside of law, such as project management, human resources, intercultural development, psychology, general business, and technology.
- Ensure self-assessment activities are guided. Use resources developed by experts in an area to guide your self-assessment activities. For example:
- Find a list of competencies and performance indicators for the area under assessment. This will help you know what to look for as you self-assess.
- Use a table of contents from a relevant book or a course outline as a checklist. If there are any headings that seem out-of-place or non-obvious, pause. This may be an area you don’t know you don’t know.
- Connect with people who have expertise in the area. Ask them: “If you were hiring someone who needed expertise in this area, what 3 questions would you ask them to give you confidence as to their proficiency?” Ask yourself these questions—and confirm your answers reflect best practice.
- When self-assessing, look for external validation of your self-assessment results. For example:
- Look for comparative data. If you spend a lot of time word-processing documents, don’t just assume you are fast at data input. Compare your results to a benchmark.
- Solicit feedback from others who are known for their proficiency in the area. Compare this feedback to your own self-assessment.
- Take advantage of independent assessment tools. Look for those that have been shown to be valid, reliable, and fair. Compare the results to your own self-assessment.
Leaving lawyers to self-assess their own competence in the absence of any of the above poses a risk to the public interest and to the lawyers’ sustainable and successful futures.
Stay in touch
If you are looking for more information about competency identification, aids to self-assessment, or independent testing, please feel free to connect with us.
Jennifer
References
Kruger, Justin, & Dunning, David. (1999). Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments. Journal of Personality and Social Psychology, 77(6), 1121. Available online at: https://www.avaresearch.com/files/UnskilledAndUnawareOfIt.pdf.