Search Committee Training and Process
Before Reviewing Candidates
- Watch the “Welcome to Outsmarting Implicit Bias” video from the Outsmarting Implicit Bias website: https://youtu.be/-9OjvkR7Ynk?si=3XZfkj4ta0B_02ya
- Review, individually, the “Rising Above Cognitive Errors” summary from JoAnn Moody.
- As a committee discuss the video and Moody piece and consider how the ideas can be incorporated into your search process.
- If you did not already do so when forming your position description, as a committee make a list of important qualities the successful candidate should have (e.g. form a set of evaluation criteria). In doing so you also might consider using the Criteria Matrix that is included with these materials. In full you should discuss and agree on what will be your evaluation criteria, ways the criterion can be met, and what evidence will demonstrate that the candidate excels or exceeds in this area, with a mind toward “keeping the funnel wide” when it comes to applicants in the screening stage. Our goal is to rely less on general impressions in evaluating candidates and more so to make decisions based on agreed upon standards and evidence. In advance of beginning candidate review, submit your evaluation criteria, or the full Criteria Matrix, to your Division Chair.
While Reviewing Candidates
- Allow enough time to review candidates. When committee members are rushed, it is easier to fall back on implicit biases or read only parts of an applicant’s portfolio.
- Avoid discussing candidates in informal settings.
- Decide what you will do as a committee if you receive additional notes or nudges about a particular candidate, for instance, an email endorsing a candidate from a dissertation advisor whom you know well. How will this affect the committee’s screening process?
- Use the evaluation criteria/Criteria Matrix as a rubric for keeping the committee committed to its qualifications.
- Submit your selection criteria/Criteria Matrix, candidate recommendations, and brief rationales to the Dean of the College and your Division Chair. Please wait for feedback from the personnel committee before scheduling any short-list interviews.
Before and During the Interview Phase
- For the short-list interview, prepare a structured interview. You might find some concepts from the “Outsmarting Implicit Bias” Structured Interview Action Sheet helpful in doing so (https://outsmartingimplicitbias.org/wp-content/uploads/OIB-ActionSheet-StructuredInterview.pdf). To the extent practical, consider how structured interviews might also help you evaluate candidates consistently and in full during campus interviews.
- In preparing for your interviews, review the “Interviews - What is Inappropriate to Ask” section in the Department Chair Handbook.
Rising above Cognitive Errors
Guidelines for Search, Tenure Review, and Other Evaluation Committees
JoAnn Moody, PhD, JD
www.diversityoncampus.com
Common Errors of Individual Members
Not errors just made by the ‘bad guys' but things we all tend to do if we are not motivated to avoid them.
1. Negative Stereotypes. "A stereotype can he defined as a broad generalization about a particular group and the presumption that a member of the group embodies the generalized traits of that group." Negative stereotypes are negative presumptions such as presumptions of incompetence in an area, or presumptions of lack of character or trustworthiness.
2. Positive Stereotypes. A halo effect where members of a group are presumed to be competent or bonafide. Such a member receives the benefit of the doubt. Positive achievements are noted more than negative performance, and success is assumed.
3. Raising the Bar. Related to negative stereotypes, when we require members of certain groups to prove that they are not incompetent by using more filters or higher ones for them.
4. Elitism. Wanting to feel superior through certain attributes or selectivity that highlights how we characterize more positive stereotypes (accents, schools, dress, ratings).
5. First Impressions. Drawing conclusions in a matter of seconds based on our personal likes/dislikes.
6. The Longing to Clone. Devaluing someone who is not like most of 'us' on the committee, or wanting someone to resemble, in attributes, someone we admire and are replacing.
7. Good Fit/Bad Fit. While it may be about whether the person can meet the programmatic needs for the position, it often is about how comfortable and culturally at ease we will feel.
8. Provincialism. Similar to cloning, this is undervaluing something outside your own province, circle, or clan. For example, trusting only reference letters from people you know.
9. Extraneous Myths and Assumptions. Undermining the careful collection and analysis of information, such as we can't get a person like that to come here, or we have all of them we need.
10. Wishful Thinking. Opinions rather than facts and evidence. Examples are assumptions that we, and certain other institutions, run on objective meritocracy, or we are colorblind.
11. Self-Fulfilling Prophecy. Some call it ‘channeling,’ where we structure our interaction with someone so we can receive information congruent with our assumptions, or avoid information incongruent with our assumptions.
12. Seizing a Pretext. Hiding one's real concern or agenda (e.g., excessive weight) behind something trivial, or focusing on a few negatives rather than the overall performance.
13. Character over Context, or Attribution errors. For example, failing to recognize the context of a situation—was it social, late in the day, outside of the professional arena, or an attribution of responsibility for a situation that is misplaced on one person rather than others.
14. Premature Ranking/Digging In. Rush to use numbers, as if they are objective, to drive a decision.
15. Momentum of the Group. It is difficult to resist consensus when the majority seems to be heading one way without a full hearing on other considerations.
Developing Screening Criteria: Step-by-Step Guidance for Search Committees
The goal of this step in the search process is for committee members to articulate a shared understanding of the qualifications for the new position, rather than relying on unspoken assumptions or past iterations of the department and position. The full search committee participates to develop this tool, discussing the key criteria for candidate evaluation, which qualifications most strongly predict better performance, and how/when to evaluate each qualification. Doing this before advertising double-checks the qualifications for clarity and priority, allows them to be refined as needed, and uncovers differences of opinion or assumptions that may stall the committee later in the process. Tying evaluative conversations to the matrix at each successive stage helps mitigate cognitive and structural bias.
Evaluation Criteria and Required or Preferred – Each evaluation criterion should be identified, one qualification in each cell. Typically, these criteria should be directly represented in the ad text. In the second column, indicate R for required or P for preferred. A candidate must meet all required qualifications to be hired. Preferred qualifications predict better performance.
Relationship to Job – To understand how broadly we can evaluate/interpret a qualification, we must understand what it enables the appointee to do in the position. Which position duties require it? Why is it needed? What aspects of the job might be difficult or impossible without it? Does this qualification tie directly to duties described in the job description, or have we failed to show the part of the job this qualification supports? Is it a proxy for skills not otherwise articulated? If so, what skills? Might it be better to list them individually?
Ways Criterion Can be Met – This column broadens our understanding of how candidates may meet each qualification, so we can consider more candidates and those who are qualified in less typical ways. Given its relationship to the job, what experiences, accomplishments, or learning meet this qualification? If it could be assessed in an interview, what MUST the candidate include in their answer—at a minimum—to meet the need? Go beyond quantity (such as number of years) to define indicators of acceptable quality in their performance or understanding. Ask “who might we miss?” and “What ways to meet this have we overlooked?” to broaden the criteria.
Complete these first columns for all required and preferred qualifications before continuing to Priority.
Priority – How important is strength in this qualification compared to strength in other qualifications? Required qualifications: If meeting a qualification strongly (vs. minimally), strongly predicts better performance, it is a high priority for the committee to spend more time evaluating. Conversely, if once an objective numeric qualification is met, there is no need for the committee to spend more time on it, it is low priority. Preferred qualifications: Those that most strongly predict better performance are high priority. Those that least strongly predict better performance are low priority. High, medium, and low priority apply to both preferred AND required qualifications.
Evidence the Candidate Excels – For medium and high-priority qualifications, what are indications that a candidate meets them strongly (beyond just meeting them) in relationship to the job? What ways of meeting (or exceeding) the qualification predict better performance? Go beyond quantity (how much) to include quality (how well).
When to Assess – at what stage will we have enough information to assess this qualification for all applicants? If it is high priority and will be evaluated at more than one stage, what are we looking for at each stage? When will we eliminate candidates for not meeting it?
