Thursday, January 8, 2009

Academic Hirings: Getting Rid of Bias

If your university's approach to hiring new faculty members is anything like ours, the drill is as follows. Candidates are short listed by a committee based on their academic record, research and writing achievements, references, and prior teaching experience (if any). Once the short list is generated, those on it are invited to come to the faculty to give a "job talk", meet faculty members, the dean, and the hiring committee. In-put is then solicited from the faculty, and the hiring committee advises the dean, whose responsibility it is to make the hiring decision. Simple and straight forward. Is this the best process? I would argue that it is not. My suggestion - get rid of the job talk and the informal visit to the faculty. Hire on the best evidence - the hard evidence. Get rid of the shmoozing and you will hopefully get rid of the bias.

Everything one needs to know about a candidate's record of achievement and scholarly potential is in the file. A one hour chat with faculty about the candidate's research interests is no substitute for knowledgeable faculty members actually reading what that candidate has published. An informal faculty seminar to determine a candidate's communication skills and teaching ability is no substitute for looking at teaching evaluations or assessments of earlier teaching. A short visit to determine a candidate's "personality" traits is no substitute for considering serious references from people who actually knew and worked with that candidate.

It is true of course that one can and should do all of these things in addition to the job talk and visit. What harm, you may ask, is there in adding the visit to all of the other evidence that you have? My answer is that in addition to the practical difficulties that setting up the visit entail, including the faculty's expenses, the scheduling nightmare, and the inevitable delays in the ability to make quick offers, the visit introduces the real potential that irrelevant factors will either consciously or subconsciously affect the hiring decision. Things such as the candidate's looks, manner of dress and speech, or personality traits that have absolutely nothing to do with being a successful academic, may actually play a role in that person's career opportunities and the faculty's decision to hire the best people that it can.

If one did a cost/benefit analysis, I would venture to say that the costs of having these types of visits far outweigh the benefits that they produce. Colleagues opposed to my idea (and that basically is everyone who has heard it) say that visits may reveal flaws in the candidate's academic and research abilities, intellectual stature, or personality which are important to the hiring and would not have been exposed otherwise. That would mean that we would have found something relevant in a short visit that the candidate's previous professors, doctoral or masters defence committees, students, colleagues, employers, referees, and peer reviewed journal editors, missed. How likely is that?

As I stated, I have found no-one who supports this idea. I do not know if any Canadian or U.S.A. university has abandoned the job talk visit as an integral part of its hiring process and based its hiring decision solely on the candidate's hard record. So I guess the odds are that I am being simply contrarian or crazy in proposing this. What do you thnk? Crazy idea or an idea whose time has come?

12 comments:

  1. Although there might be good reason to consider a candidate's documented credentials as 'hard evidence',and although the university should hold this evidence in high regard, I believe there is good reason to continue holding job talks.

    I don't believe that the job talk is meant to discover hidden flaws the candidate might have concealed in their application. Instead, I would suggest that it provides the candidates with an opportunity to stand out amongst their peers. Likely, candidates who have made the short list will all have impressive credentials. To distinguish between those who might look similar on a paper, meeting the candidates in person allows a university to further compare highly qualified individuals.

    Additionally, an institution should not vet a candidate based solely on credentials, but should be checking to see if a particular candidate would fit in or compliment the academic atmosphere of the institution in question. Something that I argue can only be assessed in a physical visit. There is always a subjective aspect to hiring a person based on how the they interact as a human being; not assessable in the analysis of documented evidence. I believe that the job talk provides a university with a unique opportunity to vet a candidate accordingly.

    Finally, the job talk provides the opportunity for those within the institution who might not otherwise get the chance, to assess a candidate and provide their own input. For instance, as a member of our department's undergraduate students' association, I have had the opportunity to sit in on job talks and question candidates. In this way the students are invited into the hiring process. This certainly would not be the case were universities to cancel their job talks.

    ReplyDelete
  2. I think however that you have identified the very factor which I want to eliminate - the would the candidate "fit in" factor. The "collegiality" requirement is a vague and perhaps even dangerous one, especially for academic institutions which should be home to independent minds and idiosyncratic personalities. At any event, my experience convinces me that that it is impossible to predict from the short visit who will best compliment the environment. Students should indeed be included in the process; but they too should be required to assess based on the evidence, without the visit.

    Thanks for the input Jess.

    Lewis

    ReplyDelete
  3. The job talk might also be beneficial from the candidate's perspective; it allows the job candidate to size up the faculty, gauge their research interests, their enthusiasm, dynamics, etc. It is not merely an interview of the candidate but also an interview by the candidate of the law school.

    ReplyDelete
  4. http://www.newyorker.com/reporting/2008/12/15/081215fa_fact_gladwell?printable=true

    ReplyDelete
  5. Thanks.. I thought of that. Although my initial post did not deal with it, my proposal would allow the successful candidate to visit the law school at our expense to size up the faculty, the city, schools etc. But only after the offer has been made. After the visit the candidate can then decide whether to accept or reject the offer.

    ReplyDelete
  6. If face-to-face interaction invites bias into the hiring process, can one take it to mean that by extension, all types of interviews are biased and should be eliminated? Should the hiring process (in general, of any and all jobs) be solely based on the "hard" evidence to be found on one's CV?

    Perhaps the "bias" that you are concerned with is actually a way of evaluating intangible human qualities that are difficult to represent on paper.

    ReplyDelete
  7. I don't know if that gladwell link above relates to it, but I remember reading in one of his books about "blind auditions" that were undertaken by a major symphony orchestra in order to ensure that gender was not affecting hiring decisions for the musicians. As I recall, once the judges were blind, the number of women hired went way up.

    So there may be something to this. And there are certainly subconscious preferences toward things like beauty and height.

    Still, Einstein, I think, is credited with saying "Not everything that can be counted counts, and not everything that counts can be counted." It seems to me to be an open question whether everything that a dean wants in a faculty member is going to be adequately measured in a hard-facts scenario only.

    Can you eliminate the extraneous factors without eliminating any important ones?

    It would also seem strange to move the selection of law professors by law schools any further toward the ridiculous method we have of selecting law students. Strange, but somehow poetic.

    ReplyDelete
  8. Both Emma and Gauntlet assume that there are "intangible human qualities" and extraneous factors that are "important" in hiring academic candidates, which can be discovered or discerned in the interview. Since it is fairly clear that there are clear negatives to the interview ( for example, costs,and delays in hiring) and arguably the interview actually introduces unwarranted biases whih are counter productive, I think it is incumbent on those who are supportive of the interviews, to identify these important, intangible qualities. What exactly are they? Can they be discovered in the interview? And do they outweigh the costs and disadvantages which I identified?

    In terms of other types of jobs, I would argue that unless good looks, nice clothes, and the ability to shmooze are job qualifications, get rid of the interview there as well. It of course depends on the job. But in so far as academic positions are concerned, these are irrelevant distractions.

    ReplyDelete
  9. What is the Gladwell piece that "anonymous" has referenced? I cannot seem to link to it, and would like to read it.

    Thanks

    ReplyDelete
  10. I want to test the limits of your "no need for an interview" theory.

    Assume that you have the job of matching one of your kids with a life companion--someone they would marry (i.e. commit to sharing the next 25+ years of their lives with). You have the CV and some reference letters for potential life partners. [And, for the sake of argument, the CV includes details of their personal lives and some information about their religion, finances, and family--i.e. not just the details you'd ordinarily find on a CV but information that you may find pertinent to the marriage decision. You can even assume a photo of the candidate exists so that you can evaluate the potential cuteness of your future grandchildren]. Are you comfortable selecting a person without first meeting the person in question? If not, ask yourself what it is that you would expect to learn from a meeting with the person that you wouldn't get from the documentation and the references? And remember, you are being asked for your choice of person only, regardless of whether your child has or has not met the candidates.

    ReplyDelete
  11. Here's the Gladwell piece. Sorry I neglected to leave my name earlier-- it asked for my URL, I don't have one, I'm easily confused, &c&c. The more recent 'anonymous' isn't me.

    --scott matheson

    ReplyDelete
  12. Anonymous:

    I'm not sure that the analogy works. . . If my parents were picking a spouse for me, I'd hope the candidates' professional/educational experiences would have little bearing on the final decision. If they were choosing an academic to train me in my graduate studies, I'd hope that charm and affability played an equally small role.

    I think it's needless to say that academics do not need to fill the same role as family members or friends. (Present company excluded, Ivory Tower Pundit!)

    ReplyDelete