“Do you have any 360 evaluations for nonprofit Executive Directors or CEOs?”

I get this question a lot. The simple answer is, yes, but I don’t like the simple answer because this is a complicated topic. I’ve seen 360s used well, ignored, and used destructively, so a little explanation may be in order. Let’s explore how we can get the most out of 360 Evaluations.

The 360 Evaluation is a multi-rater evaluation. It’s used as a tool for organizations to obtain anonymous comments from those who are in the best place to observe the individual’s performance—those who work beside, under and over.

The theory behind getting opinions from a variety of colleagues, coworkers, managers and direct reports goes back to something we learned in Psych 101. As Erich C. Dierdorff, Robert S. Rubin write in an HBR blog, “self-awareness is understanding who we are and how we are similar to or different from others.

That understanding derives from two perceptions:

  1. How we view ourselves.
  2. How we think others see us.

And we behave accordingly!

However, what is so often missing from that formula is the other and equally important component to self-awareness: how the other actually sees us. As a result, we may behave in ways that are annoying, foolish, or even contemptible—and we have no clue. Such negative behaviors if unchecked can damage workplace relationships, chances for promotion, or even having the job at all.

There are good and bad points to anonymous evaluations. The fact is the higher up one moves in an organization, the less open, candid, and direct responses he or she receives. An anonymous evaluation usually elicits more honest and direct comments from those who would be reluctant to give negative feedback face-to-face. However, anonymity can also result in disrespect and self-serving rhetoric.

Many varieties of multi-rater instruments are available, some are better researched and valid than others; some are costly, others are free. Online versions of the 360 seem to be the most popular (over 10 million Google hits) although whether the tool is electronic, or paper and pencil does not affect the effectiveness, just the ease of administration.

The Internet is full of blogs and commentary regarding the usefulness or lack thereof related to the 360 process. Marshall Goldsmith, highly regarded Executive Coach, is one of the tool’s biggest critics, especially of its subjectivity. However, I believe when there are multiple-raters, the most subjective feedback loses effect if it is the minority. Furthermore, don’t forget: annual reviews by one person can be the most subjective and ineffective of all.

I have personal experience with several 360 tools and methods—as the administrator, subject, and participant. Some were incredibly beneficial; others were a waste of time, dollars, and good will.

What Works Best with 360 Evaluations?

Many variables determine the success or failure of the 360 Evaluation. As with most organizational efforts, how the process is introduced, how participative it is, and its real and perceived purposes are critical factors. When the stated and actual purpose is to enhance professional development, and it is designed to identify ways the individual can learn and grow, chances for success soar. If the real purpose is to “fix” people, remind them of their weaknesses and nag them to change—good luck!

While several of the instruments available are excellent, what I believe works is a comprehensive process embedded in the human resource psyche. By that, I mean the managers evaluated involve themselves from the beginning: clarifying the purpose and use, designing the process, identifying available instruments, and tracking results.

The best process I’ve been a part of incorporated a relatively simple online instrument combined with a carefully crafted set of interview questions for individual interviews conducted by an outside consultant. Following the interviews, the consultant compiled the responses about each and shared both a written and a verbal report with each person privately. With everyone’s knowledge, the CEO received all the reports but the individual interviews were confidential.

This dual approach seems to allow for different learning styles as well as providing an opportunity for clarification and a backup for information easily “forgotten,” dismissed or not heard due to anxiety.

Note: Designing an effective tool or interview questions is not a simple or quick task. Inc.com published an excellent article by Karen Carney describing a good process. Below are a few of the competencies or behaviors listed to be considered for inclusion:

LISten’s Checklist of Competencies/Behaviors (partial list)*

  • Leadership. Motivation, team building, role model, visionary, troubleshooting, positive reinforcement.
  • Communication skills. Persuasive, frank, and productive communication, listening, writing, presenting, reporting.
  • Interpersonal skills. Productive one-on-one relations, adapts to different styles, emotional maturity, respect, values diversity.
  • Personal characteristics. Integrity, trustworthy, manages stress, responsible, organized, disciplined, good work ethic.
  • Adaptability/change readiness. Effective in varying circumstances, eager to learn new skills and tackle new responsibilities.
  • Judgment/problem-solving. Systematic problem-solving skills, analytical, conceptual, detail oriented, thorough, creative, innovative, proactive.
  • Quality focus. Attention to detail, continuous improvement, design of processes and procedures, development of expectations and standards….”

(*For the full list: Better 360° Evaluations, Inc.com.)

Two 360s I particularly like include the Workplace Big 5 Profile (www.centacs.com) and the Center for Creative Leadership’s “360 BY DESIGN” (www.ccl.org).

What Do the Experts Say About 360s Effectiveness?

A fair amount of research has been done on the effectiveness of 360s, although there are far more opinions than facts. Here is one finding that is rarely factored in:

“The most accurate ratings come from those who have known the individual being reviewed long enough to get past the first impression, but not so long that they begin to generalize favorably (Eichinger, 2004).”

Interesting point about 360 reviewers: “direct reports are the least reliable and, therefore, more participation is required to produce a reliable result.” (Source: Wikipedia.org)

The primary reason to pursue 360 evaluations is this: “Studies have… indicated that self-ratings are generally significantly higher than the ratings given from others (Lublin, 1994; Yammarino & Atwater, 1993; Nowack, 1992).” Without contributions from others, self-assessment is too often self-inflated, although the reverse situation (not self-inflated enough) can be an even greater problem for teams and productivity.

According to Dierdorff and Rubin, “companies spend millions of dollars and countless hours every year on self-reported assessments that only target self-knowledge. The core problem is that we’re notoriously poor judges of our own capabilities.”

There Are No Magic Bullets

The first big question is, do 360s improve company performance? Research findings are mixed, so that’s when common sense needs to kick in.

There are no magic bullets. Employees who receive accurate, direct, caring and frequent feedback about their performance can maximize what they do well and work on managing barriers to excellence. If they are not aware that there are barriers, there is no hope that there will be any significant change.

The second big question is, will the 360 alone be enough,? The answer is simple: No.

Excellence depends on a variety of factors in the organization. Increased self-awareness for the organization’s managers is a key way—no, the only way—to enable those managers to manage their behaviors effectively towards achieving excellence. 360s increase the chance of managers’ self-awareness but it’s the immediate, frequent, direct, genuine and caring feedback–both positive and negative– that, in my experience, is the best vehicle to facilitate a manager’s growth.