Circular argument

Multi-source feedback systems, usually called 360-degree feedback, have been adopted with such enthusiasm that they are now commonplace

They normally involve “target” managers being rated by subordinates, peers, bosses and sometimes customers, and examining how these assessments match up with their own self-ratings.

The history of these systems is still a relatively short one, especially outside North America. Surveys on both sides of the Atlantic in the mid-1980s showed that about 10 per cent of US firms were using these techniques, compared with none at all over here. But since the early 1990s, 360-degree feedback has spread quickly across a range of public- and private-sector organisations in the UK.

In what might be called phase one of 360-degree feedback, the emphasis was on its use as a development tool. It was usually applied in the context of career development workshops or as a one-off exercise for a group of managers. But there is no doubt that we are now into phase two, as more firms seek to use it as part of the ongoing evaluation of employees. In some cases, they are contemplating linking it with pay.

This shift in emphasis from development to assessment is causing alarm among a number of professionals working in the management development field. So is it really something we should be concerned about?

Before trying to answer that question, it is important to realise why organisations want to make 360-degree feedback part of appraisal. Mainly, this reflects the failings of conventional, top-down appraisals. They are often seen to be limited, because they reflect the perspective of only one person, and ratings have been shown to be prone to bias. Also, top-down appraisals too often seem to achieve little behavioural change.

Including 360-degree feedback in appraisals seems to offer a solution to some of these problems. In theory, multiple levels and sources of data should lead to a more objective picture of an individual’s contribution, strengths and development needs. It should consequently promote higher levels of trust in the fairness of the process. This, in turn, makes it more likely that some changes in behaviour will ensue. Quite apart from that, making the feedback part of appraisal gives it some “teeth” – it sends a message to people that this is something that the organisation takes seriously.

But things can go badly wrong. In the US, one study showed that half of the firms it surveyed that had implemented 360-degree feedback for appraisal had later dropped it. The potential problems are not hard to grasp.

The first is that incorporating 360-degree feedback into appraisals may affect trust, which is necessary for the whole thing to work. People giving the ratings may fear some adverse consequences if they give negative feedback, or that the feedback may be misinterpreted by the individual’s manager. The result would be lower-quality information, particularly from subordinates.

Associated with this, target managers may become less ready to accept feedback if it has potentially damaging consequences. This scenario could lead to political game-playing. You might have subordinates asking for, say, a pay rise just before they give their assessments. And managers might be tempted to court popularity.

The second concern is whether the ratings given in 360-degree feedback really are more objective than those of a traditional appraisal. The system undeniably gives more perspectives on an individual’s performance, but are assessments by subordinates and peers any less prone to bias? There is evidence to support its use in appraisal. For example, data from a variety of organisations has indicated that appraisees are more satisfied with ratings from multiple sources rather than from one alone. But the research mostly tends to highlight the potential pitfalls.

When the purpose of ratings becomes evaluative rather than developmental, up to 35 per cent of those giving the ratings change their assessments – and the changes can be in either direction. This seems to support the notion that trust may be affected. The research findings are not reassuring in relation to accuracy, either. A study of a pilot 360-degree feedback system run at Shell showed that it was not measuring the competencies it was supposed to, and that it had other shortcomings in terms of its psychometric qualities. Fortunately, the same study showed that Shell’s newly redesigned system did work much more effectively.

There is broad acceptance that 360-degree feedback can be a valuable developmental tool. Feedback of this sort has a potentially powerful impact, and the general view is that it is better to expose people to it as part of a development exercise before attempting to use it in appraisals. But the research findings should make people wary of grafting it on to appraisal systems. This is not to say that it can’t be used successfully, but it does need to be handled with care.

There are several issues that have to be addressed if 360-degree feedback is to be switched from a primarily developmental tool to a primarily appraisal tool:

  • Is it to be mandatory or optional? If the system is an aspect of the appraisal process, it has to be mandatory. Organisations cannot have people opting out of part of the appraisal.
  • Will it be carried out annually? This is usually the case if it is part of the appraisal process, which has implications for the resources required to administer it. As a developmental event, it is usually done intermittently or as a one-off.
  • Who decides on who is to contribute to the assessment process? In a developmental system, the subjects generally choose their own assessors – a practice that is less likely to be acceptable in appraisals. Letting people choose who makes an input can give an opportunity for the more Machiavellian appraisee to influence the process by arranging some reciprocal back-slapping.
  • Who is responsible for follow-up action? In development, the target manager usually discusses this with an HR manager. In appraisal, the individual’s manager is more likely to be involved.
  • Is it to be linked to reward? In a few UK companies, such as parts of BAe, this link already exists. It has also been related to pay in a number of US firms – Federal Express, for example – and their operations in this country for some years.

So a 360-degree feedback system that forms part of the appraisal process is unlikely to resemble one intended to be purely developmental. Even more importantly, the attitudes of the participants may well be different – and it is this aspect that requires most sensitivity.

Establishing trust is the crucial first step in moving 360-degree feedback from developmental to appraisal tool. This entails genuine consultation with those receiving feedback and those giving it about how the process is to work, what the content of the feedback form is to be and how the output will be used.

Although some of this may have been done earlier when the system was used for development, the shift to appraisal puts an even greater onus on consultation. Everyone should feel comfortable with the relevance and fairness of the process. Relevance is not only about the importance of the competencies assessed for the individual’s job, but also about those giving feedback being in a position to make informed judgments. The latter also has a bearing on fairness, as does the use made of the feedback – how much weight is it to be given in the overall appraisal?

That question raises another necessary step: the training of appraisers in evaluating feedback. Interpreting data of this kind is not at all straightforward. Does a very favourable response from subordinates automatically mean that their manager is an effective supervisor? Perhaps not, if it chiefly reflects the fact that the manager deflects pressure away from his department on to other units and produces only modest results.

In appraisal, 360-degree feedback has to be considered alongside all the other performance information available, and the appraiser has to piece together a coherent picture. It might make the appraisal a better process, but it certainly does not make it an easier one.

From my own experience, I would say that the demands of running 360-degree feedback annually are considerable, and far greater than some firms seem to think. A manager who has four subordinates, four peers and who has links with two superiors ends up with 10 feedback forms plus a self-rating to complete every year. “Ratings fatigue” can set in quickly.

It is all the more important, then, to ensure that the system is as economical as possible. Among other things, this means keeping the feedback forms reasonably short and focused. Using a computer-based system is undoubtedly the least cumbersome approach to generating feedback on a regular basis. It also allows for the data to be stored effectively, which leads on to my last point.

The fact that 360-degree feedback systems do not always measure what they appear to has already been mentioned. If such data is contributing to appraisals, the dangers are clear to see. Developing and running such systems is not rocket science, but there is no excuse for not making some simple checks to see that they are working in the way intended, as opposed to merely finding out whether participants feel good, bad or indifferent about them (important as that might be).

Examining the distribution of ratings by, say, checking whether the individual items on the form line up with the competencies they are supposed to will reveal a great deal about the quality of the system. There are various other evaluation measures that can be taken, but they are more long term in nature.

Many organisations are intent on making multi-source feedback part of the appraisal process. Many individuals who have been using feedback for development purposes have voiced alarm about this trend. This is perhaps justifiable, considering that some firms try to apply 360-degree feedback in this way without considering the implications.

But while it can all go wrong, it does not necessarily have to. Given a professional approach (only some aspects of which I have mentioned), it is possible to make this kind of feedback a valuable input to performance appraisal.

Clive Fletcher is professor of occupational psychology at Goldsmiths’ College, University of London