The practicalities of measuring the RoI of Coaching: A reflection by Doug Montgomery (Guest)
What is the Return on Investment (RoI) of coaching?
How do you measure the RoI of coaching?
I have come across these questions a great deal over the last few years. And it keeps occurring to me: Who wants this information and why? Is RoI the right or most useful thing to measure?
Who’s asking?
In executive coaching, there are multiple parties involved and interested in how to evaluate coaching. There is the coach, the coachee, the coachee’s manager, the sponsor (who may or may not be the payee), the client – who is the payee, potential new clients and coachees, the board the client is on or reports to, others in the client organisation etc. etc.. Who is it that wants to know the answer, and what question do they actually want the answer to? Are they all asking the same question or different questions?
Break down of ROI
RoI, Return on Investment, is the straightforward ratio of money made (or saved) divided by the cost of the activity. In cost of the coaching is the cost of the coach in terms of coach’s fees, any additional expenses (travel, room, psychometric test fees etc.) plus the cost of the time taken out of the office by the coachee. For an internal coach, the cost of their time out of their day job may be included. All relatively easily quantified: the time components though are not always included in the calculation.
The value of the coaching outcome is much more difficult to calculate. What was the bottom line impact for the organisation in terms of savings or revenue? Assuming that this can be calculated, what proportion of this was due to the coaching intervention and what proportion was due to all the other factors at play (e.g. simultaneous leadership training or other courses, change to line management attitude or personnel, actions of others, market changes, sales peaks or troughs, other campaigns, projects or strategies etc.) You are hopefully starting to see the challenge here.
And what do you use when direct savings or revenues are not visible? How do you calculate, for example, the value of increase in motivation, the value of better relationships for a manager with her colleagues or customers, the value of a key talent finding the right career path or the value of a new leader letting go of old activities and spending more time on what the new role needs? What is the value of coaching that results in the exit of an employee or the retention of an employee? RoI can be difficult to calculate with any certainty, so how else can we evaluate the benefit of coaching.
Evaluating the outcomes/impacts of coaching
We can come at evaluation from lots of different angles.
Does it work for the individual coachee?
Does it work for the
Will it work for me?
What impact am I having as a coach and could I be even more effective?
To systematise this there is a hierarchy of enquiries we can make to co-design the framework for evaluating coaching with the coachee and client. (This is adapted from DL Kirkpatrick’s work Evaluating Training Programs 1994, published by Berett-Koehler. A summary can be found at http://www.businessballs.com/kirkpatricklearningevaluationmodel.htm .)
To what extent did the coachee feel the time was well-spent?
To what extent did the coachee learn something useful about themselves?
To what extent did the coachee’s behaviour change?
To what extent did the behaviour affect some positive change in their local system or organisation?
To what extent did the change result in some financial benefit to the organisation? (RoI can be calculated from here)
Different measures are needed as we move through questions 1 to 5, and the coach and the client gradually need more input from others as they progress towards question 5
Questions 1 and 2 seek subjective feedback from the coachee and can be answered during each session to check that the conversation is on track, at the end of each session and the end of the coaching assignment.
The behavioural change asked about in Q3 and the impact on the organisation in Q4 require additional feedback from peers, reports and managers, and can be elicited using specific non- leading questions for e.g.
Scaling of the desired behaviours at or before the start of coaching and close to the end are useful ways of measuring change.
Comparing the change perceived by the coachee with that of their peers may provide additional useful information for the coachee and the coach and the organisation.
The peer feedback gathered at the end of the assignment may show positive change; and how sustainable is that change over time?
Inviting others to look out for changes often has a positive effect by provoking useful feedback during the process as well as it incentivising the coachee.
One aim of coaching is to resource the coachee for the future, so I am experimenting with some recent coachees by getting their permission to follow up with them 6 months after the coaching completed. I am interested to find out what changes they are aware of sustaining, what resources they are accessing in themselves and where their behaviour and thinking have moved to now.
As for putting some estimation of value on the desired and actual changes in behaviour of the coachee, I am firmly of the opinion that only the coachee and their organisation (e.g. their manager or HR sponsor) can give any useful answer to this question as the personal and organisational context are key factors.
Interestingly, none of the five questions ask about the quality of the coaching technique or the quality of the relationship. A coach who asks for feedback focusing on levels 1 and 2 will not learn much about their coaching technique or skills and competencies (I’m thinking about the ICF core competencies). A coach who wants to know how good their technique is compared to some standard criteria or their previous level of competence, is also looking at their input; I am focussing here on the coaching output – the impact as a result of coaching. Interestingly, as far as I can tell, the majority of the professional coaching bodies (ICF, AC, EMCC) focus their accreditation on inputs. The Association for Professional Executive Coaches and Supervisors (APECS) takes a more output focussed approach to its accreditation (see Culture driven from the centre: Comparing two coaching bodies by Yvonne Thackray).
Applying modes of evaluation for coaching in organisations and private practice
There is no single answer to how to evaluate coaching. To a large extent it is in the eye of the beholder. So, if you want to bring coaching into your organisation you will probably need to sell the idea to a board or executive team to gain support and resources. What evidence do they want to see or hear? Introduction, development, growth and sustainability of a coaching program, both internal and purchased from external providers, needs a strategy that aligns coaching to the desired organisational culture and direction. Matching your lead and lag metrics to the strategy and its milestones provides useful data on progress and gaps. In addition to the 5 evaluation questions above, measuring the geographical and organisational distribution of demand for coaching, supply of coaches, and analysis of the gaps between supply and demand all support the delivery of a coaching into the organisation. Simple data of this nature can generate lots of useful enquiries: What is the demand? Where is demand highest and lowest? Where might it be needed most? What are the barriers to coaching being asked for? What is creating the demand where it is high?
Persuading stakeholders to engage with and resource coaching?
High level sponsorship is critical for creating a sustainable coaching program. So what will be persuasive to your stakeholders? Maybe success stories from other companies? Are the competition using coaching as a competitive advantage? What example are the blue chip companies setting with coaching? Maybe having a successful personal experience of coaching themselves? Maybe they want an estimate of cost and benefit? Maybe something you have not thought of!
It is easy to make assumptions about what your stakeholders want to know and will be persuaded by, so ask them!
Once your program is in place and have some assignments under your belt, convincing the stakeholders of the ongoing value of coaching can take various forms. Testimonials on the impact from the coachee, their manager, their team members and their colleagues. Summaries of satisfaction scores in post coaching surveys (Q1 and Q2), post coaching 360 surveys (Q3). Careful attention needs to be taken with such data so as to respect confidentiality and permission to share. Changes in staff attitude surveys that can be related back to coaching interventions can be powerful – GlaxoSmithKline has generated convincing positive evidence of coaching impact through their global staff survey by comparing leadership impacts in the teams of those coached with those not coached.
In an ideal world there would be research evidence from controlled, suitably powered experiments that would provide statistical evidence about whether and to what extent coaching works. These seem to be rare. Such experiments are complicated by there being a potentially large placebo effect. In such a relational intervention simply taking someone out of their busy day job for an hour and sitting with them can have a profound effect on their thinking and reflecting. It is a very challenging experiment to design, made even more challenging by the multiple and contradictory definitions of what coaching is.
As usual, starting with the end in mind serves us well when seeking to evaluate executive coaching. “What was the coaching setting out to change and what is the value of that change to the coachee and the organisation?” is a good starting point for evaluating its success. Establishing at the outset how to measure that change enables the baseline and success criteria to be agreed.
When taking on a new corporate and private clients, I now ask “What is the value to you of achieving this goal to you (and your organisation)? and “How will you know you have succeeded?”
When I’m asked “what is the RoI of coaching?” I will continue to answer “What is the value of the change you wish to make?” because I can certainly tell you what my fee will be!
Doug Montgomery is an executive coach, mentor coach, ILM trainer and trainee coach supervisor. He is also an APECS Accredited Executive Coach and an ICF Associate Certified Coach.
He was a Director of Coaching at GlaxoSmithKline’s Coaching Centre of Excellence until 2014 when he set up his private coaching practice Elmbank Coaching Ltd.
Here's how you can connect with Doug:
e: doug@elmbank-coaching.co.uk
m: +44 (0)7712 255297
Or via LinkedIn
