"Beyond personality assessments: What the coach can learn about patterns of behaviour and their implications" by Lynne Hindmarch (guest)

The purpose of this blog is not to defend or promote the use of assessments in coaching.  Lots of coaches are very successful without using them at all.  What I am interested in sharing (and debating) is:

  • How they can be used

  • What they can teach the experienced practitioner about patterns of behaviour.

In my first introductory piece I will discuss what we mean by psychometric assessments, give some examples of ones that are appropriate for coaching, and show how coaches can deepen their understanding of behaviour by using assessments.

Psychometric tests are designed to measure differences between people.  The term covers both ability tests and personality assessments.  I will focus on personality assessments in these blogs, as ability tests (such as verbal, numerical and abstract reasoning) are not generally used as part of a coaching programme. Examples of personality assessments that are frequently used in coaching are: the Myers Briggs Type Indicator, the16PF, the OPQ, and NEO (which will be discussed in more detail in my next blog).  They are different to most of the free tests that can be accessed online in that they have to meet certain technical criteria in the way they have been developed.   They must be:

  1. Reliable (consistent across time with different people and different applications),

  2. Valid (measure what they say they are measuring),

  3. Free from bias such as gender and race, and

  4. Standardised to minimise human error and bias.

This is why access to psychometric assessments is only provided to people who have been trained in their use.  Training involves learning about the technical and statistical underpinning of the assessments, and how to feed back the results to the client. 

Using a personality assessment allows the trained practitioner to form hypotheses about the client’s behavioural style, and that is what is explored in the feedback session.  Basically, the assessments form the framework for a good conversation; the focus of that conversation will be determined by the purpose of the coaching, such as personal development or career exploration. 


Coaching as a goal-oriented activity

Early in a coaching programme the coach is likely to ask the client in some form: ‘Where are you now?  And where do you want to be?’.  There are different ways to explore where a client is now, such as gathering biographical information, and using a model such as the Wheel of Life.  Using a personality assessment (or a combination of assessments) is an additional approach to help raise the client’s self-awareness, and help the coach better understand the client.

However, my interest in the benefits of using personality assessments is rather more than that.   Many coaches train in assessments once they have qualified with a coach training institute, and personality assessments are viewed as an additional tool in their kitbag. 

My own experience is somewhat different.

I was first trained in personality assessments over 25 years ago, and had been using them extensively for a long time before I became involved in coaching, team building and coaching supervision.  In fact my route into coaching was through the understanding of behaviour, and patterns of behaviour, that using personality assessments had provided.  I started working as an associate for a major outplacement consultancy, using assessments (at that time mainly the 16PF and the MBTI) with redundant senior executives.   The feedback conversation usually focused on:

  • What their behavioural style meant in terms of organisational ‘fit’,

  • How they would approach the job search (interview style, networking inclination, commitment to the process), and

  • Career implications.

I found the process of giving feedback, and the interaction with the client, absolutely fascinating.  The consultancy allowed a generous amount of time with clients: an initial session lasting about 2 hours, enabling me to obtain biographical information and administer the tests - paper and pencil in those days!.  This was followed a week or so later by a 3 hour feedback session, providing a good amount of time for me to explore the implications of the results with the client.   These sessions were, in effect, mini coaching sessions.  From this, I was asked to take on actual coaching programmes, particularly in the area know as emotional intelligence, still using the assessments as an initial part of the coaching relationship.   As this part of my practice developed, I took further qualifications in coaching and coaching supervision.

The point of this description of my route into coaching is this: one of the exciting things that a lengthy experience of using personality assessments can give the practitioner is a deepening understanding of behaviour, and patterns of behaviour.  These can provide insights that feed into the coaching goals.  The opportunities for the coach to learn more about people’s behaviour are presented every time he or she carries out feedback.  Using assessments in a development setting, the client is more likely to be open in discussing the results than when it is used in selection.  The alert practitioner can learn a lot from each feedback meeting.   Each discussion can yield rich information about how the client ‘lives’ the profile.  Over time, this depth of understanding can provide great insight into how behaviours impact on each other, and what that means in the way the individual manages themselves and their relationships.   


For example, what does it mean for the coach if a person is low on self-discipline, is emotionally resilient and is also a conceptual thinker?  I can tell you as I saw such a client recently.   I’ll call him Peter.  It was the first time I’d met Peter, as I was profiling him for his coach, who isn’t trained in psychometrics.  I analysed his personality assessments before I met him, so I knew it was likely that he would be late for our session (he was).  He was late because typically he doesn’t plan ahead (so he hadn’t allowed time for parking), has a positive outlook (so doesn’t factor in negative possibilities such as traffic delays), and is absent-minded (likely to mislay his car keys).

This is a light-hearted example, and of course this pattern of behaviour has much wider implications.   But the point is that a depth of understanding of personality assessments can provide the coach with insight into the client’s behavioural style, with implications for how the client is likely to interact with the coach and the coaching programme and clarify what their development needs may be in relation to their coaching goals.  In the example I gave above, the feedback to Peter on these particular aspects of his behaviour included discussion about the positive aspects of his creativity, flexibility and generally upbeat outlook, and also how those around him might perceive these in a negative light on occasion: he might come across as having somewhat eccentric, unrealistic ideas, not following through on commitments, and overlooking or ignoring potential difficulties.  

I had contracted with Peter to share the results with his coach.  I was able to share with her that Peter was likely to be effective at coming up with imaginative options for addressing his coaching goals, but one of the coaching challenges would be to help him break down his ideas into practical steps that he could work through.   Planning his goals and the steps to reach them would also be helpful in encouraging him to focus and organise himself more effectively.  His positive outlook is likely to help him in believing it is possible for him to achieve his goals, but may mean he underestimates the difficulties along the way.

Over time, as one’s understanding of personality increases, it is fascinating to be able to observe just one aspect of a person’s behaviour (without using assessments) and see how a pattern may emerge. Let me share some other examples,

  • If you note that a person is ideas-oriented rather than practical, they may be imaginative but also absent-minded, accident-prone and unrealistic. (They don’t generally like DIY. You wouldn’t trust them with a hammer.)

  • Observing that a person doesn’t plan very much suggests that they may be flexible and adaptable, and also disorganised, last-minute, and only motivated to complete something if they really enjoy it.

  • Those who have a positive outlook may be cheerful and upbeat and may not be aware of drawbacks and may take risks.

This is just to provide a taster of what I plan to cover in future blogs.  I will be working through other examples of behavioural clusters, exploring what they mean from the perspectives of the person giving the feedback, the client, and the coach.   The purpose is to share insights I have gleaned over the years into the subtleties of behaviour, and how observing one aspect of behaviour can provide an understanding of associated characteristics.   That is the really fun bit!

To connect with Lynne Hindmarch:

  Business Psychologist

Business Psychologist

Organisational Behaviour Consultancy

 

email: lhindmarch@obc.org.uk

mobile: 07977 129955.

Collecting Stories from Our Experience by Dr. Lilian Abrams (guest)

Lately I have been exploring practical ways to collect evidence of our impact as executive coaches (please see APECS Symposium 2015 Papers “ROI:  Collecting Evidence of Our Success”, L. Abrams.

In my paper, I advanced the basic idea that, in order for coaches and others to collect actual, true, valid evidence of coaching impact, it is helpful and likely necessary at this point to focus on gathering qualitative data. 

“The Need for Stories from Coaches as Indicators of ROI!”

Spontaneous recollections, or stories, often and unpredictably occur during coaching sessions with a client.  These stories usually reveal how that client has implemented what we had covered during prior coaching conversations, in their subsequent daily life.  They tend to describe the impact of those actions for the client and others in their organization.  However, they don’t necessarily include a description of the impact on the organization, of their changed behavior.

 For the purposes of assessing the impact of coaching, then, I suggest we divorce our expectations of validity (what really happens) from reliability (that it happens every time, allowing a priori prediction), two of the classic hallmarks of solid, well-done research.  Why?  Because at this point in time, isolating and identifying true behavior change in practice requires a more exploratory, qualitative approach.  This should help research in the field of coaching better access the core variables and dynamics, to the point where hypothesis-testing is more realistic.

Qualitative investigation, which would include capturing actual client stories, would thus seem to be appropriate and pragmatic at this time, for both researchers and practitioners.  An example of this kind of qualitative data is what I’m calling in this post, ‘collecting stories from our experience.’

Why Coaches Need to Be Alert for Stories

Coaches are among the best-situated stakeholders to realize that a valuable story is being told and capture it, because:

  • These stories are your client descriptions of their own thought processes and consequent behavior changes. These examples of how they have put into practice what they have learned, and any new impact from that new behavior, can occur at any time.

  • It is most impactful for all concerned if the coach remembers to ask questions of their client that attempt to specify the tangible impact of their new behavior in terms of meaningful business results, where it is possible to ascertain such. This can extend to dollars earned or saved, as appropriate.

What to Capture:  Characteristics of Your Clients’ Stories, & How to Tell Them

What I’m suggesting and sharing with you is what I am experimenting with at the moment, and what I’ve learned from that, to date.  Based on that, I thought I would put forward a few ideas on what one might want to do, to collect one’s own stories of impact:

- Be an Applied Anthropologist/Qualitative Researcher.  Among the key points that apply from my prior qualitative research courses and work is that it is always necessary to be honest about your biases.  What you notice, and don’t notice; what you ask others, and what you don’t ask; what you record, and don’t record…all of this and more are part of your subjective, inherently biased point of view.  It’s OK, and inevitable.  But to the degree that you can be as honest about your own biases as possible, and do your best to restrain them and make every effort to be as objective as possible, it will help your data gain validity, credibility, and durability.

- Be a Story-Teller. While you are reporting a real, true sequence of events, it is a story nonetheless.  Provide the details that make it real, for yourself as well as for any reader, even while protecting confidentiality.  Give the person a name (though I always freely admit to using fake names, for the sake of confidentiality.)  Provide the approximate size of organization, and the type of industry (i.e., “global pharmaceutical”,) though I usually avoid giving the organization’s name.  Alternatively, you can name the client’s function.  Just ensure there is not enough information to describe exactly who they are, when there is only one of them (i.e., “IBM’s North American head of supply chain.”) 

After describing the client’s basic demographics, specifically and vividly describe their behaviors and thought-processes, prior to coaching, which had been problematic, as per their stakeholder feedback and/or self-report.  Describe the (presumably negative) impact of those prior behaviors as much as possible as well.  Then, describe the insight or transformative experience your client had during coaching that lead to a new way of thinking and acting.  Describe a situation where there was a temptation for the old behavior, but where they instead displayed new behavior, in line with the transformative coaching experience they have described.  Include a description of their new thought-process and/or the actual behaviors of your client.  Then, include the results for the client as well as others in the organization, and/or the organization itself, from the new behaviors.

Ask the Question.  In terms of results, this is the most important thing to do:  Ask the question.  You may well not hear the impact of the story on the organization, or others, if you don’t ask for it.  And your client is among the best-placed to describe the impact and outcomes of your coaching work, and certainly the best one to describe the before-and-after thought-processes and effects.  Only that person knows all the details of what they used to say, and do, and what the change in their thinking and actions via coaching were.  They are centrally-placed to observe the results of their own changes, for themselves and for their organization, from their application of their learning from your coaching work together.  Their success is your success.  But you may never learn what that is, unless you remember to consciously ask them the specific questions that elicit it.  For example, when they tell you a story about what they have thought about and/or applied since your last conversation, remember to ask them the follow-up question about impact:  So what happened, as a result of that new behavior? And don’t forget to…

- Quantify the Impact. In addition to asking about the impact on others, look for the impact on the organization, which includes ways to quantify their impact.  For example, “What was the amount saved/earned that was new?”  I have noticed that this last point is not typically an organic part of a coaching conversation – either for them to volunteer the concrete impact, or to specify a financial amount relevant to that impact.  You will likely have to ask, specifically, for the indexing number.  You may not get one, but you can try! 

- Be an Excellent Librarian.  The rest is housekeeping.  Record the story immediately.  Store it in a (virtual) place where you will be sure to find it again.  Soon you will have a collection!  So consider giving each incident keywords.

Now, I’d like to share an example of how I use Story-Telling (what is in bold and italics is an item that illustrates the suggestions above):


Tony* is head of Supply-Chain for a global pharmaceutical company.  Over the course of our coaching conversations, he has realized that the team he inherited is comprised of nice, complacent individuals, who overall are performing their work to outdated standards.  Tony and the organization both need them to step up their performance, to ensure his function’s effectiveness and longer-term success.

Tony himself is strategic, action-oriented, organized, and driven.  However, one of his personal challenges is conflict avoidance. In other settings, he had allowed mediocre performers to continue long-term in his group, which, as one of his stakeholders told me, left both him and the organization vulnerable when crises arose.  Early on in his leadership tenure, he single-handedly initiated a new vision and set of initiatives for his team, to raise their level of client service and upfront relevance to the business.  Since then, however, over the course of our coaching work together, Tony has realized how far of his ideal infrastructure his team is falling short, in both mindset and behavior. 

To do this, Tony needs to effect change in some of the core beliefs and behaviors in his team.  Here’s an example:  Tony’s long-tenured predecessor rewarded this team’s loyalty and tenure over performance.  They therefore learned to do this as leaders themselves.  One manager, Gloria, needed an open position filled.  At first, she wanted to promote Marcy, an existing direct report, into that spot.  Tony sees Marcy as competent but not excellent in her current job, and certainly not ready for promotion.  Gloria was initially taken aback by his dissent, but once she accepted it, she had to break the news to Marcy and her team that none of them would be accepted for the promotion.  Gloria emailed Tony immediately after that group meeting, describing it as personally quite “tough” for her, since her team was very upset by the news and tearful.  She then asked Tony to come himself to talk to them, presumably to explain the decision and somehow assuage their upset. 

Tony saw from this request that he needed to confront and coach Gloria clearly on her assumptions and behavior as a leader.  He needed to ensure she understood his new expected levels of performance for her, and therefore her team, including both making and owning leadership decisions.  She also needed to clarify the new higher performance standards required of her own team, and coach, develop and/or manage them appropriately depending on their success in attaining these. 

By discussing this incident and others preceding it during coaching, Tony developed a list of competencies he now wants to implement when hiring, coaching, and evaluating staff, to create a team that is better suited to the organization’s current and future needs.

I asked Tony if there is any way to concretize the benefit of implementing these higher performance standards, for the organization. He pondered that, and we agreed that the benefits were likely more in the realm of long-term efficiency and effectiveness, since his goal is strengthening the team’s effectiveness and capabilities into the future. We will return to the topic of quantification of these standards before the end of the coaching engagement, and see whether anything appropriate emerges.

*All names changed for confidentiality purposes


Moving forward:

As you read the above, what thoughts did you have regarding:

  • What you might do, to start to collecting your own stories of coaching impact?

  • What use(s) you might put these to?

  • What else you might suggest or do differently?

  • What benefits do you see in sharing such stories amongst like-minded professionals?

  • What difficulties, and any solutions to those, might you suggest, to explore our impact using our stories?

  • Anything else?

I’d love to hear your thoughts.  Please feel free to share your thoughts and/or ideas in the comment box below, and later I can share how this experiment is progressing.

To connect with Dr. Lilian Abrams:

The practicalities of measuring the RoI of Coaching: A reflection by Doug Montgomery (Guest)

What is the Return on Investment (RoI) of coaching?   

How do you measure the RoI of coaching?  

I have come across these questions a great deal over the last few years.  And it keeps occurring to me:  Who wants this information and why?   Is RoI the right or most useful thing to measure?   

Who’s asking?

In executive coaching, there are multiple parties involved and interested in how to evaluate coaching.   There is the coach, the coachee, the coachee’s manager, the sponsor (who may or may not be the payee),  the client – who is the payee,  potential new clients and coachees, the board the client is on or reports to, others in the client organisation etc. etc..   Who is it that wants to know the answer, and what question do they actually want the answer to?  Are they all asking the same question or different questions?

Break down of ROI

RoI, Return on Investment, is the straightforward ratio of money made (or saved) divided by the cost of the activity.   In cost of the coaching is the cost of the coach in terms of coach’s fees, any additional expenses (travel, room, psychometric test fees etc.) plus the cost of the time taken out of the office by the coachee.  For an internal coach, the cost of their time out of their day job may be included.  All relatively easily quantified: the time components though are not always included in the calculation.  

The value of the coaching outcome is much more difficult to calculate.   What was the bottom line impact for the organisation in terms of savings or revenue?   Assuming that this can be calculated, what proportion of this was due to the coaching intervention and what proportion was due to all the other factors at play (e.g. simultaneous leadership training or other courses, change to line management attitude or personnel, actions of others, market changes, sales peaks or troughs, other campaigns, projects or strategies etc.)   You are hopefully starting to see the challenge here.  

And what do you use when direct savings or revenues are not visible?   How do you calculate, for example, the value of increase in motivation, the value of better relationships for a manager with her colleagues or customers,  the value of a key talent finding the right career path or the value of a new leader letting go of old activities and spending more time on what the new role needs?  What is the value of coaching that results in the exit of an employee or the retention of an employee?  RoI can be difficult to calculate with any certainty, so how else can we evaluate the benefit of coaching.

Evaluating the outcomes/impacts of coaching

We can come at evaluation from lots of different angles.

  • Does it work for the individual coachee?

  • Does it work for the

  • Will it work for me?

  • What impact am I having as a coach and could I be even more effective?

To systematise this there is a hierarchy of enquiries we can make to co-design the framework for evaluating coaching with the coachee and client.  (This is adapted from DL Kirkpatrick’s work Evaluating Training Programs 1994, published by Berett-Koehler.  A summary can be found at http://www.businessballs.com/kirkpatricklearningevaluationmodel.htm .)

  1. To what extent did the coachee feel the time was well-spent?

  2. To what extent did the coachee learn something useful about themselves?

  3. To what extent did the coachee’s behaviour change?

  4. To what extent did the behaviour affect some positive change in their local system or organisation?

  5. To what extent did the change result in some financial benefit to the organisation? (RoI can be calculated from here)

Different measures are needed as we move through questions 1 to 5, and the coach and the client gradually need more input from others as they progress towards question 5

Questions 1 and 2 seek subjective feedback from the coachee and can be answered during each session to check that the conversation is on track, at the end of each session and the end of the coaching assignment. 

The behavioural change asked about in Q3 and the impact on the organisation in Q4 require additional feedback from peers, reports and managers, and can be elicited using specific non- leading questions for e.g.

  • Scaling of the desired behaviours at or before the start of coaching and close to the end are useful ways of measuring change.

  • Comparing the change perceived by the coachee with that of their peers may provide additional useful information for the coachee and the coach and the organisation.

  • The peer feedback gathered at the end of the assignment may show positive change; and how sustainable is that change over time?

Inviting others to look out for changes often has a positive effect by provoking useful feedback during the process as well as it incentivising the coachee.  

One aim of coaching is to resource the coachee for the future, so I am experimenting with some recent coachees by getting their permission to follow up with them 6 months after the coaching completed.  I am interested to find out what changes they are aware of sustaining, what resources they are accessing in themselves and where their behaviour and thinking have moved to now.

As for putting some estimation of value on the desired and actual changes in behaviour of the coachee, I am firmly of the opinion that only the coachee and their organisation (e.g. their manager or HR sponsor) can give any useful answer to this question as the personal and organisational context are key factors.  

Interestingly, none of the five questions ask about the quality of the coaching technique or the quality of the relationship.   A coach who asks for feedback focusing on levels 1 and 2 will not learn much about their coaching technique or skills and competencies (I’m thinking about the ICF core competencies).  A coach who wants to know how good their technique is compared to some standard criteria or their previous level of competence, is also looking at their input; I am focussing here on the coaching output – the impact as a result of coaching.   Interestingly, as far as I can tell, the majority of the professional coaching bodies (ICF, AC, EMCC) focus their accreditation on inputs.    The Association for Professional Executive Coaches and Supervisors (APECS) takes a more output focussed approach to its accreditation (see Culture driven from the centre: Comparing two coaching bodies by Yvonne Thackray).

Applying modes of evaluation for coaching in organisations and private practice

There is no single answer to how to evaluate coaching. To a large extent it is in the eye of the beholder.  So, if you want to bring coaching into your organisation you will probably need to sell the idea to a board or executive team to gain support and resources. What evidence do they want to see or hear?  Introduction, development, growth and sustainability of a coaching program, both internal and purchased from external providers, needs a strategy that aligns coaching to the desired organisational culture and direction.  Matching your lead and lag metrics to the strategy and its milestones provides useful data on progress and gaps.   In addition to the 5 evaluation questions above, measuring the geographical and organisational distribution of demand for coaching, supply of coaches, and analysis of the gaps between supply and demand all support the delivery of a coaching into the organisation.  Simple data of this nature can generate lots of useful enquiries: What is the demand? Where is demand highest and lowest?  Where might it be needed most?    What are the barriers to coaching being asked for?  What is creating the demand where it is high?

Persuading stakeholders to engage with and resource coaching?

High level sponsorship is critical for creating a sustainable coaching program.   So what will be persuasive to your stakeholders?   Maybe success stories from other companies?  Are the competition using coaching as a competitive advantage?  What example are the blue chip companies setting with coaching?   Maybe having a successful personal experience of coaching themselves?   Maybe they want an estimate of cost and benefit?  Maybe something you have not thought of! 

It is easy to make assumptions about what your stakeholders want to know and will be persuaded by, so ask them!

Once your program is in place and have some assignments under your belt, convincing the stakeholders of the ongoing value of coaching can take various forms.   Testimonials on the impact from the coachee, their manager, their team members and their colleagues.  Summaries of satisfaction scores in post coaching surveys (Q1 and Q2), post coaching 360 surveys (Q3).   Careful attention needs to be taken with such data so as to respect confidentiality and permission to share.   Changes in staff attitude surveys that can be related back to coaching interventions can be powerful –  GlaxoSmithKline has generated convincing positive evidence of coaching impact through their global staff survey by comparing leadership impacts in the teams of those coached with those not coached. 

In an ideal world there would be research evidence from controlled, suitably powered experiments that would provide statistical evidence about whether and to what extent coaching works.   These seem to be rare.  Such experiments are complicated by there being a potentially large placebo effect.  In such a relational intervention simply taking someone out of their busy day job for an hour and sitting with them can have a profound effect on their thinking and reflecting. It is a very challenging experiment to design, made even more challenging by the multiple and contradictory definitions of what coaching is.  

As usual, starting with the end in mind serves us well when seeking to evaluate executive coaching.   “What was the coaching setting out to change and what is the value of that change to the coachee and the organisation?”  is a good starting point for evaluating its success.   Establishing at the outset how to measure that change enables the baseline and success criteria to be agreed.

When taking on a new corporate and private clients, I now ask “What is the value to you of achieving this goal to you (and your organisation)?  and “How will you know you have succeeded?”

When I’m asked “what is the RoI of coaching?” I will continue to answer “What is the value of the change you wish to make?” because I can certainly tell you what my fee will be!

 Doug Montgomery is an executive coach, mentor coach, ILM trainer and trainee coach supervisor. He is also an APECS Accredited Executive Coach and an ICF Associate Certified Coach.

Doug Montgomery is an executive coach, mentor coach, ILM trainer and trainee coach supervisor. He is also an APECS Accredited Executive Coach and an ICF Associate Certified Coach.

He was a Director of Coaching at GlaxoSmithKline’s Coaching Centre of Excellence until 2014 when he set up his private coaching practice Elmbank Coaching Ltd. 

Here's how you can connect with Doug:

e: doug@elmbank-coaching.co.uk

m: +44 (0)7712 255297

Or via LinkedIn