Lately I have been exploring practical ways to collect evidence of our impact as executive coaches (please see APECS Symposium 2015 Papers “ROI:  Collecting Evidence of Our Success”, L. Abrams.) 

In my paper, I advanced the basic idea that, in order for coaches and others to collect actual, true, valid evidence of coaching impact, it is helpful and likely necessary at this point to focus on gathering qualitative data. 

“The Need for Stories from Coaches as Indicators of ROI!”

Spontaneous recollections, or stories, often and unpredictably occur during coaching sessions with a client.  These stories usually reveal how that client has implemented what we had covered during prior coaching conversations, in their subsequent daily life.  They tend to describe the impact of those actions for the client and others in their organization.  However, they don’t necessarily include a description of the impact on the organization, of their changed behavior.

 For the purposes of assessing the impact of coaching, then, I suggest we divorce our expectations of validity (what really happens) from reliability (that it happens every time, allowing a priori prediction), two of the classic hallmarks of solid, well-done research.  Why?  Because at this point in time, isolating and identifying true behavior change in practice requires a more exploratory, qualitative approach.  This should help research in the field of coaching better access the core variables and dynamics, to the point where hypothesis-testing is more realistic.

Qualitative investigation, which would include capturing actual client stories, would thus seem to be appropriate and pragmatic at this time, for both researchers and practitioners.  An example of this kind of qualitative data is what I’m calling in this post, ‘collecting stories from our experience.’

Why Coaches Need to Be Alert for Stories

Coaches are among the best-situated stakeholders to realize that a valuable story is being told and capture it, because:

• These stories are your client descriptions of their own thought processes and consequent behavior changes. These examples of how they have put into practice what they have learned, and any new impact from that new behavior, can occur at any time.

• It is most impactful for all concerned if the coach remembers to ask questions of their client that attempt to specify the tangible impact of their new behavior in terms of meaningful business results, where it is possible to ascertain such. This can extend to dollars earned or saved, as appropriate.

What to Capture:  Characteristics of Your Clients’ Stories, & How to Tell Them

What I’m suggesting and sharing with you is what I am experimenting with at the moment, and what I’ve learned from that, to date.  Based on that, I thought I would put forward a few ideas on what one might want to do, to collect one’s own stories of impact:

- Be an Applied Anthropologist/Qualitative Researcher.  Among the key points that apply from my prior qualitative research courses and work is that it is always necessary to be honest about your biases.  What you notice, and don’t notice; what you ask others, and what you don’t ask; what you record, and don’t record…all of this and more are part of your subjective, inherently biased point of view.  It’s OK, and inevitable.  But to the degree that you can be as honest about your own biases as possible, and do your best to restrain them and make every effort to be as objective as possible, it will help your data gain validity, credibility, and durability.

- Be a Story-Teller. While you are reporting a real, true sequence of events, it is a story nonetheless.  Provide the details that make it real, for yourself as well as for any reader, even while protecting confidentiality.  Give the person a name (though I always freely admit to using fake names, for the sake of confidentiality.)  Provide the approximate size of organization, and the type of industry (i.e., “global pharmaceutical”,) though I usually avoid giving the organization’s name.  Alternatively, you can name the client’s function.  Just ensure there is not enough information to describe exactly who they are, when there is only one of them (i.e., “IBM’s North American head of supply chain.”) 

After describing the client’s basic demographics, specifically and vividly describe their behaviors and thought-processes, prior to coaching, which had been problematic, as per their stakeholder feedback and/or self-report.  Describe the (presumably negative) impact of those prior behaviors as much as possible as well.  Then, describe the insight or transformative experience your client had during coaching that lead to a new way of thinking and acting.  Describe a situation where there was a temptation for the old behavior, but where they instead displayed new behavior, in line with the transformative coaching experience they have described.  Include a description of their new thought-process and/or the actual behaviors of your client.  Then, include the results for the client as well as others in the organization, and/or the organization itself, from the new behaviors.

- Ask the Question.  In terms of results, this is the most important thing to do:  Ask the question.  You may well not hear the impact of the story on the organization, or others, if you don’t ask for it.  And your client is among the best-placed to describe the impact and outcomes of your coaching work, and certainly the best one to describe the before-and-after thought-processes and effects.  Only that person knows all the details of what they used to say, and do, and what the change in their thinking and actions via coaching were.  They are centrally-placed to observe the results of their own changes, for themselves and for their organization, from their application of their learning from your coaching work together.  Their success is your success.  But you may never learn what that is, unless you remember to consciously ask them the specific questions that elicit it.  For example, when they tell you a story about what they have thought about and/or applied since your last conversation, remember to ask them the follow-up question about impact:  So what happened, as a result of that new behavior? And don’t forget to…

- Quantify the Impact. In addition to asking about the impact on others, look for the impact on the organization, which includes ways to quantify their impact.  For example, “What was the amount saved/earned that was new?”  I have noticed that this last point is not typically an organic part of a coaching conversation – either for them to volunteer the concrete impact, or to specify a financial amount relevant to that impact.  You will likely have to ask, specifically, for the indexing number.  You may not get one, but you can try! 

- Be an Excellent Librarian.  The rest is housekeeping.  Record the story immediately.  Store it in a (virtual) place where you will be sure to find it again.  Soon you will have a collection!  So consider giving each incident keywords.

Now, I’d like to share an example of how I use Story-Telling (what is in bold and italics is an item that illustrates the suggestions above):

Tony* is head of Supply-Chain for a global pharmaceutical company.  Over the course of our coaching conversations, he has realized that the team he inherited is comprised of nice, complacent individuals, who overall are performing their work to outdated standards.  Tony and the organization both need them to step up their performance, to ensure his function’s effectiveness and longer-term success.

Tony himself is strategic, action-oriented, organized, and driven.  However, one of his personal challenges is conflict avoidance. In other settings, he had allowed mediocre performers to continue long-term in his group, which, as one of his stakeholders told me, left both him and the organization vulnerable when crises arose.  Early on in his leadership tenure, he single-handedly initiated a new vision and set of initiatives for his team, to raise their level of client service and upfront relevance to the business.  Since then, however, over the course of our coaching work together, Tony has realized how far of his ideal infrastructure his team is falling short, in both mindset and behavior. 

To do this, Tony needs to effect change in some of the core beliefs and behaviors in his team.  Here’s an example:  Tony’s long-tenured predecessor rewarded this team’s loyalty and tenure over performance.  They therefore learned to do this as leaders themselves.  One manager, Gloria, needed an open position filled.  At first, she wanted to promote Marcy, an existing direct report, into that spot.  Tony sees Marcy as competent but not excellent in her current job, and certainly not ready for promotion.  Gloria was initially taken aback by his dissent, but once she accepted it, she had to break the news to Marcy and her team that none of them would be accepted for the promotion.  Gloria emailed Tony immediately after that group meeting, describing it as personally quite “tough” for her, since her team was very upset by the news and tearful.  She then asked Tony to come himself to talk to them, presumably to explain the decision and somehow assuage their upset. 

Tony saw from this request that he needed to confront and coach Gloria clearly on her assumptions and behavior as a leader.  He needed to ensure she understood his new expected levels of performance for her, and therefore her team, including both making and owning leadership decisions.  She also needed to clarify the new higher performance standards required of her own team, and coach, develop and/or manage them appropriately depending on their success in attaining these. 

By discussing this incident and others preceding it during coaching, Tony developed a list of competencies he now wants to implement when hiring, coaching, and evaluating staff, to create a team that is better suited to the organization’s current and future needs.

I asked Tony if there is any way to concretize the benefit of implementing these higher performance standards, for the organization. He pondered that, and we agreed that the benefits were likely more in the realm of long-term efficiency and effectiveness, since his goal is strengthening the team’s effectiveness and capabilities into the future. We will return to the topic of quantification of these standards before the end of the coaching engagement, and see whether anything appropriate emerges.

*All names changed for confidentiality purposes

Moving forward:

As you read the above, what thoughts did you have regarding:

• What you might do, to start to collecting your own stories of coaching impact?

• What use(s) you might put these to?

• What else you might suggest or do differently?

• What benefits do you see in sharing such stories amongst like-minded professionals?

• What difficulties, and any solutions to those, might you suggest, to explore our impact using our stories?

• Anything else?

I’d love to hear your thoughts.  Please feel free to share your thoughts and/or ideas in the comment box below, and later I can share how this experiment is progressing.

To connect with Dr. Lilian Abrams:

LinkedIn profile:  Dr. Lilian Abrams

Email:  labrams@abramsandassociatesllc.com

Website:  www.abramsandassociatesllc.com