Measuring Automation Benefits - what's your approach?

[Michael_S]

Hey everyone,

Internally, we've been sharing examples of how automation teams prove that their efforts are realizing benefits for their organization. We all know the default approach - FTEs and time saved - but we're increasingly seeing other examples of "benefits realization" that innovative organizations are using to prove ROI, and increase budgets.

These examples include:

• Efficiency gains
• Costs avoided
• Risk mitigation
• Opportunities gained by time returned to business
• Customer experience improvements

My question to you all is:

1. How do you measure the value of your automation program now?
2. If you had complete control over how you measured ROI through automation - what would you change?

My hope for this discussion is that we can share examples and ideas that inspire each other to find new ways of realizing, and proving, the benefits of automation.

Please dive in and share your examples!

[Michael_S]

Would love to hear from some of our automation leads and MVPs - @grbanares, @mwulff, @michaeloneil, @asilarow, @Jignesh-JK 💙

[stepher]

This is probably not exactly in line with the question, but I do think it speaks to other forms of 'benefit realization.'

In Healthcare, ‘Call Campaigns’ are a constant.  We contact patients to schedule appointments, to confirm appointments, to confirm demographic/payment information, and a whole host of other reasons.  These contacts (email, text or call) are documented in the patients’ medical records.  Most of the related ROI calculations are the standard ‘it takes a person this long’ against ‘it takes a digital employee this long’ for time and the dollar savings.  One project comes to mind that was a bit different.

We had a “Mammography Outreach Call Campaign”.  The idea/workflow was straightforward.  Every month a list of patients who met specific criteria (the most critical was that they had not had a mammogram in the last twelve months) was created and set into a pool.  Everyday, a set number of these records were contacted from that pool to remind them of the benefits of the scan and offered a number to call back and schedule it.  These patient contacts were then documented in their medical record.

Again, nothing out of the ordinary in the process…except that this was net new work.  So, the standard ROI calculations are a bit off, as no one was doing this work beforehand.  Fortunately, because of the engagement of the Business Owner, we were able to tie when patients scheduled a scan after having received a reminder call.  The Business Owner was then able to estimate the increase in revenue to the organization (millions of dollars).  More importantly, though, was an estimation of the number of patient lives positively impacted because something was identified in a related mammogram.

I have automated a number of Call Campaigns within our organization.  One of the common critiques is that they generally have a low patient response rate.  This project was no different--I think the response rate was under 5%.  But that still amounted to a very meaningful return on the investment.

For reasons beyond our control, the automation was shuttered late last year.  My Manager would be happy to confirm that I still bring it up from time to time.  It is one of those projects that, in hindsight, reminds me why I do what I do, and, more importantly, why I choose to do this work in Healthcare.

Thanks,

Red

[Michael_S]

That's beautiful @stepher - thanks so much for sharing! Achieving a human-centric mission, as well as the value ROI, is such a motivating part of the job. I hope you're as proud of that story as I am grateful to read it 💙

I imagine @jessetutt and @LesleyCase would love to give this a read too!

[jessetutt]

Hi Michael.

Thanks for the mention. 

My experience is that benefit realization specific to RPA or Intelligent Automation needs to report on:

- effort freed to date,

- effort freed annually,

- $ freed to date and

- $ freed annually.

My personal opinion is that non-measurable benefits are great to track on intake and potentially report on in an update, but the primary focus must be on the effort and $ freed.

More mature organizations then split effort and $ freed into cost savings (immediately freeing up money to be spent on other things) and cost avoidance (avoidance of something you would have in the future needed to spend money on).

This is complicated naturally as we have automations in our organization that automated work that needed to be done, but couldn't since the team didn't have capacity so the source team couldn't give up the budget...is that cost savings (I think) or cost avoidance?

An automation which frees 5 years of work per year in a department that has 5 years more demand than they can handle truly frees 5 years, but again the department doesn't have peopel to do it all so you are really just automating their backlog. Again in my eyes cost savings, but for most finance people it's cost avoidance.

Because of all of this, we just record effort and $ freed (merging both cost avoidance and cost savings together).

I do know some organizations record effort and $ freed that ultimately reduced the team or budget and effort and $ freed which did not which I like, but we haven't got there.

No matter what don't limit your reporting to pure cost savings or your program will fail is my advice.

Good luck and feel free to reach out anytime - jesse.tutt    @this domain-->      albertahealthservices.ca.

[mwulff]

I think we took the most down to earth way of measuring the benefits of the program. We simply track each completed task from a process and we assign a monetary value to the task.

So we do a simple tasks * value multiplication and sum up all of our processes. This gives us a monetary value over time (tracked pr day) and we can report that upward in the chain of command.

Now it is well understood that this number is fictional and has little to do with reality, since the real benefits are much more varied and diverse. This is just something we accept and move on with our lives.

In short we track as a little as possible to ensure we have an indicator of our progress but nothing more. We would build automations than metrics.