Top 10 Reasons Why Process Automation Projects Fail

In this post, we discuss the common reasons leading to automation project failure. First, what sort of projects are we talking about?

We refer specifically to projects that aim to deliver automated versions of existing business processes. These include producing billing reports, management exception reports and revenue forecasting. What these processes have in common is:

• They are data intensive, typically using multiple spreadsheets, extracted files from legacy systems and other data sources
• They are being performed by skilled staff in a highly manual fashion
• They are recurring processes, usually running monthly, weekly or daily
• They comprise many steps (30 or more) that must be repeated manually each time the process is executed
• They produce outputs that are important or critical for running the business
• They have been running for years, had multiple owners and no-one is 100 per cent sure of how every step in the process should be done. 

Processes like these take place every day, week, month or quarter in every organisation. We find them in finance departments, sales teams, operations environments and elsewhere; they are the data and information pipelines that add value to the modern organisation. When a company decides to digitise one of these processes, with the goal of making it faster, more efficient, more transparent, less risky and more flexible, that’s where Solvexia steps in – and that’s when some familiar scenarios arise. So, with this type of everyday business process in mind, here are the top 10 problems we see occur in real-world projects...

Coming Up

1 - Underestimating the impact of re‑engineering the existing process before automating

2 - Choosing to automate a process that is not suited for automation

3 - Not understanding how the various organisational processes fit together

4 - Subject matter experts not available to capture and codify their expertise

5 - The project runs for too long due to a start-stop-start again resourcing mode

6 - Insufficiently active senior executive mandate

7 - Lack of clarity around how success will be measured

8 - Insufficient skills and experience in the project team around structured and procedural thinking techniques

9 - Tackling a large or difficult process first

10 - Not taking the time to fully understand the manual process or requirements

About SolveXia

1 - Underestimating the impact of re‑engineering the existing process before automating

When seeking to automate a business process, a natural tension emerges between automating what is happening now, and a projection of what the various stakeholders would like to happen in the future.

This refinement is normally called ‘cleaning up’ or ‘streamlining’ and is communicated as something that will make the project easier to deliver. In our experience, close analysis of the existing process often reveals conflicting understandings of that process – even surprise and bewilderment at how it really works. Some people were not aware of particular details; others didn’t fully understand the process to begin with. Sometimes this happens from successive miscommunication, passed on from outgoing to incoming owners of the process, or because certain steps have become redundant. Without exception, we have found that most significant processes being used at any business for more than a year contain redundant or erroneous steps.

In this situation, most project teams will attempt to re-engineer or clean up the process. They will seek to remove or refine steps that don’t seem to fit anymore, viewing the whole exercise as a simplification measure.
They will reduce the project risk and possibly even compress the timeline. Unfortunately, this is rarely the right approach.

When only a few steps are being remodelled (for example, fewer than three in a 40-step process), the reduction could indeed help the project. However, most steps in a process are dependent on other steps. Refining step A may have an adverse effect on steps B, C and F that must then also be addressed. The process is similar to pulling the loose thread in a sock, which only results in more unravelling.

Before long, the mere ‘cleanup’ has structurally changed the nature of the process, and the scale of change is larger than intended. Meanwhile, the planned effort for testing and verification remains minimal. This results in an automated process that is (a) effectively a new process for the organisation and (b) inadequately tested and verified. This problem, however, creeps up on the project while it is running – it rarely manifests itself at the start of the project.

Our recommendations:

• When drawing up the project scope, make clear decisions on the extent to which the process will (or won’t) be refined during the project. 
• Establish a review committee to evaluate the impact of any proposed process refinements as the project progresses. This group should meet at least weekly and assess the impact of changes on the testing and verification budget allocated to the project. They should also look for downstream side effects from the changes.

2 - Choosing to automate a process that is not suited for automation

Not all business processes are equally suited to being automated. Further, not all steps in a process are equally suited for automation.

When teams choose to optimise ill-suited processes, the benefits derived may seem disappointing even if all has gone smoothly.
Some common examples of poor choice include:

Too simple – A process that is too simple or too small – for example, it has fewer than five or six steps, or takes less than 10 minutes to complete manually – is generally not large or complex enough to warrant the investment. There are exceptions to this rule, but ideally, the benefits of automation should scale up with the size and complexity of the process.

Too infrequent – A process that is performed infrequently, such as once or twice a year, is not a great candidate for automation. Because the environment often changes between each run, the automated process is no longer a faithful representation of what needs to be done. Even if automation reduces a three-day process into three hours, it will only save five working days over one year – a minimal return for the business.

Too changeable – Some business processes change significantly each time they are done; for example, invoicing scenarios where there are highly specialised formats and structures applied on a per-client basis. In these cases, it is difficult to get the repeatability benefits of automation.

Too qualitative – Processes that largely require human or qualitative analysis and action (for example, writing commentaries) are not suited to automation.

Our recommendations 

• As early as possible, define a clear set of benefits that are expected from automating the target process. By being explicit about these benefits in the context of a specific process, businesses can identify early potentially disappointments – triggering either a review of the expected benefits and expectations or a review of the selected process.

3 - Not understanding how the various organisational processes fit together

Very few processes exist in isolation. Business processes usually form a network within an organisation that connects information sources, people, teams, regulators, business partners and other organisational stakeholders.

It is difficult for any one person to fully understand all the connection points for a business process.

A consequence of this hidden complexity is that when changes are made in one process, another process in the organisation breaks. This might be because no-one realised that team XYZ used the information in column Z that someone removed from one of the output files (thinking it wasn’t being used). It might be because one business partner’s IT system uses an older technology that doesn’t support extended character sets. Less direct, but equally important, can be compliance with regulatory requirements. A process can accidentally slip into non-compliance because it does not keep a specified level of detail available for future audit, simply because no-one knew there was a dependency between the risk and compliance team.

The ultimate result of not fully understanding all the dependencies and linkages of a process is an automation project that does not deliver value for the business and its customers.

Our recommendations

• Before automating a process, draw up a ‘process map’ that identifies other processes surrounding the target process, including where the inputs come from and outputs go. This will help identify project dependencies.
• Explicitly identify the dependencies of a process (early) in the automation project plan, including linkages between the target process and other processes, people and teams.

4 - Subject matter experts not available to capture and codify their expertise

Most of the work needed to automate a process is at the lowest level of detail – understanding exactly what elements of data do, how they fit and how they need to be changed.

This is where the subject matter expert (SME) in the business process becomes invaluable. While there are no hard and fast rules governing the amount of project effort delivered by differing roles, you should expect the attention and availability of SME's to describe and confirm their understanding of the process.

In most businesses, the SME typically comes from the business side and would be able to perform the process manually if they had to. Look for your resident SME by starting with individuals titled ‘business analyst’, ‘business manager’, ‘operations manager’ and the like. In industries such as insurance, some professional roles (‘actuary’, ‘quantitative analyst’) are also SMEs. Conversely, SMEs are not often found among the managers, testing managers or IT team.

Without sufficient involvement from the SME, businesses can end up with automated processes that do not produce the required results. Further, the project team often does not realise that the process is incorrect until its erroneous outputs are discovered by another team. Part of the value an SME brings to the project is being able to verify if outputs from a process are correct or not.

Our recommendations 

• When formulating a project plan for an automation project, make sure you include SMEs.
• Measure how much of the total project effort is being delivered by SMEs. It if it less than half, something is wrong and you will want to revise the plan. 
• Aim for dedicated involvement from SMEs – avoid part-time and as-needed arrangements. All SMEs will have a day job with business as usual deliverables. It’s critical that the senior stakeholders and sponsors understand and support the investment in time from the SME.

5 - The project runs for too long due to a start-stop-start again resourcing mode

Because process automation projects are often run while the existing process is still running, they can lack a sense of urgency.

After all, even if the new automated process is delivered late, the business can continue operating normally for months or years. Often, any deadlines that exist are management decisions and relatively arbitrary in nature. This can generate the impression that the process automation project is secondary to other projects or everyday business events.

The process automation project can then become periodically starved of focus as resources are diverted to other activities deemed more urgent. Typically, a key person on the automation project will have to put the project on hold to participate in some other business activity for ‘just one or two weeks’. Apart from losing momentum, this interruption introduces blockages and stall points as other project members find they cannot progress until the key person is ready again.

Very quickly, the project team becomes fragmented and unproductive. Just as quickly, the project itself acquires an unpleasant image within the organisation – it becomes the project that drags on, runs past deadlines, and feels like a failure in the making.

Believe it or not, this happens in nearly every organisation where resources can be ‘borrowed’ from one project to another.

Our recommendations

• Plan to run the project at full momentum from start to finish, and recognise that losing momentum can be a project killer.
• Avoid using SMEs on a when-needed basis. Formalise their time commitments. 
• Avoid arrangements where staff are borrowed temporarily during a project. Negotiate staff availability for the full project duration. 
• Enlist the support of executives to turn down requests to borrow resources. This support can help make up for the lack of natural date-criticality

6 - Insufficiently active senior executive mandate

If you are a senior manager or executive, prepare to cringe when reading this section.

Executives and senior managers love projects that promise to boost productivity, increase efficiency, lift transparency, reduce risk and so on. They will generally make themselves available and visible at the start of a project, to spearhead the initiative. Over time, however, their involvement tends to wane. They stop turning up to project meetings and they communicate less frequently about the project, even to the broader team or organisation.

This behaviour creates two problems. Firstly, it makes it difficult for the project team to secure support during any difficult stages. Teams can only fend off requests to borrow project team members when they have the backing of senior management. Secondly, it sends a strong but silent signal to the organisation that the project doesn’t really matter. It becomes difficult to sustain enthusiasm and work hard on a project when even senior management has lost interest.

A prime cause of this situation is the overly full nature of most managers’ agendas. When new issues and problems demand executive and senior management attention, they naturally spend less time on existing projects.

Our recommendations 

• Make sure the project has an executive sponsor. 
• Discuss the expectations of their role for the life of the project before it starts. 
• Arrange for the executive sponsor to build personnel connections early in the project implementation stage

7 - Lack of clarity around how success will be measured

Automation projects serve many purposes, but unless the project team has a clear idea of the benefits they will deliver, problems will arise as they make poor trade-off and prioritisation decisions throughout the life of the project

For example, one common goal of automation projects is to improve the transparency and available information for a process, often to improve regulatory compliance or reduce key person risk. This implies that the project team should focus on adding rich supplementary notes to each step in the automated process, and this priority should be reflected in the project plan.

In reality, however, the time required to add these supplementary notes is not factored into the project plan because the project goal was never quite clear to all team members. Worse, during tests to verify that the planned tasks have been completed adequately, no-one reviews the quality and richness of the supplementary notes. The result is an automated process not maximised to meet the project goals. People outside the project team, including the project sponsor, may be disappointed in the end result because it falls short of their expectations.

Our recommendations

• Make sure the project has prioritised goals in place and that everyone knows what they are. 
• Clarify how and when these goals will be measured, and how the results will be reported. Communicate these goals and measures regularly, both within the project team and externally. Make sure the project plan encompasses tasks specifically design to measure its progress periodically against these goals.

8 - Insufficient skills and experience in the project team around structured and procedural thinking techniques

Being able to look a complex situation and break it down into a series of well-structured steps is a distinct skill. Oddly, many projects assume everyone has this skill or can easily acquire it. Sometimes the project is lucky and does indeed have one or more people with this skill.

In a process automation project, structured thinking skills are crucial. People with these skills are needed to make sense of an existing manual process and design an automated solution, after first recognising what processes constitutes steps that can be automated. Furthermore, as steps in a process are wired together to create increasingly complex processes, individuals with these skills help the project team competently and efficiently modify and develop a solution.

In our experience, many organisations fail to recognise the necessity of these skills for a process automation project, or the fact that not everyone has these skills

Our recommendations 

• Make sure the project has prioritised goals in place and that everyone knows what they are. 
• Clarify how and when these goals will be measured, and how the results will be reported. 
• Communicate these goals and measures regularly, both within the project team and externally. 
• Make sure the project plan encompasses tasks specifically designed to measure its progress periodically against these goals.

9 - Tackling a large or difficult process first

Often, a process automation project takes place in the context of a broader program of transformation.

The project team will often have to automate several processes, several of them interlinked. The desired end result is often to replace a suite of manual processes with well-crafted automated process intended to boost departmental productivity.

Enthusiasm, as well as some risk management methodologies, can lead to the larger or more difficult processes being automated first. The idea is to break the back of the work as quickly as possible or to use scarce project resources (such as subject matter experts) early in the project when support and enthusiasm are high. We think this is a mistake.

Without exception, we observe that the quality of work delivered by a project team increases noticeably through the first two to four processes they automate. The team members establish a shared language and working patterns between them that improve quality, efficiency and the correctness of the automated process.

Our recommendations 

• When planning to automate several processes, start with two or three smaller ones first to bring the team up the learning curve together. 
• Consider putting the team through a training exercise together, maybe to automate a hypothetical process. The team can then learn and make mistakes that will not be carried over into the actual business processes. This is a two or three-day investment in time up-front, but the project will usually recoup the time through greater team efficiency and quality later down the track.

10 - Not taking the time to fully understand the manual process or requirements

The outer layer contains activity-level information, such as ‘the process is run monthly and produces a sales report’. Subsequent layers will reveal an understanding of what is done (for example, ‘populate an excel report‘) and finally how it is done (the detailed or specific steps that take place). It is at the ‘how’ level that organisations start to discover things that even the SME was not fully aware of.

It is also at the ‘how’ level that the project team can, in their enthusiasm, be tempted to feel they have enough information to start developing an automated solution and that any unknowns can be dealt with later in the project. While these unknowns will surface, usually late in the project, by this stage honeymoon period is over and the enthusiasm of the team has waned, with team members busy trying to meet other deadlines within time and budget

Our recommendations

• When writing the scope documentation, ensure several revisions. At each revision, have the project team scan through the document and list all questions that need to be answered by the SME. Repeat these revisions until nobody can think of any more questions.

About SolveXia

SolveXia automates reporting, reconciliations, and workflow. We help CFOs innovate, reduce costs and increase the accuracy and timeliness of their reporting. Companies across APAC, UK, EMEA and USA have been using our cloud-based software to automate their processes since 2008.
SolveXia works by providing functionality to automate data processing and reporting across four key categories

• Data wrangling: Collect, combine, cleanse and validate data
• Calculations: Including support for formulae, aggregations and advanced analytics
• Reporting: Data storage, business intelligence and visualisation
• Governance: Process orchestration, version control, admin and audit

Our unique, cloud-platform, allows companies to start automating in minutes, with unlimited scale – whether you automate 1 or 1,000 processes.