Automated B2B Order Processing Blog | Lucy

Automation is a Journey – Step 4: Evaluating results

Written by Ben Dalton | 19 Jul '22

Well it’s hard to believe, but we’re already past the halfway point in our Automation Journey. We hope you’re finding the series enjoyable, and we’re so glad you’re along for the next step.

At the start of the series, we outlined the overall process of embarking on automation. Then we discussed each of the steps in more depth:

  1. Step one: identifying the process to automate
  2. Step two: researching vendor options
  3. Step three: calculating ROI

If you’ve worked through the first three steps and implemented your chosen solution, it’s time to move on.

Step four: Evaluating early results 

Okay so you’re not an automation expert – not yet anyway.

Therefore, it’s really important to work closely with your vendor through this early, post go-live period. Your vendor should have set clear expectations around what success looks like in the first months. Those expectations should align with your business goals, which will help inform the results you choose to measure.

When interrogating early results, start by examining the following:

I. What do the numbers look like one month post-implementation?

Your vendor should be tracking all the statistics on those previously agreed metrics. After about 30 days of automation, it’s time to check in on yours.

To give you an example of common metrics, with Lucy our initial review would include the number of purchase orders Lucy processed during the month. In addition to total orders, how many order lines were successfully integrated? How many staff interventions were required due to mismatched data? How many trading partners did Lucy look after?

There are always two sets of expectations at play here: 1) realistic performance in the first month, and 2) automation nirvana.

If you can view early results through a realistic lens, it will help identify ways to reach nirvana later.

II. Has uptake been as expected?  

Here it can get interesting. Hopefully your staff were all on board with this automation project at the outset. Now that it’s launched, how has it impacted them? Metrics and statistics may not paint the full picture, so it’s important to talk to those staff who’ve been impacted - do a debrief and get the whole story.

By our very nature, we humans can be resistant to change. Customer service reps or other staff who see automation as a threat to job security are unlikely to embrace it. Engaging and supporting affected staff early in the project can head off possible disengagement (or even subconscious sabotage attempts) later.

The project champion should lead the charge on staff engagement, backed by plenty of support from the vendor. Now is a good time to revisit staff training to ensure the automation software is well-understood. If there are new staff or you’ve identified knowledge gaps, get your vendor to provide more training. Listen to staff feedback – there may be a pain point they’re experiencing that the vendor can address.

Open communication is key. 

III. What is the feedback from internal & external stakeholders?  

This links back to the previous question around uptake and staff engagement, but you should also be listening to other voices around the business.

How are warehouse staff experiencing the change (if any)? With sales order automation, Lucy customers have found that demand on their warehouses smooths out, with less dramatic peaks and troughs to navigate.

Perhaps customer orders are being packed and dispatched more quickly? Are the finance team processing fewer credits due to a reduction in errors?

Gather as much feedback as possible. If it’s quantifiable, all the better.

IV. What do the numbers look like 2, 4, 6 months in?  

At the risk of overstating it, we urge you to revisit performance continually with your vendor. Automation is a journey and your vendor should be a partner who travels with you long after implementation.

As time goes on, look for ways to analyse the feedback you’ve been collecting since go live. Gauge customer satisfaction by comparing pre and post-automation CSAT scores. Check whether the business’s DIFOT (delivered in full, on time) figures have changed since automation began?

Watch for usage trends within your automation stats. Regular monitoring can help identify areas to scale. Perhaps additional trading partners can be onboarded now that staff are confident using the software.

Leverage in-built smarts if your automation software offers them. For example, Lucy can be taught to remember data corrections and order patterns, which means she gets smarter over time. This is reflected in fewer interventions required as Lucy takes on more of the purchase order processing.

In summary 

The aim of this step is to gain a very clear picture of what is happening within your business as a result of this automation.

During the initial weeks and months, your vendor should be close at hand, providing guidance, data, and support. If your early results aren’t aligned to expectations, how can your vendor help get you there?

Next time we'll look at Step 5: Acting on insights

Until then..... we’re always happy to chat automation. If you are too, please get in touch!