Worthwhile automation

In many situations, software exists to automate what would otherwise be done manually in business. For example, calculating interest on bank accounts. It also enables us to do things we’d never be able to do by hand. By investing in automating the mundane, we’ve created tools that open up new opportunities. Things not possible, nor fathomable before.

Between manually doing things and moving toward automation, there seem to be a set of steps we follow. We start out doing things manually, relying on memory to recall what’s necessary.

The problem is, memory fades. Even our future self may not remember all of what’s necessary to carry out a particular task. Let alone a new employee or someone else that wants to help pick up some of our work load. When memory fades, we’re left with a game of trial and error to reproduce what we need to do.

And eventually we get bitten by trial and error. If it were the old days of calculating interest by hand, perhaps we made a mistake and credited or debited an account incorrectly. At some point we come to the conclusion that we should write some of this down.

We go about creating written documentation of what we do, a guide to follow. Unfortunately, that guide is all to often tucked away and forgotten. We may not even recall we have it when the time arises to take advantage of it!

We don’t have any incentive to update it either. Nobody enjoys updating instructions. As we change how we work and find new ways to do things, the instructions grow out of touch with reality.

Even if we get things written down, the instructions are often too detailed or not detailed enough. Perhaps the person that authored them was unfamiliar with the process and wanted lots of detail. Theoretically, in their mind, anyone could pick up the instructions and know what to do. Unfortunately, many will find themselves skimming and missing the important parts.

Or, if someone wrote the instructions that was very familiar with the process, they may take for granted what someone else knows. There are things they assume anyone would know, just because it’s not something they have to think about routinely. They won’t be aware they’re leaving out critical information. At best, the instructions serve as an outline.

Because of these challenges, it’s difficult to make written documentation worthwhile.

The next step is to automate the process, if it proves worthwhile. Some things can’t be automated. For those, a written list is as good as it gets. If a process is important enough, the instructions will be updated, or individuals will learn to simplify the process to make it memorable.

For the processes that can be automated, the next step is to start automating portions of the process. Perhaps, using excel for interest calculations. And then later, a way to update balances. And then produce a report for someone responsible for sending a monthly statement to customers.

In time, more and more of the process is automated.

When creating software, it’s valuable to keep track of the history of these automated instructions. In software, the instructions are the code. Version control systems track not only what we do, but why. And, when we started doing it that way, and who set it up. It gives us a picture of the history of decisions that led to the current process.

This accountability gives us the ability to safely evolve a process to take advantage of things we wouldn’t have dreamed possible. To reinvent what we do.

What was once manually applied, is now automatically applied. And it typically follows the described progression:

Automation Trend

I refer to the automation as scripts. Scripts are the portions of the process that get automated first. Scripts are little tools that carry out a part of the process.

Of course, it’s important to consider the value and cost of progressing toward fully automated application. In many situations it makes no sense to automate a process. For example, things that require human judgement.

At some point, the value has to surpass the cost to make the investment worthwhile. Otherwise, why not just continue to do things manually and rely on memory?

Every situation is unique, but I’ve noticed a trend in terms of cost versus value.

Automation ROI

Upfront, cost tends to take off before value. Writing things down consumes time, and often provides little value. Writing helper scripts requires even more time, but starts to add a little value. Assuming people remember to use the scripts. As more of a process is automated, the automation is often put under some means of version control. This provides a history of decisions.

Value starts paying dividends at this point. Often surpassing cost. As a process becomes fully automated, new opportunities open up. New ways of doing things. But even at this point, when value is rising quickly, it will eventually level off. Further investment will cost more than the marginal return it provides. At that point it makes sense to consider what we have automated as good enough.

What I’ve found fascinating, is to apply this same understanding of automation to the various aspects of software development. In terms of automating how we automate.

There are many processes in software development that are still manually applied in a majority of situations. For example, setting up development computers to create software, setting up servers to host software, creating and changing databases that store information, compiling and packaging custom software, installing custom software, and releasing custom software.

Sometimes there are written instructions to follow. Sometimes there are scripts to help out. Sometimes things are fully automated. If you surveyed the software community, you’d find a different distribution for each of these software development processes.

Unfortunately, many of these development techniques are ripe for paying dividends if automated, but the up front cost to move toward automation keeps many organizations from investing in automation.

Take a minute and write down the processes that you rely upon. Make a list of the important ones. See where on the trend from manual to automated you fall. Sketch out what you think the cost and value curves might look like.

Is the point where the investment breaks even not as far out as you might have imagined?