Why Batch Processing Will Never Die

placing a gear

An attendee at one of our recent webinars made this comment, “Batch is dead. Why would I need a workload automation tool?”

Yes, the definition of batch has changed over the years. But, background processing will never die.

It’s just not visible to the end user. For example, when you place an order on a website that order triggers a stream of automated background processing.

First, the vendor’s system has to process the web order, notify the sales representative, send the order to the warehouse, produce a pick list for the distribution center, produce mailing labels for the order, and route it to the correct truck for delivery all within a matter of minutes or hours.

In the 1980s those same processes had to occur, but the triggers were manual. Someone called the order in to a call center, someone wrote the information on an order form, and the form was delivered to the data processing office where cards were punched with the correct information.

Cards were fed into the mainframe computer, which produced the pick list during the batch run that evening. The lists were distributed and used to pull the correct orders. More cards were punched for mailing labels and delivery instructions. Once the cards were fed into the mainframe again the next evening, another batch would run to produce the needed instructions.

Each step in the process would take a round of card punching and another batch run for processing.

The 1990s brought order entry systems into the mix so that punch cards were no longer necessary, but data entry clerks still entered the orders and started the same job stream with multiple batches to get the orders processed.

The 2000s brought more distributed and open systems into main stream production. After Y2K passed without a hitch, internet use as a sales tool exploded and processing methods compressed.

Now those orders are placed, picked, and shipped within minutes. So ‘batch’ isn’t run in ‘batches,’ but events trigger other processes throughout the day and night. Systems are able to handle interactive and background processing concurrently.

Your workload automation tool handles all of those various triggers to make sure that orders are processed correctly and quickly.

What drives your schedule? It may be another process starting or ending, a new file arriving or an existing file changing. Whatever that driver, your automation tool must be able to handle it.

So, what’s the new definition of batch? It may be that outside events are driving the schedule and those events can occur at any time of the day or night. The scheduler needs to be able to monitor for those events and trigger the next step in the process when they occur.