Workload Automation: The Heart of Enterprise Operations
Originally, job scheduling software automated the tedious batch submission of IT workloads. While most systems have some rudimentary ability to run tasks locally, those capabilities fall short in terms of ease of administration and the ability to define jobs across distributed systems and servers. As the number and variety of platforms has spread, distributed job scheduling systems emerged to simplify workload management across the enterprise.
These tools, however, have advanced far beyond their origins. Modern enterprise workload automation and service orchestration software provides a reliable backbone to enterprise operations, ensuring that complex business and IT processes are executed on time in the correct sequence. They orchestrate data extracts, transfers, and loads between systems, reconciliations, distribution of reports to end users, and many other business-visible tasks. At the infrastructure level, they control backups, logging, database maintenance, and other core infrastructure tasks. Choosing a strong workload automation software thus remains essential to IT production operations for any company with complex system interactions supporting end-to-end business processes, or significant numbers of servers and other devices under management.
These tools can be very expensive when acquired from traditional vendors of IT management software, and more cost-competitive offerings are becoming available.
Figure 1 shows a typical enterprise workload automation use case. Retail point-of-sale systems feed raw sales figures to a Sales Audit system, perhaps on a nightly basis. That system processes the sales figures in batches, feeding that data in turn to the GL and perhaps the pricing system, often by FTP or an expensive single purpose file transfer system. Executive reporting may be run by automation and the previous day’s audited sales figures are delivered to executives’ inboxes in the morning. In the meantime, updated retail pricing is downloaded to the point-of-sale systems for the next day’s operations. All of these feeds and reporting are complex and mission critical for the retailer. The workload automation capability at the center directly supports the enterprise business model and ongoing management.
Enterprise Scheduler Requirements
In a distributed environment, the job scheduler essentially acts as an administrator, remotely logging into a system, executing commands, monitoring their progress and outcome, and taking further action based on those results. Enterprise job schedulers excel at repeatable processes like extracting data, auditing, running reports, transferring files, and so forth.
Traditionally, these job sequences have been tied to calendar events (for example, “every night at 2 AM”), but modern job scheduling software can also react to events such as SNMP traps, file arrival/modification alerts, or application-specific triggers to start processing. The job scheduler itself must be robustly integrated with monitoring, and able to send alerts to operations. In impersonating a user or administrator, the scheduler must often maintain complex sets of environmental variables. Agents can easily number in the hundreds or thousands, and therefore require management in terms of configuration and security.
It is therefore not simple to build a reliable workload automation tool. The agents (software processes which proxy for the administrator in running commands) must be robust, secure, and compatible with a wide variety of operating systems. The central hub that controls the agents is a potential failure point for a critical enterprise process and needs to be designed for high availability and continuity. Communication between hub and agents must be reliable and secure.
Workload automation software is a shared resource and ideally supports delegation of ownership and administration. Since job definitions are critical production configurations, a full audit trail of all new and changed jobs is advisable and often required by auditors and regulatory authorities. And since a job scheduler may support Service Level Agreements (SLAs), some degree of integration with IT service management practices is advisable, such as configuration, incident, and change management.
JAMS Approach to Enterprise Workload Automation
We view production scheduling within a broader topic of WLA. Workload automation is an evolution of production scheduling that automates complex IT processing and includes support for event-driven workloads, multiple platforms, Web services, composite applications, SOA, virtual systems, dynamic resource allocation, ITSM integration, and business service alignment.
Production scheduling is perhaps the most important foundational element in all of this. The ability to control an automated workflow across multiple IT resources is essential. While more advanced applications may be built on this foundation, its core services remain a requirement.
JAMS’s parent company, Fortra, implemented its first scheduling tool in 1982 for the IBM i platform. Their years of experience in supporting the IBM i platform (legendary for its stability) clearly show through in their entry into the distributed systems market, which boasts a clean, usable interface, solid coverage of essential scheduling requirements, as well as many advanced features. Inclusion of an integrated FTP engine is significant alone, as this can replace costly dedicated file transfer infrastructure, such as Connect: Direct. The ability to delegate administration on a job- by-job basis ensures that the scheduler can support a federated management model, and the ability to control poorly behaved Windows applications for “lights-out” processing is also a differentiator.
JAMS sees traditional production scheduling as something that will increasingly overlap with the runbook automation and BPM/Web Services Choreography spaces. For example, JAMS’s ability to monitor and respond to changes in system processes starts to move it into the runbook automation space. As workloads become more dynamic, we think that production schedulers will need to interoperate more with management infrastructure to route workloads for optimal usage of resources (for example, if a workload is estimated to require a certain amount of capacity, perhaps a new virtual server should be spawned.)
Take the next step in learning about workload automation
Our WLA Buyer’s Guide is a great read for those scoping out an enterprise WLA project and determining what they need in a WLA tool.
Get the Guide