Let’s face it. Scheduling batch jobs with native batch automation tools isn’t easy, or flexible. Whether you’re wrestling with the complex format of cron, or dealing with the limited features in Windows Task Scheduler or SQL Agent, it is never as easy as 1-2-3. Let’s break down some real-world examples of scheduling batch jobs with those native tools. And then, let’s see how easy and flexible those same tasks are in JAMS.
Task Scheduler vs SQL Server vs Cron
Before we dive into these native schedulers individually, let’s first discuss who uses these types of tools and why. These are just three examples of some of the most popular native schedulers. A native scheduler is a job scheduling tool that’s already included natively within a given software. They can schedule jobs and batch processes only for jobs and processes within their platform.
Most users stick with native schedulers for a few reasons, mostly that they’re free and already included with software they’ve purchased. They also might continue using a native scheduler because they don’t know of the alternatives, and there may already be thousands of jobs running in one or more of these tools.
While they may seem convenient, native schedulers are extremely limited in their capabilities. They’re not only lacking enterprise features, like triggers and dependencies, but also the ability to view and orchestrate anything beyond their own software. This leaves your jobs siloed and adds more to your IT team’s already full plate.
Windows Task SchedulerWindows Task Scheduler is fine as long as the schedule you’re applying to a job is fairly “flat”. You can quickly define a single job to run Daily, Weekly or Monthly. In this example, I’ve setup a Job which needs to run every Monday and Friday at 3:00 PM, starting on July 25th, 2016.
These settings work fine for very simple tasks that need to be repeated on predictable schedules. But, if my scheduling requirements grow even slightly more advanced, I have no options.
What if I need to schedule a job to run on only the last workday of the month, so long as that day wasn’t a company recognized holiday? Unfortunately, I’d be in trouble. While Windows Task Scheduler can cater to scheduling around the “last xxx of month” where xxx is a specific day, it lacks any connection to the concept a workday. It also lacks connections to custom calendars, such as bank holidays, and fiscal periods – calendars that are extremely relevant when automating business processes. The limitations of Windows Task Scheduler become even more acute when your requirements include awareness of the failure of dependent jobs, or on their return data.
SQL Server Agent
SQL Server Agent can be considered slightly more robust, because it leverages a concept called “Job Steps”. It also allows you to configure schedules based upon relevant date terms, such as the “last weekday (workday) of a month”. However, when jobs have more advanced requirements – e.g. run times of every other week, on a specific date, or for a set number of days – there are no options. SQL Agent also lacks any ability to run jobs across multiple platforms. This limits you to batch and SQL-based automation. In environments where a large number of applications and platforms need to be integrated to accomplish enterprise workflows, SQL Agent quickly comes up short.
I can link jobs together through SQL Server Agent, which is something that is impossible in both cron and Windows Task Scheduler. However, I have to rely on “pillow-based scheduling” to run them in sequence. What I mean by pillow-based is that I have to add artificial buffers in between steps. For example, if my first job is scheduled for 8:00 AM, and I expect it to take approximately 3 minutes to run, I schedule the second job in the series to run at 8:05 AM. Since the first job may take a little more than 3 minutes, I have to build in a 2-minute buffer to make sure my second job doesn’t start prematurely. This workaround fills a jobs schedule with slack time. As more jobs are added to the schedule, more and more time is left on the table. And, given the competitive nature of most enterprises, time wasted is money wasted!
Cron is arguably the weakest native scheduling tool. Cron also uses a very specialized syntax to define job schedules. Other native schedulers, at least, have fill in the blank forms for creating a basic schedule. But, even a simple daily recurring job in cron requires syntax akin to:
0 0 8 ? * MON-FRI *
Now, what about a job that repeats every hour of the day? Well that expression looks like this:
0 0 0/1 1/1 * ? *
Needless to say, this arcane language doesn’t align well with today’s agile team strategies. Only cron experts can decipher how automation is being applied to a particular business process. Cron syntax is so opaque that a number of 3rd party websites have been created to help users build valid schedules. Validating those schedules and adding buffer time between steps requires exceptionally complex scripting.
Next Level Automation
In any of these tools, what would you do if you needed to…
…wait for a file to become available on the server?
…if you only need a process to run if another job happened to fail?
…parse email inboxes and download attachments?
…run a shell script on Linux based upon values returned by a PowerShell script on Windows?
Sure, you could script, or write custom applications for much of this. But, that just means yet another product to manage in-house, another product that could potentially break as protocols evolve. Moreover, it means time spent and wasted by your development team.
True Enterprise Scheduling
JAMS is an enterprise scheduling software that is scalable and easy to set up. It fits the automation needs of a comprehensive array of business initiatives, from database teams running SSIS packages in our SQL Scheduler, to DevOps teams looking for continuous deployment, and updates to a database post-build, to BI teams looking to run complex data analysis and reporting engines.
Let’s revisit the earlier examples, this time leveraging JAMS, to see how efficiently we can automate similar tasks.
In this example, I’ve schedule a job to run every other Thursday, as well as one day prior to the last workday of the month. I’ve also gone ahead and created a custom CompanyHolidays calendar, which contains all of public and company-observed holidays. This custom calendar can then be referenced in our Schedule Exceptions. This ensures the job does not run the day after a company holiday has been observed.
I also have the need to ensure a specific file is present from our Tableau Server. In this case, I need the specific backup file to also contain today’s date stamp in it. There could be the case that the file is still being written to, so I want to ensure that JAMS has exclusive access to it. All of these dependencies are easily set in our job definition (screenshot below). I’ve referenced a specific built-in variable that resolves to today’s date. The file has the criteria set to be “Available” which means JAMS will check for exclusive write access to the file before releasing the dependency on the job.
For DevOps teams looking to integrate JAMS deeply into existing applications, or hook into APIs, we also have you covered. Through either our .NET or REST API we’ve made it easy to hook JAMS into any application.
JAMS provides the scalable and robust scheduling interface that is lacking in so many of the native tools used today. Sure, native tools fit a need for smaller organizations and sometimes even for a specific project. However, these tools will never truly scale in your environment, and will always cost you more time and money.