This is a repost of an article from my old blog about CodaServer. It’s being published here for posterity.
Benjamin Franklin once said “Lost time is never found again.” That is doubly true if that time is when your company’s batch processing was supposed to run, and now none of your suppliers are going to get their checks. And they cancel your accounts, and you can’t get more supplies. Your customers rebel and go to your competitors, your dog leaves you to find a master who can remember not to screw up the mundane details.
That time is the most precious of all.
In programming, we constantly deal with events. These can be user-generated (mouse clicks, form submissions) or machine-generated (a web service call) and we can easily build systems that capture the events and respond to them. Less easily captured, however, are events generated by time.
How important are these events for a data model? Pretty darned important. There are many tasks which have a heavy data component that simply cannot be run on real-time production systems; rebuilding search indexes over millions of rows data, for instance, or moving millions of transactions from the denormalized transaction schema into the general ledger. There are also operations that are done at a particular time of day as part of a business rule. Electronic voting systems need to stop accepting votes at a particular time, and student registration systems need to open their course catalogs to certain students at certain times.
It’s a big deal.
The Old Way
Traditional, time-based events are the domain of the venerable cron daemon. This little guy reads cron tab files which specify times for which certain scripts are to be run. Scripts can be set to run every month, on the 10th, at 3:34AM, or every Wednesday for each minute of the day. It’s a remarkably flexible system with a few drawbacks.
- The scripts are not necessarily part of the application’s main codebase. If you have a PHP web app based on CakePHP, it’s difficult to reuse that system’s configuration to drive your cron script, which needs to run via the command line. It also makes class libraries more difficult to share.
- The cron daemon can die, or the server it is on can die, or the script can die or time out. All of these could leave the database in a inconsistent state and can be difficult to troubleshoot after the fact.
- Cron is generally handled as an outlier to the rest of the system. It is difficult to version control, often has an owner who is not a developer (usually a sysadmin), and requires special consideration during code pushes.
The CodaServer Way
CodaServer’s solution is to bring cron inside the database.
Instead of scripts, CodaServer’s CRON runs stored procedures which have been previously defined by developers. The syntax should be familiar.
CRON check_cron ON accounting.dev IS 0 0 1 * * process_checks;
This will create a new cron called “check_cron” in the development instance of the accounting application that runs the “process_checks” procedure on midnight on the first of each month. By placing cron inside the business rules engine, CodaServer addresses each of the problems above.
- The cron isn’t separate from the rest of the code. In fact it doesn’t require anything in application code at all. They become part of the application’s business rules.
- The cron is guaranteed to run as long as the business rules engine is up. There are fewer moving parts.
- The crons can be created and maintained by developers with the MANAGE_CRONS permission on the server.
Crons are unique amongst CodaServer objects in that they are not part of the application’s transaction log. They need to be created explicitly for each instance under which they should run. This was meant to ease system administration and prevent the situation in which unintended events are pushed from environment to environment during code promotion with no way to turn them off. The procedures they call are part of the transaction log, however, so these will be moved between environments automatically.
Cron has been a great friend for many years, and I’m really happy we can take him along and give him a new home.
Til next time.
