An automated lighting programmer is often called upon to help determine the best backup plan for Front of House. Technologically speaking, there are many types of backup solutions, some of which are very simple while others are extremely complex. A backup plan can range from nothing to full redundancy. Usually the importance and the budget of the event will be a key factor in determining how much backup protection is available. Of course every system has weaknesses, but we should always strive for no single point of failure.
In our industry there are many different types of problems that can occur and cause the failure of a lighting console. During the course of my career, I have seen failures due to power loss, console hardware failure, console software failure, unplugged cables, loss of communication, liquid spills, malicious attacks and even acts of nature. As I learned in Boy Scouts, we must always be prepared, and this applies to planning for the unexpected at Front of House.
When you’re specifying a lighting control system for a production, you should always consider backup gear. Will the event require a spare console? What about redundant DMX512 distribution? How complex should the backup be? The type of production will help with some of these choices. For example, if you are programming the dance floor for the Bitterman’s wedding reception you probably do not need to be too concerned with backup equipment. However if you are programming the Opening Ceremonies for the Olympics that is aired live to over four billion people worldwide, then a solid backup plan is essential. For a show of this caliber, you should have two completely redundant systems. That means two consoles, two sets of DMX512 processors (if applicable), a DMX512 switching device, and a method for the consoles to track each other.
With two completely separate systems, each with their own power sources and UPS, you should be able to switch to the backup with only a minor interruption on stage. However every system has an Achilles’ heel. You need to really think through things such as separate network cables and switches, independent DMX512 runs, individual MIDI and timecode inputs. It may be impossible to create a system with no single point of failure. But by trying to achieve this you will create the most secure backup system possible.
Most productions fall somewhere between the wedding reception and the Olympics. This means that your common backup plan may be as simple as having a spare console on site. It’s fairly common to have an extra console loaded with the same show file. This way if something goes wrong with the main console, the backup could be set up and running in a few minutes. Lighting designer John Broderick once taught me that it is important on the first day to fully test the spare console. He stressed that I must turn it on, save a show on the main console, and load this show on the backup console. Then I need to connect the spare to the rig and confirm DMX512 output. This way I have established that the spare console is in working order early enough to order another if necessary. As always, his wisdom was proven correct once when I had a spare desk that could not load in the show file from the master. I was able to get another spare desk right away.
When planning to swap from one desk to another, you must have a plan. You must decide if you will have time to unpack a desk, boot it and connect the DMX512 outputs or if you need the backup to be instantly ready. There are many DMX512 devices on the market that will allow you to connect the output from two consoles to the rig. Then you can simply toggle a switch to change which console controls the show. Some of these devices can also automatically switch the DMX512 input if they detect data loss on one of the lines.
Most automated lighting console manufacturers include a method of tracking a backup console. These features typically use MIDI, MIDI Show Control, or networking to ensure that the spare console is always in the same cue state as the main desk when it’s configured correctly. This way if some audience member throws his beer cup on your main console, you can instantly switch to the backup console without a hiccup or glitch on stage. The backup console tracks along with the main console so it is always ready to take over. Typically this type of tracking is used for playback only and does not track programming actions — more on that later.
As a console operator, you need to understand how the tracking backup needs to be set up and how to make the switch happen in an emergency. I remember working with one well-known operator who seemed to get very flustered when something went wrong. In his panic, he would forget how to properly switch to the backup system. This left the band on stage in the dark unnecessarily for a long time. The system was set up for a smooth transition, but because he was not prepared, the operator slowed down the plan and the show suffered. Just like a fire drill, you should practice with your backup plan to ensure that it is working and that everyone understands the routine.
A few consoles offer tracking backup functionality for programming sessions. This is also accomplished with either MIDI or networking. The promise is that as you program your show on one desk the backup desk will mirror all your actions, creating an exact copy of your work. While this is a handy feature, it should never take the place of a regular routine of saving to external media. Usually what happens is that a bug in the software will crash both the main console and the backup console because they are both running the same software and receiving the same keystrokes. At this point you have two dead consoles and a show file that might be corrupted. I prefer to make regular backups to external media. That way, playback tracking isn’t my only backup solution.
Remember, there are no perfect answers for a console backup solution. The best idea is to carefully plan and determine the best backup scheme for your production. No matter how complex your system is, consider that there is probably at least one point of failure. Try to identify all the possibilities and plan accordingly. Above all, you should always make regular backups of your show files onto external media, as this is the best protection of the show data. A good console backup strategy will ensure that the production on stage never suffers due to the unexpected at front of house.
