slate blog

As a sister educational institution with a long history with Virginia Tech all of us here have been watching the events in Blacksburg, VA closely. Our thoughts and prayers are with them. While I could delve into the failure of email or the promise of text messaging I?m limiting this post to the challenges of crisis communication on the web. The VT web team has said they will be sharing their own lessons learned at some point in the future but here are my points to take away after reflecting on our own crisis communication plan.

Have emergency contact information and procedures updated and ready to be posted during the crisis.

The web will be the primary means for users on campus, residents in our community and concerned parents far away to learn about events on campus. It is a critical component in helping deliver important content and notices. It is also immediate. An institution cannot be caught scrambling, have piecemeal information or incorrect information. I would go a step further and argue that basic or general information should be available 24/7 on a dedicated web site. I’ll be curious to see how many institutions move to that kind of model. Tulane has an interesting example .

Have standard templates for communicating during a crisis.

I believe this goes hand-in-hand with the first lesson. An institution cannot struggle to get a new design up. It takes too long, will be poorly thought out, be at least partially ineffective in the short-term, and will take too much time to tweak during a crisis. An institution should make sure the design and template they use when communicating in a crisis online are clear, usable, and easily available to whoever will be responsible for posting the relevant information before a crisis.

It must be easy to turn on, edit and manage emergency information during a crisis.

The power to ?flip the switch? on an emergency response system cannot be left in the hands of a few. The broader the responsibility is spread the easier it will be to act in a timely manner to a crisis. Not every crisis will happen during work hours. Also, if it?s easy (or at least sort of easy) to manage the content the more likely hopefully the content will get updated with relevant information as the crisis evolves. Obviously there?s a risk that the left-hand might not know what the right-hand is doing but I?d rather risk that then worry when someone goes out of town. The caveat to this is that while it may be easy to turn on and delegated to many people the decision to actually make the change will probably be left to a few.

Reach out to ?non-traditional? media to make sure users know where to find the correct information.

Be ready to find the major MySpace or Facebook groups posting about the crisis and post messages about where users can find the latest information about the crisis. Look to Fark, Digg, and Slashdot as well. There will always be rumors and innuendo that an institution won?t be able to respond to directly but making the effort to proactively reach out to users I feel would go a long way to calming the situation.

Be ready to handle a large number of users (aka load) at any moment.

As much as the content of the message matters to clearly communicate a response to the crisis being able to keep the service available so anyone can access it and benefit from that message is just as important. In the early hours of the crisis Virginia Tech was unable to do so. Their home page was slow or unavailable as users (students, family, the curious) sought to learn about the event, what they should do, and what the response of the university would be.

As the day wore on Virginia Tech tried several strategies to handle the load. All are worth noting:

• At first (~9.58AM) they posted a short message on their regular home page about the event. This is similar to our banner system.
• Then their home page became slow or unavailable as news spread beyond Virginia and users tried to learn about the events by visiting the home page.
• Virginia Tech then attempted to change to a ?light? version of the home page to lessen the load on their servers some time during the morning. Even with the ?light? version the site was still slow though reachable.
• They also attempted to work with the Virginia State Police to post information on the VSP home page.
• They stuck to the ?light? version until the afternoon of April 17 when they switched to a special ?In Memoriam? home page.

See examples of the home page styles used by Virginia Tech during the crisis.

Measures to handle load would include:

• Have at least two ?modes? for a home page to scale to the appropriate response for an event. In our case a snowstorm would only warrant a banner but a chemical spill might warrant more visibility.
• Have ?light? versions of a home page or, as noted earlier, an emergency specific site that is already light. Lessening the weight of a page and number of requests required to render a page will lessen the load on a server.
• Using multiple servers to serve content in a load-balanced cluster. Could use hardware load balancing, software load balancing or DNS round robin.
• Avoiding multimedia-style updates in favor of text-only updates.

Conclusion
To sum up the lessons:

• Have the correct information pre-prepared
• Be ready to react quickly
• Be flexibile
• Look to non-traditional outlets
• Be ready to handle load

Some Other Ideas to Consider

• providing a method to optimize the home page or emergency site for WAP-enabled devices (e.g. cellphones)
• if you are a centralized group of developers look to other groups on campus to provide “reinforcements” when a crisis happens. Get approval for this before a crisis. Not only does your site have to “scale” but the human resource you bring to bear on an event will need to “scale” as well. Especially if your institution is going to be proactive and monitor social networking web sites.
• while an institution may not be able to provide or afford a load-balanced cluster specifically for emergency information I would encourage institutations to see if they can pool together an “emergency network of servers”. basically the idea being to create a distributed, redundant, fault-tolerant set of servers to answer to a single emergency hostname. it would probably be possible using a combination of capistrano, subversion and DNS round-robin.