Wednesday, April 21, 2010

Lessons learned on BCP

"No, we said we wanted CASH, not ASH!". And that's just one of the many jokes now circulating the Internet and media everywhere about Iceland. #ashcloud already exists on Twitter, and today I got my lesson as well. Well deserved probably. Let me explain.

I've been scheduled to speak at the "National security day" here in Norway today (April 20th) for quite some time. A yearly event, it's a one-day conference mainly on information security awareness. My employer is one of the sponsors, and I've spoken there earlier as well.

I ordered my plane tickets a long time ago, to get them as cheap as possible for my employer. Flying from Bergen to Oslo is 40 minutes, but it'll take you some 7 hours to drive, or by train. Although the train is a beautiful journey you should enjoy sometime, you'll probably do the airplane for events like this. Then Iceland^H^H^H^H^H^H^H a volcano decided to attack with massive amounts of ashes whirling into the higher atmosphere. With winds moving it towards Norway, flights got cancelled due to risk of ashes entering the turbines and turning the engines into miniature volcanos. Seriously dangerous stuff.

So in the evening of friday 16th, I decided to initiate a Business Continuity Plan. The show must go on, as artists tend to say. :-) The website of our national railroad company, NSB, was down. No surprise, the media was already telling us to forget about summer, flights to the mediterranean in june and the end of the world as we know it. (or pretty close, as far as I would judge some of the headlines used)

I continued of course, probably joining the "REFRESH EVERY 5 SECONDS" DDoS attack against NSB by desperate passengers seeking alternative ways of transportation in every direction. At 01:00 saturday I got through, and reserved 2-way tickets with a bed to sleep in, leaving Bergen at monday evening 22:58, arriving Oslo central station at 06:30 something tuesday morning. Plenty of time before my presentation at 11:00.

Skip forward. Monday evening 22:15. Media, airlines, Norwegian authorities (Avinor), everyone says that flights will be pretty close to normal on tuesday morning. They're on a 6-hour schedule on evaluating the #ashcloud situation, but says new update will come at 09:00. Until then; normal traffic. So I turned down a bed to sleep in withNSB across Norway. I dropped my backup plan (" I'm cancelling my ticket. Bye!"), in favor of my own bed at home, and a quick 40 minute flight in the morning. *STUPID*

At 03:44 I get an SMS message on my phone (just in case something happened, I had it beside my bed). 03:44. Not my favorite time for getting a wake-up call. "Your flight has been cancelled at 06:10. Have a nice day" (or something close to that). Thanks. Perhaps 30 seconds later I'm online. (my computers@home do password cracking 24/7). Avinor says my flight is cancelled, but others are en route. Just my luck. Next flight is at 06:45, different airline. Darn #1.

Credit card, shelling out NOK 1800,- for a one-way trip. Someone's gonna whack me at work. Dont'care at 03:49, I'm doing this. Received confirmation by SMS, "have a nice flight". Taxi pickup at 05:10. Airport at 05:21 (short trip, I'm always early. Lessons learned from mandatory military service for 1 year, military police in the army).

Dazed & confused airline employees waving at us, saying that air traffic is closed until at least 0800. WTF? Information boards says flights are en route. DARN #2.

Local cute female radio/television reporter, maybe a little dazed and confused as well? "Hey, how you doin'?" WINK-WINK. "See, here's my story...". "You wanna say that on radio?". SURE! (Wink).

3 short interviews. Local radio, national radio, national news, saying:
"Well, I was supposed to speak at the national security day on how the Norwegian government doesn't do good on e-mail encryption, instead I'm on radio criticizing their lack of information to airline passengers instead".

How hard can it possibly be to inform passengers every 30 minutes on speakers that you do or that you don't have any new information to provide? The airport manager is interviewed as well (of course). Nice chap, polite and with good replies to all questions, although no estimates or guarantees. "We're waiting for information to make a decision."

08:10 Go to gate. It's been some time since I last ran like that with shoes, shirt and a tie. Got to admit security controls seemed a bit relaxed, but I guess guys with ties can't be bad guys? :-) Oh, and the airport manager said to me "Good luck with your presentation! We did this for you, we're probably closing the shop at 11:00. Have a nice flight!". Couldn't do much more than say "Thanks!", although I didn't really believe him on that one. Nice try though. :-)

10:40: I'm there.
11:00 "Hi, my name is..."
11:01: Report is published online (English version coming)
Good responses from the audience, thanks!

Lessons learned:
1. You should have a backup plan (Business Continuity Plan, could also be known as "the show must go on plan")

2. You should have a backup plan (repeated especially for you)

3. If your primary plan seems as the easiest plan, but also the least probable plan, your backup plan is probably your best choice. Stick to it.

4. If you're the provider, you should have a crisis information plan. Test it. Again. Again. Again. (Repeat indefinately). Stick to it, and for !"#¤'s sake, EXECUTE IT WHEN NECESSARY (Media won't treat you good if you don't)

5. Be nice. It's not their fault (at the moment blaming it all on Iceland is.. well... fun.) Yelling rarely helps you out.

6. Exception from #5: poking at little fun at government agencies should be allowed

I'm no better than anyone else, trying to pick the easiest way. We're made like that. Adding a bit of common sense as well as simple risk evaluations would probably be a good idea from time to time.

--
PS: this blog post was written onboard a train going from Oslo to Gothenburg in Sweden. Yes, I've got return tickets. All the way back to Bergen, that's close to 16 hours on a train, all for a 45 minute speech. I'll be speaking today at the Scandinavian ISACA conference on "top level security". Members of the board, I'm auditing you.

No comments:

Post a Comment

All comments will be moderated, primarily for spam. You are welcome to disagree with my posts of course.