Morning Coffee

[DOUG]

Sitting at Hartsfield now waiting for the flight to Kansas City. Looks like delta got their stuff back together. At least I'm not going to Milwaukee. If I was I would be checking a over size hard case.

I do need to upgrade my luggage to a more secure hard case so I can fly with one of those special items in my checked bag.


Sent from my iPhone using Tapatalk

They better have their stuff together.  Half my projects are f'd at the moment as Delta has sucked up all the contracting work from my vendors to fix their issues.

Really? Said it was just a fire. I feel like they were hacked. 

Why didn't they switch to mirror servers somewhere else and start running again?

Heck I don't know what to ask, tell what you know

[BigMike]

Sitting at Hartsfield now waiting for the flight to Kansas City. Looks like delta got their stuff back together. At least I'm not going to Milwaukee. If I was I would be checking a over size hard case.

I do need to upgrade my luggage to a more secure hard case so I can fly with one of those special items in my checked bag.


Sent from my iPhone using Tapatalk

They better have their stuff together.  Half my projects are f'd at the moment as Delta has sucked up all the contracting work from my vendors to fix their issues.

Really? Said it was just a fire. I feel like they were hacked. 

Why didn't they switch to mirror servers somewhere else and start running again?

Heck I don't know what to ask, tell what you know

I don't know a whole lot except their 'mirror servers' or Business Continuity / Disaster Recovery Plans were f'd.  You would think a company their size would have redundant geographically dispersed hot active-active data centers but guess not.  Not sure what all was ruined by the fire(and possible resultant suppression systems) versus what they are fixing so it doesn't happen again but all my professional services contract networking resources were called in to help with Delta.

Their CIO started in February.  Hopefully he looked did a Risk Analysis or reviewed their existing Risk Plan (most companies that size have them).  They had to know their exposure and with any luck, the CIO made his bosses aware and the cost to remediate it. 

I heard the problem was bringing up ~400 applications and getting systems talking to each other again.  They'd never tested taking them all offline at once.

[SemperFiArms]

Well everything seems to be back to normal now. Boarded fine and got out on time.

[DOUG]

More about Deltas crisis.
I am appalled at the CEOs comments. Not worthy.

Cloud backup could have prevented Delta’s meltdown

Traditional disaster recovery can be too expensive to implement, but cheaper cloud options render that risk unacceptable

I hope you weren’t flying Delta this week. If you were, you know that Delta’s systems experienced an outage, and of course passengers bore the brunt of it in hundreds of canceled and delayed flights.

When Delta performs a postmortem on this outage, it will likely find that the outage was caused by a common occurrence: network failure. However, Delta could not recover or switch to backup systems. As my InfoWorld colleague Andrew C. Oliver wrote, Delta neglected the four pillars of high availability.

Delta’s CEO admitted as much, telling the Wall Street Journal that “it’s not clear the priorities in our investment have been in the right place. It has caused us to ask a lot of questions which candidly we don’t have a lot of answers for.”

Although cloud computing is not always the answer, the absence of effective business continuity and disaster recovery (BC/DR) is normally due to a lack of funds. Traditional backup sites cost many millions of dollars to establish and many more to operate as a hot standby.

That’s why the use of IaaS clouds as redundant analogs to primary systems is becoming a common practice. More and more, organizations are placing copies of their processing and data in the cloud, which run constantly paired with their primary systems. When the primary goes down, the secondary (cloud) system takes over automatically. That is, they fail over to the cloud.

Unlike traditional BC/DR approaches, public cloud services don’t require hardware and software, they don’t require many operators, and they have their own BC/DR mechanisms in place. They’re not that hard to set up either; I could show Delta how to do it in an afternoon.

I suspect that, at Delta, the lack of effective BC/DR was a money issue, and the company considered it an acceptable risk. But with cloud resources now available for a fraction of the price of traditional backup systems, it’s no longer a risk you can take.

http://cloud-platform.net/cloud-backup-prevented-deltas-meltdown/

[jc79]

Also, I would expect that some of their core apps still run on non x86 platforms. Guessing it's not easy to move those into anyone's standard cloud.


Sent from my iPhone using Tapatalk

[clark123456]

I'm on a flight next week with delta...I'm hoping this crap doesn't happen again, especially if they are actually working on OR implementing a DR solution. 

Sent from my XT1254 using Tapatalk

[jd30005]

I was just on Delta this past week and everything was fine (as fine as air travel can be these days).

[DOUG]

Headed to China Monday

[jc79]

I fly Southwest. I feel confident that if they have a critical system failure there's another clipboard on the wall they can grab to get things running again......


Sent from my iPhone using Tapatalk

[kvom]

Flew to Baltimore n SW Wednesday no problem.  Flying home in 3 hours.

[SemperFiArms]

I was just on Delta this past week and everything was fine (as fine as air travel can be these days).

Same here, flew out on Monday and and back in on Friday. The bird was 30 minutes late getting to Kansas City but that was due to weather that morning.

Both flights went off without a hitch.

[DOUG]

Hell getting to the airport today