Sidekick failure rumors point fingers at outsourcing, lack of backups
Backing up your personal PC to external media might still be a novel concept for some, but any IT manager fresh out of school can tell you that regularly backing up mission-critical servers -- and storing those backups in multiple physical locations -- isn't merely important, it's practically non-negotiable, and it only becomes that much more critical before undertaking hardware maintenance. Alleged details on the events leading up to Danger's doomsday scenario are starting to come out of the woodwork, and it all paints a truly embarrassing picture: Microsoft, possibly trying to compensate for lost and / or laid-off Danger employees, outsources an upgrade of its Sidekick SAN to Hitachi, which -- for reasons unknown -- fails to make a backup before starting. Long story short, the upgrade runs into complications, data is lost, and without a backup to revert to, untold thousands of Sidekick users get shafted in an epic way rarely seen in an age of well-defined, well-understood IT strategies.The coming weeks are going to be trying times for both Microsoft and T-Mobile, a sideline player in this carnage that ultimately still shoulders responsibility for taking users' cash month after month and keeping tabs on the robustness of its partners' workflows. We're betting that heads are going to roll at both of these companies, formal investigations are going to be waged, users are going to be compensated in big ways, lawsuits are going to be filed, and textbooks could very well be modified to make sure that lessons are learned for the next generation of college grads tasked with keeping clouds running. Why there weren't any backups -- even older ones -- that could've been used as a restore point is totally unclear, so we're hoping Microsoft has the stones to come clean for the benefit of an entire industry that wants to understand how to make sure this never happens again.























Reader Comments (Page 1 of 1)
brokensticks @ Oct 11th 2009 8:07PM
man, what a bone headed move. I'd be royally pissed if I was a sidekick user. Fortunately I'm a Palm Pre user... which brings me to the next point:
Palm, please backup our data. Thanks.
Donald @ Oct 11th 2009 8:19PM
"I suggest a new strategy: Let the device win".
Would this have been much of a problem if the device was pushing up instead of the server pushing down?
Kyle @ Oct 11th 2009 8:27PM
Don't be so wishy-washy Engadget. Tell us how you REALLY feel about all this.
Vince Cardillo @ Oct 11th 2009 9:52PM
I completely agree that ultimately, T-Mobile is fully culpable and accountable in this situation. Because like you said, in the end, they are the provider of the services. I just wrote a blog about this point-of-view as well.
http://www.vincecardillo.com/blog/2009/10/11/t-mobile-destruction-of-the-sidekick-brand/
vincent @ Oct 11th 2009 10:34PM
I've been there and it is easy to lose data even when you think you are backing up. Most shops never review backup coverage, verify backups, test restore procedures, or test disaster recovery plans, so these things are really hopes versus known entities. It takes a lot of diligence to keep data safe. That diligence is expensive and I doubt many consumers in a price conscious market segment would be willing to pay for it.
Vince Cardillo @ Oct 11th 2009 10:41PM
I guess the next billionaire is the one who invents the fool-proof backup system. In a society that is becoming exponentially more dependent on technology, we need service providers capable of guaranteeing its long-lasting viability and security. How many of us use Gmail? What would happen if Gmail *poofed* and vanished forever? Society tells us to adopt and rely upon technology, but the history of technology tells us not to.
carcomptoy @ Oct 12th 2009 5:49AM
Yes, things are definitely going to fail at some point, but it still strikes me as odd that practically NOTHING is stored on the Sidekick handsets themselves. I mean, yes Blackberries and iPhones depend on cloud networks as well, but things are still stored locally on the device itself. Just because my Curve isn't connected the internet, doesn't mean I can't see my contacts or view emails.
Peter F @ Oct 12th 2009 8:39AM
You can blame the outsourcing company all you want. Chances are, they said,"Where are the backups" and they were told," You shut up and do the upgrade, that's what you're being paid for"
Tim @ Oct 12th 2009 10:38AM
Hitachi makes SAN equipment, so it makes perfect sense that they'd be the ones doing the maintenance. Normally, with a SAN, the vendor does the really complicated stuff like backplane replacement, large firmware upgrades and the like.
This sounds like something wiped the disks without them knowing, or silently corrupted writes somehow. I'd like some more details on this one.
Jak Crow @ Oct 12th 2009 2:56PM
Uh, saying a SAN upgrade was "outsourced" to Hitachi is inaccurate. Most SAN vendors do all the maintenance for the customer as part of their support contracts for their high end units. It is not unusual to do so. I'm also willing to bet Danger's infrastructure is using virtualization tech like MS Hyper-V, and so suffer from the all to common attitude that they didn't need backups because everything was in a VM, even though servers being in VMs provides nothing in terms of disaster recovery.
AngryCustomer @ Oct 13th 2009 3:40PM
Forum.sidekickfail.com has recently been created as an open and neutral place sidekick customers can exchange ideas and vent without the fear of their valuable thoughts, ideas, and opinions being deleted and disrespected as T-Mobile has been doing on their forums.