Welcome to the data apocalypse

The best thing about conferences is that you get to hear about all the newest things and learn how great things are … going to be.

At Storage Network World (SNW), we learned how much easier life was going to be for IT folks when they really start using Software-Defined Storage (SDS).

At the recent Cloud Expo, we discovered that IT folks are just like the rest of us. They don’t worry about their data/content management issues; they just park them it in the cloud.

All of the personal (and business) stuff that you create – with your new tablet’s 32GB of storage; new smartphone with 32GB of storage; new ultrabook or notebook with your choice of 256 GB of SSD or 1TB HD – it all goes into the cloud.

You put it there for “later” at places like Facebook, Google, YouTube, DropBox, you name it.

Computers Count

Sure, you look down on us when we mumble the word “computer;” but the device we previously called a computer took a breather from sales for a year or so and yes, it has changed/been redefined.

IDC says PCs include desktops, portables, mini-notebooks and workstations; not handhelds, x86 servers or tablets.

Gartner says, they include desk-based PCs and mobile PCs, including mini-notebooks; not tablets.

Jon Peddie Research redefines the category slightly to include X86-based devices and says overall, the “refreshed” category sales are increasing slightly through 2014.

Strategic Analytics’ recent U.S./EU consumer survey agreed because they found that nearly 23 per cent of respondents are very or somewhat likely to buy a portable PC during the next year, compared to less than 17 per cent in the same period a year ago.

It’s a good thing, because we looked at data center storage sales and the capacity certainly wasn’t being added there.

It turns out you aren’t dumping all of that fantastic content of yours in the data center for storage and you aren’t carrying it around with you on your mobile device (tablet, smartphone).

Storage Homes – For a number of years, IT managers were worrying how they would store and protect not just their vital data but also all of the content/information people comingled with their personal stuff on their mobile devices and parked in the data centre. It turns out people are a little more focused on protecting their content so they store it on their systems, in their homes.

Nope, true to form, you’re banking close to home.

You’re keeping your stuff – business/personal – on your computer and home server. After all, no self-respecting hacker/whacker would bother to dig there when there are so many easy places to pick out there in public!

Digital Content

Once the Internet, Web and other money-making places took hold, folks went crazy with their digital business, creative, personal, healthcare, governmental content production.

Appy World – With the advent of portable computing and mobile devices, individuals and organizations were able to develop millions of apps to make it easier to create, work, exchange information/content and be entertained anywhere. Equally important were the Web 2.0 locations that let you store and share all of that important stuff.

You know, that’s how Google built its global profit-producing machine … on the back of your fantastic work, fantastic content.

But you don’t mind because as they say, “any exposure is good exposure.”


That’s you, not me but it doesn’t really matter because the world is drowning in digital whatever – good, bad, indifferent.

Wikipedia does a good job of explaining it and impressing you.

Data Storm – It is difficult to remember when the data and content you wanted wasn’t instantly at hand. Mobile computing devices, the Internet and Web put the stuff in your hands instantly. The problem is that nearly all of the material you use for business or pleasure only exists digitally and needs to be saved, preserved for future generations.

The Wikipedia site estimates that 98 times more data exists today than was around in 1986.

A McKinsey researcher made the figure even more impressive by noting that more data was produced in the past five years than was produced since the first papyrus library.

The scary part is … much of that information solely exists in digital format.

But if you take a closer look at those data stacks, you’ll notice that much of it isn’t what we call active data – you know the stuff that is accessed, used, moved around, worked with.

An executive at EMC recently noted that 95 per cent of the content/data that is prepared is ever used or touched by computers or people.

And the longer it sits in storage somewhere, the less likely it is to be used except when really, really needed.

Frozen in Time – If data is going to be used, mined, shared, re-purposed; most experts say it will be used/viewed in the first 60 days. Thereafter, access falls off dramatically, even though it some – if not all – of the information should be retained for historical or legal purposes.

A researcher at SNW noted that 75 per cent of data prepared and stored is what we call fixed data.

Legal Content

Fixed data is enterprise data that sits for extended periods but is required by legal compliance, regulations or it is precious IP (intellectual property – patent) information that could have real value down the road.

Just look at what Kodak, Nokia and a number of others recently received when they sold their patent libraries … lotsa’ money!

That’s the kind of stuff the cloud people say is best put in their clouds – public and private.

You know, those big power-hungry facilities located in central Washington where Microsoft, Google and others have their data storage farms.

Data Farming – It’s hard to believe that the guy in the picture is a data farmer. No, the building to the left isn’t a data farm; it is to the right of him in central Washington. Cloud storage sounds great when you say it but the storage is done in ultra-large data centers like the one in the background built around the globe, often clustered where they have access to cheap power to meet their electrical and air conditioning requirements.

Microsoft alone – and the others spread around the river and around the globe are similar – they use enough electricity to power about 29,000 American homes.

The largest locations for large data centres looking for huge amounts of extra power are China and Brazil with India, Mexico and Russia also being close to the top of the facility and DC investment growth locations.

That’s fine for the 40K plus data centres huddled around cheap power facilities around the globe; but the truth is, your data has reached the point where we are flying without a safety net.

Active Storage

Even SDS only handles the needs of organizations and individuals for short-term, active data.

 None of the cloud service providers (CSPs) really focus on long-term, reliable, standard and persistent focus on ways to protect and preserve data, information, content. Not just for the 5-10 years they are in the IT management job but 20-50 plus years when that content/data may be even more valuable than the Big Data mining for customer information that we focus on today.

Cheap Storage – As quickly as storage device manufacturers – flash, HD, tape, optical – produce media, organizations and individuals rush to fill the devices. Much of the content is white-noise (material not worth saving) but even then, keeping ahead of daily storage needs is difficult, let alone long-term storage/retention.

IDC (and EMC, which sponsors the firm’s annual storage report) note that even with storage media being so cheap and readily available, there is still more data around than places to store it.

The company warns the industry that there is not just one storage solution but there needs to be a complete range of well-planned out, well-executed storage strategy solutions that meet the needs of organizations, individuals and data.

Even with the solutions being used today, the common approach is to focus on:

  • High-speed, rapid information access (SSD);
  • Big, cheap, decently fast devices (HD) for archiving; and
  • If you absolutely have to use tape for frozen archiving, keep your fingers crossed no one will ask for it, access it.

Isn’t it ironic that:

  • Components with the shortest life are used for long-term archiving;
  • Components with the highest failure rate are used for archiving; and
  • Components with the highest energy requirements are used for archiving.

Real Preservation – While most people think of optical as an entertainment medium, the write-once products were first developed for long-term data/content preservation. The CD above was written in 1993 and can still be read 20 years later. Today’s advanced BD preservation media delivers a data life of 50 years or more without constant media migration or requiring power (except when read/written) or AC, delivering piece of mind and savings.

I mentioned the dilemma to an integrator friend while at SNW and he said the focus today is on “data factories.”

These are facilities based on hyperscale and hyper-standardization that handle the traditional content storage and archiving with little/no concern about preservation.

For preservation, he said write-once optical was best and constant, regular migration of content was second.

He emphasized that the stories surrounding tape just aren’t supported by real-world facts. “The real danger is that tapes don’t even support the lifespan for data that most companies want to achieve in their archival policies.”

To illustrate the benefits of optical preservation, he sent me the above photo of a ripped CD manual he had received back in 1993 and pointed out that the disc can be read even today on virtually any CD, DVD, BD player. He added that:

  • Even in large data centres, optical media is based on energy-passive technology and only requires power when read/written; and
  • Unlike other solutions, optical media doesn’t need to be maintained in an air-conditioned facility–AC accounts for 50 per cent of data centre costs today.

According to the integrator, the problems the new BD preservation solutions face today are:

  • It still has an image of yesterday’s movie and entertainment discs, which it isn’t; and
  • Most folks use the term archiving when what they really want is long-term preservation – two totally different goals.

He emphasized it would be nice to have all of the content and data preserved for future generations, but what we really need is, “just the best endeavors of human creativity and intellectualism we can proactively identify and protect today and tomorrow so they will be accessible in the future.”

Maybe folks will be able to say the same about their corporate/personal data and priceless content someday with the right preservation solution.

Or, maybe there will be a great new idea at January’s Storage Visions.

Would you recommend this article?


Thanks for taking the time to let us know what you think of this article!
We'd love to hear your opinion about this or any other story you read in our publication.

Jim Love, Chief Content Officer, IT World Canada

Featured Download

Related Tech News

Featured Tech Jobs


CDN in your inbox

CDN delivers a critical analysis of the competitive landscape detailing both the challenges and opportunities facing solution providers. CDN's email newsletter details the most important news and commentary from the channel.