Web Hosting Canada reveals cause of outage

Montreal-based web hosting provider Web Hosting Canada (WHC) has revealed the cause of this weekend’s major outage.

In a blog post, founder and chief executive officer Emil Falcon said that the issue was caused by unauthorized activity by a third-party service provider.

“Based on our investigation to date, the morning of August 28 at approximately 6 a.m., an individual with a third-party service provider used their privileged account access to connect to one of our datacenter’s management portals and without authorization, initiated server reimaging on some of our backup servers, then on some of our production servers,” he blogged.

“Within only hours our incident response team had identified the issue and disabled access to the source account, preventing any further damage. The environment was secured, the individual fully locked out, and our disaster recovery plan immediately kicked into action, but damage was already done.”

He said that both production servers and backup servers were affected, resulting in data loss that may be permanent for some. A “large number” of web hosting and reseller hosting accounts were affected; Falcon said that his team was able to recover, or is in the process of recovering, more than half of the lost accounts.

“We can confirm that Cloud, Dedicated, Weebly and Managed WordPress accounts were largely unaffected,” he added.

Some data unrecoverable

However, he said, several production servers and their backup servers are still unrecoverable by his team, and the data recovery specialists he has enlisted believe that the potential for their recovery is low. While their efforts will continue, he has shifted focus to setting up new accounts for affected customers.

For customers with their own local backups, he advised contacting WHC’s support team, who will assist in getting sites up and running. Those without local backups have to start from an empty account. For them, WHC has activated new “LifeBoat” hosting accounts, accessible from the Client Area, which will remain free until at least January 1, 2022.

 Update: by contributing reporter Howard Solomon

The head of a Quebec-based company that uses WHC and who asked not to be identified said his firm has lost three to four months of data as well as copies of websites he manages. His firm’s most recent local backup is three months old.

It will take time to recover. “It may take two or three days, we’re going to put a little work into it, get back on our feet and have our websites back and running.” But, he added, those who don’t have local backups will be in trouble.

“I’m more disappointed than nervous,” he said. Until now he was happy with the service he got from WHC. “They were on the ball,” he said.

It was on Sunday, when he realized he wasn’t getting any new email, that he suspected there was a problem. He went to WHC’s support chat page and saw there were several dozen queries already lined up. Only when he dialed into the support phone line and heard a recorded message did he learn of the seriousness of the incident.

The destruction of data, he said, is as bad as a ransomware attack.

Would you recommend this article?

Share

Thanks for taking the time to let us know what you think of this article!
We'd love to hear your opinion about this or any other story you read in our publication.


Jim Love, Chief Content Officer, IT World Canada

Featured Download

Lynn Greiner
Lynn Greiner
Lynn Greiner has been interpreting tech for businesses for over 20 years and has worked in the industry as well as writing about it, giving her a unique perspective into the issues companies face. She has both IT credentials and a business degree

Related Tech News

Featured Tech Jobs

 

CDN in your inbox

CDN delivers a critical analysis of the competitive landscape detailing both the challenges and opportunities facing solution providers. CDN's email newsletter details the most important news and commentary from the channel.