Dear Customers,
Webroot experienced a disruption to our service the week of April 24.
I am writing you to explain what happened, the steps we took to resolve the issue, and how we intend to improve our service moving forward. Above all, I want to apologize for the frustration this caused you, and let you know that we are committed to earning your trust going forward.
The issue arose at 12:52pm MTN on Monday, April 24, when we incorrectly identified multiple files as malware – a false positive. Our safeguards detected the error and stopped execution, but only after 13 minutes. By that time a number of our customers already had multiple endpoints with quarantined files, which in some cases impacted the operation of benign applications and software.
We began rolling back the false positive immediately and we developed and deployed a manual fix the same afternoon. Our team also worked in parallel to develop an automated repair utility, which we rolled out on the morning of April 26.
Webroot was not breached at any time, and our protection against threats was not affected in any way during the course of these events.
However, our service did not meet the standards you expect and deserve from Webroot. From our conversations with many of you, it is clear that there are actions we need to take to improve.
First, we repaired and strengthened our safeguards related to the false positive on the day it occurred. Going forward, we will introduce additional safeguards for faster detection and to reduce risk.
More broadly, we have initiated a full-scale review of our systems and architecture to identify any possible points of failure. We already conduct extensive testing on our software and threat intelligence processes, but we will be investing more time and resources in those efforts, with a focus on hardening any identified weaknesses. We are also scaling up our infrastructure to ensure that our console performs well and supports the high volume of agent commands that is likely during any service issue.
Lastly, we are improving our communication around product capabilities, updates and issues. It was clear that customers who had greater familiarity with the best practices in using our products were able to resolve the issue and return to normal operations faster. To ensure this knowledge is widely shared across the Webroot customer base, we will be introducing a curriculum of best practices this summer. We will also introduce a service availability page on our website, and enable updates through a subscription RSS and SMS service. And, we will increase the frequency of early communication across all our channels—email, social media, Support and Community—so that when issues arise the likely impact and status of remediation are conveyed to you as soon as possible.
We are determined to learn from this service issue experience, and build an even better Webroot. Please reach out to me personally with any questions or suggestions, or contact your sales or support contacts.
Thank you for your continued support.
Kind regards,
Mike Malloy
Executive Vice President of Products and Strategy