Update April 28, 11:45 a.m. MDT:
Please click here to see the most recent update.
UPDATE 4/28/17 11:45 a.m. MNT: We have 0 calls in queue on our phone line, and are working through about 80 tickets related to the False Positive repair utility. A good portion of those are simply awaiting customer verification.
Please note, the utility was built to address only this specific false positive issue. It will be deactivated in the future.
If applications are operating normally on your systems, you do not need to implement the utility.
If you haven’t yet submitted a support ticket and you need the repair utility, please do so here. Include your phone number as well with the support ticket.
Thank you.
Page 12 / 12
Didn't opt in for beta fix:
Agent refused to checkin to cloud console.
-Booted workstation to safe mode
-WRSA -uninstall
-Reinstalled
Agent now checks in, no new false positives yet.
Agent refused to checkin to cloud console.
-Booted workstation to safe mode
-WRSA -uninstall
-Reinstalled
Agent now checks in, no new false positives yet.
I can confirm that it is still happening. Shut down another distributor client of mine. The server is set to ALL DISABLED, but it's possible a workstation did the damage. However, all of them are supposed to be set to "all disabled" as well.
UPDATE 4/27/17 9:21 a.m. MNT: We have 0 calls in queue on our phone line, and are working through about 100 tickets related to the False Positive repair utility. A good portion of those are simply awaiting customer verification.
If you haven’t yet submitted a support ticket and you need the repair utility, please do so here. Include your phone number as well with the support ticket.
Our sincerest thanks to the MSP beta customers who worked with us to further test and validate this repair. We truly appreciate the support of our customers and thank you for your patience.
If you haven’t yet submitted a support ticket and you need the repair utility, please do so here. Include your phone number as well with the support ticket.
Our sincerest thanks to the MSP beta customers who worked with us to further test and validate this repair. We truly appreciate the support of our customers and thank you for your patience.
Still have a few concerns today. I still see agents with commands "Not yet recieved" in the console going back to 4/24 and 4/25. Any idea when this will clear up or be addressed?
I also have 26 of the 138 sites I have in GSM showing that that "Need attention" though I'm sure that the majority of that number do not need attention any longer. I'm sure one or two of those may be legitimate but certainly not all.
Is this behavior expected at this point or do I need to get back on the line with your Support Team?
Thanks,
Jared
Hey, @ .
These are currently unknown issues from the false positives, so it'd be a good idea for you to reach out to our Support Team directly.
Business Technical Support: Call 1-866-254-8400
Open a Support Ticket
These are currently unknown issues from the false positives, so it'd be a good idea for you to reach out to our Support Team directly.
Business Technical Support: Call 1-866-254-8400
Open a Support Ticket
UPDATE 4/27/17 2:46 p.m. MNT: We have 0 calls in queue on our phone line, and are working through about 130 tickets related to the False Positive repair utility. A good portion of those are simply awaiting customer verification.
If you haven’t yet submitted a support ticket and you need the repair utility, please do so here. Include your phone number as well with the support ticket.
Our sincerest thanks to the MSP beta customers who worked with us to further test and validate this repair. We truly appreciate the support of our customers and thank you for your patience.
If you haven’t yet submitted a support ticket and you need the repair utility, please do so here. Include your phone number as well with the support ticket.
Our sincerest thanks to the MSP beta customers who worked with us to further test and validate this repair. We truly appreciate the support of our customers and thank you for your patience.
For those that have not seen this email yet from Mike Malloy, Executive VP Product & Strategy, I wanted to share this with you. We sent this out earlier today.
We want to remind you that we have created a repair utility to address a false positive issue that arose on Monday.
On April 24 at 11:52 am MT, some good applications were mistakenly categorized by Webroot as malware. This created false positives across the affected systems and resulted in those applications being quarantined and unable to function.
Our repair utility will release and restore quarantined applications to working order on the affected endpoints.
To obtain the repair utility, please open a support ticket, or reply to your existing support ticket related to this issue. Please include your phone number in the ticket.
We appreciate the support of our customers and partners, and thank you for your patience.
Yours sincerely,
Mike Malloy
Executive VP of Product & Strategy
We want to remind you that we have created a repair utility to address a false positive issue that arose on Monday.
On April 24 at 11:52 am MT, some good applications were mistakenly categorized by Webroot as malware. This created false positives across the affected systems and resulted in those applications being quarantined and unable to function.
Our repair utility will release and restore quarantined applications to working order on the affected endpoints.
To obtain the repair utility, please open a support ticket, or reply to your existing support ticket related to this issue. Please include your phone number in the ticket.
We appreciate the support of our customers and partners, and thank you for your patience.
Yours sincerely,
Mike Malloy
Executive VP of Product & Strategy
UPDATE 4/28/17 11:44 a.m. MNT: We have 0 calls in queue on our phone line, and are working through about 80 tickets related to the False Positive repair utility. A good portion of those are simply awaiting customer verification.
Please note, the utility was built to address only this specific false positive issue. It will be deactivated in the future.
If applications are operating normally on your systems, you do not need to implement the utility.
If you haven’t yet submitted a support ticket and you need the repair utility, please do so here. Include your phone number as well with the support ticket.
Thank you.
Please note, the utility was built to address only this specific false positive issue. It will be deactivated in the future.
If applications are operating normally on your systems, you do not need to implement the utility.
If you haven’t yet submitted a support ticket and you need the repair utility, please do so here. Include your phone number as well with the support ticket.
Thank you.
For those that have not seen this email yet from Mike Malloy, Executive VP Product & Strategy, I wanted to share this with you. We sent this out earlier today.
As a reminder, the repair utility to address the false positive issue that arose on Monday, April 24, is available. The utility will release and restore quarantined applications to working order on the affected endpoints.
Please note, the utility was built to address only this specific false positive issue. It will be deactivated in the future.
If applications are operating normally on your systems, you do not need to implement the utility.
To obtain the repair utility, open a support ticket, or reply to your existing support ticket related to this issue. Please include your phone number in the ticket.
I want to thank each of our customers and partners for their patience during this time, and we are committed to earning your trust going forward.
Yours sincerely,
Mike Malloy
Executive VP of Product & Strategy
As a reminder, the repair utility to address the false positive issue that arose on Monday, April 24, is available. The utility will release and restore quarantined applications to working order on the affected endpoints.
Please note, the utility was built to address only this specific false positive issue. It will be deactivated in the future.
If applications are operating normally on your systems, you do not need to implement the utility.
To obtain the repair utility, open a support ticket, or reply to your existing support ticket related to this issue. Please include your phone number in the ticket.
I want to thank each of our customers and partners for their patience during this time, and we are committed to earning your trust going forward.
Yours sincerely,
Mike Malloy
Executive VP of Product & Strategy
This event certainly uncovered some big issues.
Now that it's behind us I would like to know what your plans are for making sure the trifecta of bad (detection, backlog, no kill switch) does not happen again.
I have no doubt you are taking this very seriously. Just looking for more information.
Thanks.
Now that it's behind us I would like to know what your plans are for making sure the trifecta of bad (detection, backlog, no kill switch) does not happen again.
I have no doubt you are taking this very seriously. Just looking for more information.
Thanks.
Hi dsm55 and others
We are sending an email and posting a letter from our CEO, Dick Williams, which outlines some of the many steps we have taken already and are actively working on to 1) prevent similar issues; 2) communicate more rapidly and with better coverage; and 3) improve our systems so that you can take remediation steps yourself with better information. That note and others in the weeks ahead will hopefully provide you the assurance you need to depend on Webroot as a solid partner. We know this event was a big one and have neither dismissed it nor ignored its many lessons. Thanks for your note.
Mike
We are sending an email and posting a letter from our CEO, Dick Williams, which outlines some of the many steps we have taken already and are actively working on to 1) prevent similar issues; 2) communicate more rapidly and with better coverage; and 3) improve our systems so that you can take remediation steps yourself with better information. That note and others in the weeks ahead will hopefully provide you the assurance you need to depend on Webroot as a solid partner. We know this event was a big one and have neither dismissed it nor ignored its many lessons. Thanks for your note.
Mike
Thanks.
I came across this thread while doing some due dilligence before I recommend adopting Webroot as the preferred security product for the MSP that I am Service Deliver Manager of.
You will understand I'm interested to hear more about the improvements made in quality control and incident response to avoid the headaches that other MSPs have discussed in this thread.
Can anyone direct me to a summary of changes implemented in the wake of lessons learned from this
incident?
Thanks.
You will understand I'm interested to hear more about the improvements made in quality control and incident response to avoid the headaches that other MSPs have discussed in this thread.
Can anyone direct me to a summary of changes implemented in the wake of lessons learned from this
incident?
Thanks.
Hi @ , and welcome to our Community!
The actions we have taken include:
If there are specific questions we can answer for you, we would happily jump on a phone call with you.
The actions we have taken include:
- We immediately repaired and strengthened our safeguards related to the false positive on the day it occurred. In the days and weeks following, we introduced a number of new safeguards – both technical and procedural – to reduce our exposure to similar incidents.
- We scaled up our infrastructure to ensure our console performs well and supports the high volume of agent commands that are likely during any service issue.
- We’ve improved our communication around product capabilities, updates and issues. This includes the introduction of a series of certification programs to scale our information sharing on best practices, as it became clear that customers who had greater familiarity with the best practices in using our products were able to resolve issues in their environments and return to normal operations faster. (link to partner certification: https://www.webroot.com/us/en/about/press-room/releases/webroot-launches-certification-program)
- Finally, we are increasing the frequency of early communication across all our channels—email, social media, support, and community—so that when issues arise, the likely impact and status of remediation are shared out as quickly as possible.
If there are specific questions we can answer for you, we would happily jump on a phone call with you.
Reply
Login to the community
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.