Thursday, March 29, 2012

Comcast rolls back IPv6, after telling Netgear users they must buy new modems.

This is a follow up to our article Comcast abruptly cuts off users of Netgear DOCSIS 3.0 modems. To briefly recap, on Wednesday - March 21st, Comcast cable broadband customers in the San Francisco Bay area, who were using the (then Comcast approved) Netgear CMD31T DOCSIS 3.0 (Data Over Cable Service Interface Specification) modem, suddenly had their internet service disconnected.

Comcast's Retail DOCSIS Tiered Levels of Certification
One Star
A One Star designation indicates that the device has undergone DOCSIS testing by Comcast and has received the relevant Underwriters Laboratory ("UL"), Federal Communications Commission (FCC), and CableLabs certifications. Comcast's DOCSIS certification supplements CableLabs testing by adding things like performance and interoperability tests. This helps to ensure that, for example, a given device can support a given service tier's speed.
At the return counter of a local Frys Electronics store on March 22, an employee reported that they had been experiencing a surge in returns of the CMD31T from Comcast customers.
  • On the Netgear Community Forum, on March 22, a CMD31T user reported that Comcast instructed him to purchase a Motorola modem to solve the problem.
  • At online retailer Newegg (which has stopped selling the Netgear modem), on March 27, a CMD31T user also reported being told by Comcast to purchase a Motorola modem. This, despite the fact that on March 28, Netgear reported
"We were immediately in touch with Comcast regarding the issue and within a day corrective action was taken. Comcast had rolled back to the older system and all customers were online"
Also on the 28th, Comcast, who on March 23 had said "we support the Netgear DOCSIS 3.0 CMD31T cable modem" (and had granted it the 1-star rating above), issued a new statement:
“We recently identified that the retail NetGear CMD31T device ships with and runs an uncertified version of firmware that exacerbates a critical IPv6-related defect. To ensure Comcast customers with these devices will continue to have uninterrupted Internet service, we have rolled back IPv6 temporarily in some parts of our network to give NetGear more time to address the issue. Comcast anticipates NetGear will soon address the issue for their retail devices, which we will test and deploy on an emergency basis.”
IPv6 is the next generation internet addressing system, which will replace the current generation IPv4 that is running out of available IP (internet protocol) addresses. In Comcast's statement, which came one week after they began telling customers to buy Motorola modems, they claimed to have restored CMD31T functionality. However, Comcast did nothing to address the expenses that their customers had incurred as a result of their failure to perform a simple IPv6 test prior to launch, per their Certification Process.

To make matters worse, Comcast withheld that explanation for more than two days. On March 26, Charlie Douglas, the Senior Director of Public Relations and Communications at Comcast Corporation, told the EE Daily News:
We have (uncovered an explanation) and Netgear plans to contact you. I have passed along your contact info. Pls stand by. You should hear from them shortly.
Since Comcast knew what the problem was, and had (according to their official statement) restored the functionality of the Netgear modems, why would they withold the information until Netgear offered their own explanation on March 28? 
"Last week Comcast upgraded their Cable Internet head-end systems in select cities which caused the Internet connection failure for a few customers who had the NETGEAR CMD31T Docsis 3.0 Cable Modem. We were immediately in touch with Comcast regarding the issue and within a day corrective action was taken. Comcast had rolled back to the older system and all customers were online."
Neither Comcast or Netgear has said how many "a few customers" is. Comcast has now reversed themselves once again, stating on their device information website:
The retail NetGear CMD31T device has a known IPv6-related software defect. This prevents the device from being provisioned on the Comcast network.
We asked Comcast's Douglas - "What is Comcast's response regarding compensating customers for the expenses they incurred in unnecessary modem replacements?"

To which he tersely replied:
They should contact Netgear.
We did contact Netgear, and on March 29 we received this response from Courtney Furia, Account Supervisor at Netgear's PR firm of Weber Shandwick:
I wanted to let you know that we do not have an update on compensation at this time. We will be sure to provide you with any additional details as soon as they are available.
It's difficult to say which company, Comcast or Netgear, has handled this situation more poorly. Comcast has contradicted themselves repeatedly, and deliberately withheld information that would have explained the problem earlier. In our communications with service technicians, in person and at the company's call center, Comcast repeatedly denied that any change had been made to their network. Netgear, on the other hand, has simply stonewalled, taking 7 days to respond to our request for information. It would appear that both companies are trying to avoid any liability for the expenses their customers incurred.

This does not bode well for the upcoming worldwide launch of IPv6, on 6-6-12.  This week, at the IETF (Internet Engineering Task Force) meeting in Paris, a panel - which included representatives from Comcast and Time Warner Cable, met to discuss the IPv6 Launch. Cable modem manufacturer D-Link was also represented, but not Netgear. Hopefully, before the world launch day, the ISPs (internet service providers) will conduct a thorough set of tests on all approved modems. Imagine the chaos that will occur if customers around the globe get dropped off the internet, multiplying the problem that occurred in Silicon Valley last week many times over. It might not be a bad idea to call your ISP now, and ask them if they have tested your modem for IPv6.

1 comment:

Unknown said...

I've had problems with Netgear hardware in the past and don't recommend it. They have known defects and are unable to resolve them. eg: I have a Netgear "Smart Switch" that I purchased because it can do SNMP. It reguarly reports bogus values when polled and while being a gigabit switch, does not support 64-bit counters on the interfaces. This means if an interface has a large amount of traffic, the counter will wrap quickly.

I regret buying that hardware and other netgear hardware obtained over the years.