While we were researching tools, Comcast invited us to visit an operations center to see the tools they use to investigate problems. Readers may remember that we have a couple of rental condos on Sanibel Island. While we were visiting in mid-August, we drove into Fort Myers and spent an afternoon at Comcast's new headquarters building for Southwest Florida.
When we arrived, Barbara Hagen, Comcast's Area Vice President for Southwest Florida, took us on a walk around the facility. Barb told us that Comcast's Southwest Florida system serves Lee and Collier Counties, with more than 400,000 homes passed and about 275,000 customers. Until recently, Comcast had people working at six locations spread out many miles apart. She said the new facility, which opened in June, brought most of those people together.
As we walked around, we saw areas devoted to the call center, network operations, commercial sales, and more. The signs on the restroom doors let you know you're in a cable operation.
Barb introduced us to Mark Graves, Director of Technical Operations, who has been in the cable industry for more than twenty years. He joined Comcast in 2001 and recently moved to the Southwest Florida system. He is responsible for all technical operations, including construction, engineering, head end, network, and all field techs. The field techs are organized into two groups -- "line techs" are responsible up to the tap, "service techs" from the tap into the residence.
Mark brings a strong focus on improving customer service. He said he was changing their operating procedures to focus on solving the customer's problem as quickly as possible--he said he saw his mission as "getting the right people at the right job at the right time."
We told him about a cable problem we had last winter. When we arrived on a Saturday at our condo complex for a visit, our cable modem wasn't working and many of our digital channels had disappeared; analog television was working fine. We called Comcast to report the problem, and told the customer service representative that we suspected the problem was with the complex rather than our condo. She said someone would come Monday to look at the problem.
On Monday morning, we learned that several other condos had reported problems with digital television, and the cable modem was out of service at the condo association office. When a service tech arrived later that morning, he had a handful of reported problems in the complex. It took him only a few minutes to determine that the source of the problem was somewhere outside the complex. That required a field tech, so he called dispatch to request one. The field tech came to Sanibel island on Tuesday and fixed the line problem; the service tech came back on Wednesday to make sure everything was working right.
Mark said that was a good example of what he was trying to fix. He hoped their new diagnostic tools would show that the problem was more likely on the line than on the drop side of the tap, so they would dispatch the right person.
More important "the process problem is being fixed." He said that their previous procedures had a hard division between the roles of the two field tech groups. The new procedures--in effect for two weeks at the time of our visit--blur the lines between the groups, with the goal of resolving the problem quickly: "don't complete the service order until the customer is satisfied."
While we were talking about tools, we were joined by Mark Gonzalez, Dispatch Supervisor, and Ralph Solomon, Network Dispatcher. They set up a display on a large screen in a conference room so they could show us the tools they use to identify and resolve field problems.
Mark Gonzalez said Comcast has developed many kinds of tools for diagnosing problems--in the head end, at the CMTS level, for optical nodes, and to read out specific levels at specific customer devices. He said they maintain a mapping of customer accounts by MAC address so they can display the geographic locations of problems.
One of their key tools, Line Problem Indicator v2, allows them to visualize the location of problems. The picture above shows an "account view". The status of several accounts is shown at the bottom of the screen. The location of each account is pinpointed on a Google map at the left, each color-coded to indicate the status.
The second picture shows the "problem view." The current problems are shown at the bottom of the screen, with bullets indicating the type of problem ("low downstream transmission level", "high upstream signal to noise ratio", etc). The map pinpoints the location of the selected outstanding problem.
These tools certainly appear to make it much easier for Comcast to spot the location of problems, and dispatch the appropriate field tech to fix them. We're hoping we don't see cable problems the next time we visit Sanibel, but we expect we'll see much faster resolution if we do.
( www.comcast.com )