City tries to learn from holiday water outages

While it has been over six months since the winter water outage that affected thousands of Asheville residents, several questions still linger over how such an incident could have occurred and how long the remedy will take. An independent review committee found that inadequate preparation for cold weather and a lack of training in crisis communication were the most critical missteps that amplified the effects of the outage.

The nine-member committee was formed in January to analyze the cause of the outage as well as to look at the city’s communications and emergency response. The committee concluded that the incident was largely preventable, in part because the Water Resources Department didn’t quite understand how the Asheville water system functions as a whole.

Outages south and west

Temperatures fell far below freezing from Dec. 23-26, a cold snap not seen since January 2014. The cold triggered a series of waterline breaks beginning Dec. 24, eventually leading to 27 breaks, many on larger distribution lines. Meanwhile, customers’ pipes burst on their sides of water meters, and the leaks sent water demand soaring. By Dec. 26, thousands of customers in the southern and western areas of Asheville had no water.

Even before the breaks, water demand was running about 28 million gallons, nearly 25% higher than average, likely because of holiday festivities. The breaks added to that demand, according to Ted Tyree, a water engineer with the Knoxville Utilities Board and a member of the review committee.

The water system’s rated capacity is 43.5 million gallons on paper, but the demand proved to be too much because of missteps both during the crisis and months before. As pipes were bursting throughout the city, the Water Resources Department had to shut down the Mills River plant, one of the system’s three water treatment facilities, on Dec. 24 because monitors detected particulate levels in drinking water that were close to the state’s regulatory limit. Staff members found the cause to be several clogged chemical lines that purify the water. They also found that components of the filter system responsible for loose particle filtration were frozen and nonfunctioning. According to the report, all of these issues “necessitated time-consuming corrections.”

John Shaw, an engineer with over 25 years of experience in water distribution and a former Asheville resident, gave Xpress his thoughts on how the city handled the crisis.

“Why were those facilities not cold-protected to begin with?” Shaw asks. “There are several things that could have been done, from insulation to increasing the depth of the bury to changing the parent pipe material. This is nothing new and should have been part and parcel of their design.”

Cutting off the south

With the Mills River facility down, the water department ramped up the two remaining treatment plants in northeast Buncombe County near Black Mountain. However, they couldn’t keep up with demand and maintain full pressure and service for the southern and western portions of the system.

On Dec. 26, Water Resources Department leaders decided to isolate most of the southern part of the water system to maintain pressure and service in the remaining portions. That meant even more people lost water service. At a press conference on Jan. 3, Mayor Esther Manheimer said that by cutting off service to the south, the city was able to prevent the whole system from going under a boil advisory.

“At some point, the decision was made to tie off the North Fork (facility) from the south because once Mills River came back online, if you didn’t tie it off, the whole system would have to go under a (boil water) advisory,” Manheimer said. “So, the decision to tie off the south was to basically preserve the rest of the city, so the rest of the city with the hospital system, everything, would not have to go under a boil water advisory.”

On Jan. 5, after being down for four days, the Mills River facility was slowly brought back online.

An elusive valve

Meanwhile, the Candler Knob storage tanks, which are responsible for holding and distributing water to the western portion of the system, were still not refilling as expected.

After several days of trying to find the problem, the water department determined that there must be a valve closed somewhere along the main transmission line that feeds the western storage tanks. After searching for less than an hour, crews found a closed 24-inch valve in the River Arts District on the east side of the French Broad River. The valve was immediately opened, at which time the Candler Knob tanks began to fill much more quickly, which expedited full service.

But staff had known there must have been a closed valve months earlier, according to consultant’s report. After a similar low water pressure incident the summer before, the department hired an environmental engineering consulting firm, Hazen, in January 2022. The only way Hazen could get the modeling to simulate what was occurring with the Candler Knob tanks during periods of high demand was to close a 24-inch valve. Based on that finding, Hazen strongly suspected a closed 24-inch valve existed in the River Arts District just east of the French Broad River. Water department staff members said they looked for a closed valve after Hazen’s report but couldn’t find it.

“Just to be clear, they were already aware of the closed valve,” Shaw says. “They chose to ignore their consultant and not to act.”

Wishful communications

As staff scrambled to restore water service, the Water Resources Department faced intense pressure from city officials to release a timeline to the public for restored water service. The report stated that “frustration over the inability to both resolve the situation, and properly communicate about it, led to direct pressure by the mayor to provide ‘best case’ information to the public over the significant objections of WRD and CAPE [Communications and Public Engagement] staff.”

Additionally, the report noted that there was “a strong insinuation” from the mayor that the jobs of people objecting to the rosy forecast were on the line. Behind the scenes, the water department had no idea when water would be restored but released a statement anyway that said that the department had the “goal” of renewing water services within 24-48 hours. Staff noted during the review that this was an “exceedingly optimistic goal.”

The committee found the unrealistic promise of service restoration to be the city’s most egregious communications misstep. The strenuous objections from WRD and CAPE staff should have superseded the mayor’s insistence that such a message be released to the public.

Kim “Dirt” Murphy, local farm manager and member of the Independent Review Committee, noted in her presentation to the Buncombe County Board of Commissioners on June 20 that she believed that this series of miscommunications was primarily a result of an overall lack of emergency response communication training across the city.

“That is the tragedy in this. I think that if everyone involved, including elected officials, had emergency training, everyone could have stayed in their lanes, and this situation would have never happened, in my opinion,” Murphy said.

Another misstep was issuing an outage map. The city knew that, unlike power outage systems, its meter-reading technology couldn’t provide accurate, real-time outage information.

West Buncombe Fire Chief Dennis Fagnant noted major safety concerns that the lack of an accurate water outage map posed to his department.

“Fire chiefs like myself were using that map to determine what areas in our districts did or did not have water,” Fagnant says. “In this week there were five working fires that West Buncombe responded to, and there is a chance that myself or other fire chiefs would not have been prepared to respond because we didn’t have accurate information as to where we could pull water.”

No immediate fixes

The committee presented 27 recommendations for the city, including an updated crisis communications plan, creating a Water Utility Advisory Panel and implementing the National Incident Management System. It also recommended that the city reevaluate the overall role of the department’s engineering division and hire a production engineer to oversee the day-to-day operations of the water utility.

City Manager Debra Campbell said at the June 16 city council meeting that the city is committed to making the changes but warned the public not to expect any immediate changes.

“I want to caution the community: It’s not going to happen overnight. There’s a lot in this report. And there’s a lot of budgetary considerations that need to be made,” said Campbell. “But we are committed.”

Campbell didn’t provide any cost estimates for implementing some or all of the recommendations.