Former Asheville Water director: ‘Incontrovertible evidence’ that shuttered pump station was key to water outage

by John Boyle, avlwatchdog.org

A former City of Asheville water director — who was also a member of the Independent Review Committee that investigated the holiday water outage — says the city is overlooking the key to the crisis, and another major service disruption could happen if it’s not addressed.

Mike Holcombe, water director from 1993-97, remains adamant that the city is downplaying what he says is the major cause — the decommissioned East Asheville Booster Pump Station. That multimillion-dollar facility opened in 1993 and closed in 2010. The city has little documentation as to why it was shuttered.

The pump station was designed to boost water flow during periods of high demand. The outage, which the city primarily pinned to two closed valves and the frozen Mills River water plant, spanned 11 days over the Christmas holidays, shuttered businesses and left thousands without water.

Holcombe, a South Asheville resident, lacked water for five days.

“I have no hesitation — and believe I could defend in whatever forum I had to — that the absence of the pump station was 85% or 90% of the problem,” Holcombe said. “The closed 24-inch valve, it was a problem, but if you’d had the pump station operational, you would’ve discovered that problem a lot quicker, because you would have been pumping against a closed valve.”

In its official report, the IRC said three major factors played roles in the outage, including a closed 24-inch valve in the River Arts District and another that was open only 10%. The freeze-up of the city’s Mills River water plant in northern Henderson County, one of three facilities providing water to the city, was another key, the IRC said.

Holcombe believes that assessment is misleading. He cited a study conducted for the IRC last spring by a city consultant, Hazen Engineering, that showed what the projected water flow rates would have been during the crisis if the booster pump station had been in use.

“Incontrovertible evidence was provided to the City Council in the form of the attachments to the Hazen engineering report, which indicate that had the East Asheville Booster Pump Station been running, there would not have been any water system outage during Christmas 2022,” said Holcombe, who is critical of what he says is the city’s lack of transparency on the matter.

Those flow rates and modeling have not been made publicly available.

If the Mills River water plant were to go out of service again, the city could be vulnerable, Holcombe says. Holcombe is so passionate about the topic that he’s started a Facebook page about the East Asheville Booster Pump Station and why it’s so crucial.

The IRC report called for a preliminary engineering study of the pump station to determine what would be needed to return it to service for “system redundancy during emergency events and future demand.” Water Resources Department Director David Melton initially said the preliminary engineering study could take 18 months, a time frame that Holcombe believes isn’t quick enough.

‘To a certain extent, I would agree with Mike Holcombe’

The North Fork Reservoir near Black Mountain, with a capacity of 31 million gallons per day, is the city’s main water source. Nearby Bee Tree’s capacity is 5 million, and Mills River can produce 7 million gallons a day, according to the city’s annual water report.

The nine-member IRC, which included Holcombe and two other water experts, issued its report in June.

“The failure to locate the suspected closed 24-inch transmission valve in the River Arts District, possibly closed since April 2018, proved to be a major contributor to the event,” the report said. The water department was told about the closed valve nearly a year before the outage but failed to find it, Asheville Watchdog previously reported.

The IRC report noted the water system experienced unusually high demand when the problems began on Christmas as the temperature plunged to near zero. More than two dozen city waterlines, as well as scores of customer-owned lines, broke. But water utilities are designed to handle waterline breaks, Holcombe said.

“The thing people need to understand about waterline breaks is that is normal business at a lot of utilities,” Holcombe said. “So, you go and you valve it off where the break is … and it doesn’t affect anything else. It doesn’t affect supply.”

Ted Tyree, an engineer with the Knoxville, Tenn., water utility and a member of the IRC’s Water Systems/Operations subcommittee, previously told The Watchdog the breaks likely would have left just a few hundred people out of water. The closed valves were key, Tyree said, because the city couldn’t get customers enough water once Mills River went down.

Asked about Holcombe’s assertions, Tyree said the committee “wanted to land on a consensus” for the city and the mayor in its official report, so it “stopped short of recommending staff bring that pump station back.”

But under “immediate recommendations,” the IRC report states the city should: “Evaluate the existing pumps at the East Asheville Booster Pump Station, by inspection and with pump tests, for the purpose of producing an EABPS Preliminary Engineering Report, detailing what will be needed to refurbish the EABPS and return it to useful service for system redundancy during emergency events and future demand.”

“It [the crisis] shouldn’t have been anywhere near the magnitude it was,” Tyree said previously.

The IRC described the outage as “avoidable and preventable.”

“I think the magnitude and duration was what was largely avoidable,” Tyree told The Watchdog.

“To a certain extent, I would agree with Mike Holcombe,” Tyree said. “But also, that’s not a cheap fix for staff, and they have done some significant piping improvements over the course of the years.”

The easiest, quickest fix identified was to get a valve assessment team in place to make sure all key valves are open, as the closed valves “would’ve alleviated much of the event,” Tyree said. The IRC also recommended better freeze-proofing at Mills River and communications improvements.

“With the pump station, I think where Mike is right is, yes, it would have helped, even with the valves closed,” Tyree said. “But there’s a price for that — I know all water utilities are not flush with money. You’ve got to make decisions about prioritizing improvements, so where do you get the biggest bang for the buck?”

The 2019 Hazen Engineering report put the potential cost of reinstating the station at a “few hundred thousand dollars,” Tyree said, adding “I think it’d be closer to seven figures or more.”

Tyree and fellow IRC members Keith Webb and Mike McGill made presentations at the June 13 City Council meeting. While not scheduled to speak, Holcombe told the council the booster pump station was the key to the outage.

The $3.5 million booster pump station went into service in the 1992-93 time frame, according to the IRC report, and it “enables the full combined capacity” of 36 million gallons a day from the North Fork and Bee Tree plants. 

Holcombe said in the four years he was water director, the station, designed to come on when pressure in the Haw Creek junction dropped below 200 pounds per square inch, probably did so “less than a dozen times.”

“And a couple of those times were major fires, which is another reason you need the pump station,” Holcombe said. 

Once the Mills River plant came online in 1999, the pump station went into standby mode, he added.

City has no sense of urgency, Holcombe says

Holcombe also criticizes the city’s sense of urgency in looking to restore the booster station.

Asheville Mayor Esther Manheimer is aware of Holcombe’s campaign to get the station back online.

“Of course, this council supports the recommendations of the IRC and moving ahead with the study,” Manheimer said. “I assume Mike wants it faster.”

Manheimer, who was first elected to City Council in 2009 and became mayor in 2013, said she doesn’t know why the booster station was taken offline.

City Council member Maggie Ullman said she’s glad the city is considering the feasibility of reopening the booster station. She wants to see how the station fits in with today’s water system, what it would cost to restart and its benefits. And she offered a caveat.

“Theoretically, it could bring a solution and some practicality to the system, but you just don’t turn on a truck that hasn’t run in 10 years and it’s going to run beautifully,” Ullman said. “That’s the question I have.”

Holcombe said the station’s pumps were designed to last 100 years.

Ullman is skeptical that the booster pump being down was the main cause of the outage. 

“The speculation on this that it was the silver bullet, I think that’s not what the report said,” Ullman said. “It said it was a piece of the puzzle, so to overexaggerate it as a silver bullet is not in service to us moving forward effectively.”

Concerns about transparency, too

Holcombe also has concerns about the city’s transparency, saying key parts of a Hazen Engineering report given to the city, specifically modeling reports on water flow related to the pump station, were not included in publicly released documents.

The city previously denied The Watchdog’s request for the Hazen attachments, citing security concerns regarding disclosures of locations of key water equipment.

Holcombe said he was surprised and disappointed the full reports weren’t made public. He understood they were supposed to be released once sensitive information was redacted from the report, he said.

City Attorney Brad Branham maintained those modeling reports cannot be made public because of security concerns.

“The only portion of this that was fully redacted were the modeling exhibits,” Branham said via email. “Specifically, they contained considerable information protected from public record production due to sensitive security information. However, the full executive summary of the report was produced at that time.”

The IRC and City Council received the reports in their entirety. Members of both bodies had to sign nondisclosure agreements, the city said.

Holcombe also said, and Tyree confirmed, that the IRC could not get any documents or paperwork explaining why the city chose to decommission the booster pump station.

“There’s not a single piece of paper written down documenting the retirement of the East Asheville Booster Pump Station,” Holcombe said. “We asked for it. Not a piece of paper — no email, no report.”

Holcombe said he thinks the city found the station unnecessary after the Mills River plant came online.

Holcombe also criticized the water department for saying the preliminary engineering report could take as long as 18 months.

The city disputes Holcombe’s assertions

City of Asheville spokesperson Kim Miller provided answers to The Watchdog’s questions on behalf of the Water Resources Department. Melton, the water director, has declined interview requests from The Watchdog.

Eighteen months is a “rough estimate” and it “encompasses the many steps in the entire process from identifying the scope of work to issuance of a final report,” the city said.

The city cited the language in the IRC’s recommendation that a “preliminary evaluation report” be done regarding the booster pump station.

“It is important to note that the recommendation made by the full IRC was to evaluate what would be needed to return it to service,” the city said. “City of Asheville staff are now working to implement the recommendation as suggested by the full Independent Review Committee.”

The city said its engineering staff conducted an analysis of the booster pump station, which it refers to as “the EABPS,” as recently as Sept. 11. Among other findings, it concluded:

“While it may have been true that in 1992 the EABPS was required to get additional flow from the two eastern water treatment plants into the system, assertions that in the present, without the pump station, our system is limited to 25 million gallons a day from the eastern water plants or our total safe yield is reduced by 6 million gallons a day is simply not factual.”

Holcombe takes issue with that. The booster station allowed for the expansion of service from the North Fork water plant from 25 million gallons a day to 31 million. Combined with the Bee Tree plant, this gives a combined capacity of 36 million gallons.

But that’s “only if it can be introduced into the transmission/distribution system at adequate pressure,” Holcombe said.

“When operational, the sole purpose of the EABPS was to take advantage of the full 36 million gallon-per-day treatment plant capacity from these water treatment plants,” he said. “Unless there has been an investment to improve the hydraulic capacity of the transmission lines between North Fork/Bee Tree and Haw Creek Junction, only through the operation of the EABPS can these volumes of water be delivered.

“City staff operated North Fork and Bee Tree during the outage and still could not refill the water storage reservoirs in South Buncombe and Candler Knob,” Holcombe said. “The EABPS operation would have prevented the storage reservoirs’ depletion.”

The city noted that the design of the pump station started in 1992.

“We are over 30 years removed from the water system conditions that were present and considered in the design of EABPS,” the water department said. 

The city has conducted “substantial capital investments” in the water system, with infrastructure improvements each year. The city pointed to a $40 million bond project dating to 2007, which in part included the installation of 47,000 feet of 24-inch transmission mains, replacing mostly 12-inch lines.

The scope of the recommended preliminary engineering report “should not be limited to just assessing what is needed to repair the pump station to its condition prior to its decommissioning,” the water department said. “To do so would be a massive oversight and borderline negligence, given the changes to the water system since 1992 when the pump station was designed.”

The water department “must consider needed pumping capacity for current and projected future conditions and pump size variation for increased operational control, which may make the pump station more useful than just in times of emergency.”

“We also must consider alternative locations of the pump station, as it may or may not be in the optimal location today, given the changes to the water system over the past 30 years since its original design,” the water department said.

Holcombe says the department’s call for a reassessment of the station’s location “points to a total lack of understanding of how crucial the pump station is in permitting North Fork to produce all the water that it was designed for.”

Holcombe noted that Hazen, in a 2019 analysis of the city’s system, “specifically identified the existing site as being the most cost-effective,” and the IRC agreed with that assessment. Tyree, the engineer with the Knoxville water utility, said Hazen’s assessment took into consideration that average daily demand is predicted to hit about 30 million gallons around 2028.

An assessment of the pump station system was conducted in 2009, the water department said.

“At that time, it was determined extensive repair or refurbishment would be needed to the system,” the water department said. “That would need to take place before even being able to enter into discussions about bringing [it] back online and the costs associated. 

In 2015, the city brought in Rockwell Automation to evaluate the station’s electronic systems, the city stated. Rockwell’s recommendations said: “These drives are 23 years old and have been obsolete for at least 14 years. Rockwell Automation no longer supplies parts for this product. These drives should be replaced with updated units that will enable parts and service for future years.” 

Also, Rockwell no longer had product engineers with knowledge of the system.

You have to ‘hydraulically jam’ water into the system

Holcombe, who earned a degree in business administration from Appalachian State University, started working for the city in 1978 as an account clerk in the accounting department. A year later, he became a planning and research director, a senior-level position assisting then-Water Director William DeBruhl. 

Holcombe became director of the water department in September 1993 and served in that role until September 1997. He stepped down as director for health reasons but stayed with the city to oversee the construction of the Mills River water facility. Holcombe left city employment in 1999.

Holcombe was integrally involved in the construction of the booster pump station. Located in a sizable brick building off Riceville Road, the East Asheville Booster Pump Station today sits behind a locked gate with a “No trespassing” sign.

In the early 1990s, Holcombe said, with system demand rising and the city operating two reservoirs and attached water plants, more capacity was needed. Mills River was not on the horizon then, and the pump station was the best answer.

North Fork had extra water, but it had to be pushed into the system.

“You can’t get that extra water into the system unless you hydraulically jam it in there during periods of high demand,” Holcombe said. “You have to have it, and the 6-million-gallon expansion would not have happened unless we had the pump station there to get it into the system.”

The Christmas outage was exactly what the pump station was designed for — high demand caused the pressure at Haw Creek junction to drop, which would have triggered the pump station to come on and raise the pressure at Haw Creek junction and would have kept the key water tanks full.

Had that happened, “the South Buncombe tank would have stayed half full,” Holcombe said. “The Candler Knob tanks would have declined by 2 feet. No one would have known that there was a problem at Mills River, and everyone would have had a very merry Christmas and a very happy new year.”

Asheville Watchdog is a nonprofit news team producing stories that matter to Asheville and Buncombe County. AJohn Boyle has been covering Asheville and surrounding communities since the 20th century. You can reach him at (828) 337-0941, or via email at jboyle@avlwatchdog.org. To show your support for this vital public service go to avlwatchdog.org/donate.