Who We Are Republican Views Newsroom Documents Archives Subcommittees Search the site Home

Prepared Witness Testimony

The House Committee on Energy and Commerce

 

Blackout 2003: How Did It Happen and Why?

Full Committee on Energy and Commerce
September 3, 2003
10:00 AM
2123 Rayburn House Office Building 

 

Mr. Michehl R. Gent
President
North American Electric Reliability Council
Princeton Forrestal Village
116-390 Village Boulevard
Princeton, NJ, 08540-5731

    Good afternoon Mr. Chairman and members of the Committee. My name is Michehl Gent and I am President and Chief Executive Officer of the North American Electric Reliability Council (NERC).

NERC is a not-for-profit organization formed after the Northeast blackout in 1965 to promote the reliability of the bulk electric systems that serve North America. NERC works with all segments of the electric industry as well as electricity consumers and regulators to set and encourage compliance with rules for the planning and operation of reliable electric systems. NERC comprises ten Regional Reliability Councils that account for virtually all the electricity supplied in the United States, Canada, and a portion of Baja California Norte, Mexico.

NERC is uniquely qualified to set standards for the reliable operation of North America's high voltage, interconnected grid system, and we hope soon to be able to enforce those standards. We are also uniquely qualified to assist the U.S. Department of Energy ("DOE") and the U.S.-Canada Joint Task Force on the Power Outage in investigating the August 14, 2003 blackout that encompassed parts of the upper Midwest and Northeast United States and eastern Canada.

NERC is governed by a board of ten independent trustees and brings together the best electrical system technical expertise available in the world. We are an international organization, integrating reliability across North America's electricity grids. In short, our mission is bulk power system reliability ¾ it's what we do.

As a standing procedure, NERC reviews and reports on disturbances that occur on the bulk electric systems in North America. As the entity responsible for reliability standards for the bulk electric system, NERC must understand and communicate to its members what happened on August 14 and why it happened. NERC must also determine whether any of its standards were violated and whether its standards and procedures require modifications to take into account the ways in which the bulk electric system is being used. Finally, NERC must assure that measures necessary to avoid a recurrence of the August 14 outage are taken.

Immediately after the onset of the blackout on August 14, 2003, NERC began assembling a team of the best technical experts in North America to investigate exactly what happened and why. Every human and data resource we have requested of the industry has been provided, and experts covering every aspect of the problem have been volunteered from across the United States and Canada. Shortly after the investigation began, representatives of DOE and the Federal Energy Regulatory Commission ("FERC") joined the investigative effort. The investigative team has numbered between 15 and 30 individuals from day to day, and all members of the team, regardless of their affiliation, have worked side by side to help correlate and understand the massive amounts of data that are being received.

To lead the NERC effort, we established a strong steering group of the industry's best, executive-level experts from systems not directly involved in the cascading grid failure. The steering group scope and members are described in Attachment A.

NERC and DOE representatives, including people from the Consortium for Electric Reliability Technology Solutions ("CERTS"), have been jointly conducting the fact-finding investigation of the events leading up to the August 14th blackout. We expect to have representatives of provincial and federal agencies from Canada join the investigative team shortly. The investigation is ongoing, and no causal conclusions can yet be drawn. DOE is a part of the United States-Canada Joint Task Force on the Power Outage. NERC has provided its information to DOE in support of the Joint Task Force effort. DOE has requested, and NERC has agreed, that DOE, as a member of that Joint Task Force, coordinate release of that information.

NERC and DOE collaborated on the data request that NERC issued on August 22, 2003, to those organizations who were directly involved in the August 14 outage, as well as surrounding systems. DOE and NERC are jointly developing a data warehouse to manage the thousands of data records being submitted in response to that request and all subsequent data requests. DOE and NERC also co-hosted a meeting of the major entities involved in the outage to help focus the investigation and begin to develop an understanding of the events that led to the outage; we expect to co-host additional fact-finding meetings in the future.

Understanding exactly what happened and why is an enormously complex task requiring a methodical investigation by experts from many disciplines. Analyzing what happened and why it happened has both a technical side and a people side.

The technical side begins with a reconstruction of what happened on the electrical system, within fractions of a second. The investigative team has already received many thousands of data records from control center event logs, disturbance recorders, and other system data that must be pieced together one at a time to understand how the power system broke apart and cascaded into a blackout. Unlike an airplane that has a single "black box," the power grid has thousands of event and disturbance recorders that measure events at critical points on the system. Each event, which might be a relay or circuit breaker operation, or an electrical fault, is "time stamped" as it occurs. However, we discovered that many of these time stamps were not accurate because the computers that recorded the information became backlogged, or the clocks from which the time stamps were derived had not been calibrated to the national time standard. As our data analysis progressed, we have been able to confirm those events that were accurately time-stamped, and from those events, we are in the process of aligning the event data for each system event from multiple sources until we are confident we have the precise time for each event.

I assure you this painstaking effort to synchronize event data down to fractions of a second is not an academic exercise. Most of the electrical operations in the system failure on August 14 occurred automatically over a very short period of minutes and seconds. Without such a deliberate, methodical reconstruction of events, it would be impossible to determine the exact sequence, and therefore the cause of the cascading failure and how it propagated to result in the ultimate blackout condition.

To ensure that the investigation is complete, NERC and DOE have requested data from the affected organizations starting at 8:00 AM EDT on August 14. This data will enable the investigators to form a clear picture of how that day started and what events through the course of the day may have contributed to or set the stage for events later in the day. Because that data is still being accumulated and has not been evaluated, it is too soon to determine whether events earlier in the day may have contributed to the outage.

To complete the technical investigation of "what" happened, we must also construct electrical models to simulate the exact conditions of August 14 and then subject those models to the events that occurred during the time preceding the outage to understand better its causes. These simulations will examine the electrical stability of the grid - that is, how strongly the generators were synchronized to one another - and whether there was a voltage collapse of the transmission system. We will also focus on why operating procedures that should have detected problems that developed on the grid and kept them from spreading did not prevent the cascading outage across such a wide area.

Preparing these simulations is a complex task requiring the reconciliation of power system data snapshots from multiple data recorders on August 14. I am confident that the investigation, when completed, will allow us to describe exactly what happened to the power system and why it failed.

The investigation also includes a "people" aspect. Working jointly with DOE as part of the U.S.-Canada Joint Task Force, we will be seeking to discover such things as: What were system operators and reliability coordinators doing leading up to the blackout? What indications of problems did they see or not see? What were their qualifications and training to recognize and respond to system emergencies? Did they follow established NERC and regional reliability standards and procedures? Were those standards and procedures effective? Were responsibilities clearly assigned and did operating personnel have the necessary authority to act in a timely manner to avoid the blackout? How effective were the control center computers and displays in providing information to the operators? What communications took place among system operators and reliability coordinators in different parts of the grid prior to and during the outage?

After determining what happened on August 14th, the investigation will analyze the root causes of the cascading failure ¾ looking once again at both technical and human factors. From the root cause analysis, we expect to develop a clear set of recommendations to ensure that our system operators, equipment, and reliability standards will successfully handle the kinds of events that led to the blackout.

It is too soon to identify specific equipment, measures, and procedures that worked as intended on August 14, but large parts of the Eastern Interconnection did not suffer the blackout. (Attachment B to my testimony is a map showing the Eastern, Western, and ERCOT Interconnections.) Protective relays within the distressed area operated to remove transmission lines, transformers, and generating units from service before they suffered physical damage. The system is designed to do that. It was the action of those individual relays, operating to protect individual pieces of equipment, that eventually isolated the portion of the grid that collapsed from the remainder of the Eastern Interconnection. The fact that the transmission lines, transformers, and generating units did not suffer physical damage is what made it possible to restore the system and service to customers as quickly as happened.

Another factor in the successful restoration was the restoration plans themselves. Restoring a system from a blackout requires a very careful choreography of re-energizing transmission lines from generators that were still on line inside the blacked-out area as well as from systems from outside the blacked-out area, restoring station power to the off-line generating units so that they can be restarted, synchronizing those generators to the Interconnection, and then constantly balancing generation and demand as additional generating units and additional customer demands are restored to service.

We will learn many additional lessons from this event that will enable us to improve the overall reliability of the grid. We can also build on some of the positives from this event, such as the extraordinary efforts to quickly put the system back on line and restore electric service to consumers.

I will close with one final point ¾ the need to establish mandatory, enforceable reliability standards. NERC has developed a world-class set of planning and operating standards, and I expect we will find areas of those standards that need improvement based on the events of August 14. However, as long as compliance with these standards remains voluntary, we will fall short of providing the greatest possible assurance of reliability that could be achieved through mandatory verification of compliance and the ability to impose penalties and sanctions for non-compliance.

Apart from the particulars of the August 14th outage and without knowing whether or not violations of our reliability standards occurred, one important step Congress can and should take to strengthen the reliability of the bulk power system in general would be to pass legislation to make the reliability rules mandatory and enforceable. NERC and a broad coalition of industry, government, and customer groups have been supporting legislation that would authorize creation of an industry-led self-regulatory organization, subject to oversight by FERC within the United States, to set and enforce reliability rules for the bulk power system. The comprehensive energy bills that have passed both the House and the Senate have versions of that reliability legislation. NERC looks forward to working with the conference committee to achieve passage of that legislation this year.

NERC is fully committed to finding out what happened on August 14, why it happened, and to see that steps are taken to prevent a reoccurrence. We are committed to supporting the U.S.-Canada Task Force in fully disclosing all the facts, the reasons for the cascading failure, and recommendations that will make the electricity grids in North America more reliable.

Thank you.


 

 

Attachment

 

August 14, 2003 Blackout Investigation NERC Steering Group

SCOPE August 27, 2003

Scope

The NERC Steering Group steers the formulation and implementation of the NERC blackout investigation plan, reviews the milestone progress and results, and recommends improvements. The Steering Group provides a perspective of industry experts in power system planning, design, and operation.

Members

The members of the NERC Steering Group are:

Paul F. Barber, Facilitator Barber Energy

W. Terry Boston Executive Vice President Transmission/Power Supply Group Tennessee Valley Authority

Mark Fidrych Power Operations Specialist Western Area Power Administration

Sam R. Jones Chief Operating Officer Electric Reliability Council of Texas

Yakout Mansour Senior Vice President System Operations & Asset Management British Columbia Transmission Corporation

William (Bill) K. Newman Senior Vice President Transmission Planning and Operations Southern Company Services, Inc.

Terry M. Winter President and Chief Executive Officer California ISO

M. Dale McMaster Executive Vice President-Operations and Reliability Alberta Electric System Operator

Biographies

Paul F. Barber, Ph.D. Barber Energy Dr. Barber provides transmission and engineering services to the electric power industry in areas of governance, strategic planning, electric grid management, and power system reliability. He previously served as the Chair of the NERC Market Interface Committee and as the Vice Chair (Transmission Customers) of the Northeast Power Coordinating Council (NPCC). Dr. Barber joined Boston-based Citizens Power & Light, providing transmission and engineering technical expertise and support to all business lines of Citizens Power & Light and its successors. Dr. Barber served on the NERC Board of Trustees as well as the Boards of the Mid-Atlantic Area Council, Western Systems Coordinating Council (WSCC), and the three Regional Transmission Associations in the Western Interconnection. Prior to 1994, Dr. Barber served a 28-year career as an officer in the U.S. Army Corps of Engineers rising to the rank of Colonel. Dr. Barber received his BS degree from the U.S. Military Academy and MS degrees in electrical engineering and civil engineering from the University of Illinois. He completed a Ph.D. degree in electric power engineering from Rensselaer Polytechnic Institute in 1988. He has been registered in the State of Illinois as a professional engineer since 1974.

W. Terry Boston Executive Vice President, Transmission/Power Supply Group Tennessee Valley Authority Terry Boston is Executive Vice President of the Tennessee Valley Authority's Transmission/Power Supply Group. Mr. Boston is the senior officer responsible for the planning, building, operation, and maintenance of TVA's transmission and power supply network. He joined TVA as a power supply engineer in 1972, and was named head of the Power Supply Group in 1980. Over the next 16 years, he directed three TVA divisions in succession: Transmission, Regional Operations, and Electric System Reliability. Mr. Boston has served for six years on the NERC Engineering Committee and Transmission Task Force, and is on the NERC Stakeholders Committee. He is vice president of CIGRE, the International Council on Large Electric Systems, and vice president of CERTS (the Consortium for Electric Reliability Technology Solutions). Boston holds a B.S. in engineering from Tennessee Technological University and an M.S. in engineering administration from the University of Tennessee.

Mark Fidrych Power Operations Specialist Western Area Power Administration Mark E. Fidrych has served as the Manager of Western Area Power Administration's Rocky Mountain Desert Southwest Reliability Center. Mr. Fidrych began his career with WAPA in 1979, working in maintenance and marketing, with the majority of his career having been in power system operations. He directed activities in the computer systems and power scheduling divisions before becoming the Operations Manager in 1990. A 1972 graduate of the University of Rhode Island, Mr. Fidrych received a bachelor's degree in electrical engineering. In 1980, he received a master's degree in public administration from the University of Colorado. Mr. Fidrych is the present Chair of the NERC Operating Committee. He has also served as the Chair of the NERC Security Coordinator and the Operating Reliability Subcommittees.

Sam R. Jones Chief Operating Officer Electric Reliability Council of Texas Sam R. Jones became the first Director of the Electric Reliability Council of Texas (ERCOT) on December 1, 1996. In March 2000, he was appointed as the Executive Vice President and Chief Operating Officer of ERCOT. Prior to joining ERCOT, Mr. Jones was employed by the City of Austin, Texas, Electric Utility for over 35 years. With the City of Austin, he held engineering and management positions in the areas of distribution, transmission, substation, generation and system operations. He was responsible for the development of Austin's first energy control center. He retired from the City of Austin as Director of Generation and Energy Control. He has been active in inter-utility reliability work for over 19 years. He is a two-time past chair of the ERCOT Operating Subcommittee, and a current Vice-Chair of the NERC Operating Committee, and a past chair (or member) of numerous NERC and ERCOT subcommittees and task forces. Mr. Jones has a degree in Electrical Engineering from the University of Texas at Austin and is a Registered Professional Engineer in Texas.

Yakout Mansour Senior Vice President, System Operations & Asset Management British Columbia Transmission Corporation Yakout Mansour is Senior Vice President of System Operations & Asset Management of the British Columbia Transmission Corporation. Previously, he served as the Vice President of the Grid Operations and Inter-Utility Affairs division of BC Hydro, responsible for BC Hydro's transmission, distribution and generation dispatch operations as well as the development of policies and practices related to inter-utility transmission access. Mr. Mansour currently serves as BC Hydro's principal representative and board member on the RTO West filing utilities structure and has been the Canadian representative in the RTO consultation process. Mr. Mansour is a registered Professional Engineer in the Provinces of British Columbia and Alberta with over 30 years experience in power system planning, system and market operation, design and research. He is a Fellow of IEEE, has authored and co-authored over 100 papers and special publications of IEEE and other international professional institutions, has provided training and consulting services around the world, and holds U.S. and Canadian patents. Dale McMaster, P.Eng. Executive Vice-President, Operations and Reliability Alberta Electric System Operator (AESO) Dale McMaster is Executive Vice-President, Operations and Reliability for the Alberta Electric System Operator (AESO). The AESO integrates the functions of the Power Pool of Alberta, the Transmission Administrator of Alberta, and provincial load settlement. Mr. McMaster's knowledge of system planning and his overall industry experience integrates the AESO's operational and planning areas. As President and System Controller, Mr. McMaster played a key role during the integration of the former Power Pool and the Transmission Administration. Mr. McMaster joined the former Power Pool of Alberta in 1996 as Chief Operations Officer, with responsibility for the system control function, the ongoing development of the Alberta electric energy market, and strategic planning. He is an electrical engineer with more than 25 years of experience in power systems in Canada and abroad. Mr. McMaster received his degree in electrical engineering from the University of Saskatchewan and held a variety of senior management positions at SaskPower, SNC-Lavalin, and Acres International. He is a member of the Association of Professional Engineers, and the Canadian Electricity Association.

William K. Newman Senior Vice President, Transmission Planning & Operations Southern Company William K. Newman began his career with Georgia Power Company in 1966 and progressed through positions of increasing responsibility at Georgia Power for 18 years. In 1984, he assumed the position of General Manager, Power Operations, at Mississippi Power Company, was promoted to Director of Power Delivery in 1988 and named Vice President, Power Generation and Delivery, in 1989. His responsibilities at Mississippi Power Company included the areas of fuels, environmental, generating plants, transmission, and system operations. He transferred to Southern Company Services in 1992 as Vice President, Operating and Planning Services and was named Senior Vice President, Transmission Planning and Operations in 1995. He is responsible for planning and operation of the Southern electric system's network transmission grid in order to provide economic, reliable service to all users. Mr. Newman has served in numerous academic and professional organizations and is currently Chairman, Southeastern Electric Reliability Council. He is a Registered Professional Engineer in the states of Georgia and Mississippi.

Terry M. Winter, President and Chief Executive Officer California ISO Terry M. Winter is President and Chief Executive Officer of the California Independent System Operator (ISO), a position he has held since March 1, 1999. Mr. Winter was formerly Chief Operating Officer of the California ISO, having accepted the position in August 1997. He assisted in developing operations from the ground up and oversaw the integration of the transmission systems of Southern California Edison, Pacific Gas & Electric, and San Diego Gas & Electric when the California ISO assumed control of the state's open market transmission grid on March 31, 1998. Mr. Winter was formerly the Division Manager of San Diego Gas & Electric's power operations. His 21-year career with SDG&E focused on power operations, transmission engineering and project management. Prior to his tenure with SDG&E he worked on electrical transmission and distribution engineering for Arizona's Salt River Project for 10 years. Mr. Winter holds professional engineering licenses in both California and Arizona. Mr. Winter graduated from the University of Idaho with a Bachelor of Science degree in Electrical Engineering.


Attachment B

 

Printer Friendly

Tipline: Report Waste, Fraude, and Abuse
Majority Site