SEC18 Ebinder
SEC18 Ebinder
Conference
Instructed by:
Michael Sparks, Director, Office of Secuirty Assistance, U.S. Department of Energy
Mark Hojnacke, Physical Security Specialist, Office of Security Policy, U.S. Department of Energy
Carl Pocratsky, Engineer, Office of Security Assistance, U.S. Department of Energy
David Batz, Sr. Director, Cyber & Infrastructure Security, Edison Electric Institute
Sam Sharwarko, Physical Security Specialist, Western Area Power Administration
Laurie Williams, Sr. Manager, Operations & Planning Reliability Compliance, PNM Resources
Art Weise, Transmission & Distribution Manager, Nebraska Public Power District
Luke Wollin, Director, Transmission Design, Ameren
Chris Lyles, Asset Management Specialist, Western Area Power Administration
Ruben Robles, Manager, Critical Infrastructure Protection, SRP
Gilbert D. Flores, Emergency Management Specialist, Western Area Power Administration
Tiffani DeFore, Emergency Management Program Manager, Western Area Power Administration
RMEL ~ 6855 S. Havana, Ste 430 ~ Centennial, CO 80112 ~ (303) 865-5544 ~ FAX: (303) 865-5548 ~ www.RMEL.org
*Presentations are subject to change. Please visit
www.RMEL.org for the latest topic and speaker
information.
with key standards as well as transformers, mobile spare 8:15 a.m. - 9:00 a.m. Part 3.2, and perform active
how each Region conducts transformers, and the Organizational vulnerability assessments
its audit processes including overall quantity of spare preparedness for an for high impact Bulk Electric
the data request processes, equipment required. active threat incident System Cyber Systems
and other aspects of what
Gilbert D. Flores, Emergency by July 2018. Mr. Robles
registered entities can ex- Western Area Power
Management Specialist, will discuss the guidance
pect as part of these audits. Administration.
Western Area Power and expectation given by
Transformer Risk
Administration NERC. He will also inform
Noon - 1:00 p.m. Strategy for High
Tiffani DeFore, Emergency the audience of the results
Networking Lunch Impact Low Frequency
Management Program of research he performed to
Events
Manager, Western Area Power understand how utilities are
1:00 p.m. - 2:30 p.m. Panelist #3: Chris Lyles, Administration performing these assess-
Roundtable Asset Management An active threat incident is ments to meet the objective
Specialist, Western Area a scary and terrible situation of CIP-010.
Power Administration
2:30 p.m. - 2:45 p.m. that hopefully no organiza-
WAPA’s proposed spare
Networking Break
transformer strategy to
tion will have to experience. 9:45 a.m. - 10:00 a.m.
However, preparing your or- Networking Break
mitigate high impact, low ganization with training and
2:45 p.m. - 3:45 p.m. frequency events. This exercises can lessen or even
Panel: Sparing Strategies presentation has been 10:00 a.m. - 10:15 a.m.
prevent this type of violence
for Transformers delivered to WAPA’s Attendee Announcements
in the workplace.
transmission customers Any registered attendee
Sparing Strategy at to solicit feedback on a is invited to make a short
NPPD
9:00 a.m. - 9:45 a.m.
proposed strategy. announcement on their com-
Active Vulnerability pany, new products, tech-
Panelist #1: Art Wiese,
Assessments for High
Transmission & Distribution 3:45 p.m. - 4:30 p.m. Impact BES Cyber
nologies or informational
Manager, Nebraska Public Critical Infrastructure updates. Announcements
Systems may include showing a
Power District Protection Through
Ruben Robles, Manager, product sample but not vid-
This presentation will de- Defense-in-Depth
Critical Infrastructure eos and power point slides.
scribe the process NPPD Ruben Robles, Manager, Protection, SRP Please limit announcement
went through to determine Critical Infrastructure NERC Registered Entities to 5 minutes.
what spares we believe Protection, SRP are expected to comply with
we would need as a result Defense-in-depth focuses CIP-010 Requirement R3,
of a High Impact Low on layering controls in order 10:15 a.m. - 11:30 a.m.
Frequency (HILF) event. to increase the security of Roundtable
It will also cover NPPD’s a system. This presentation
philosophy for stocking or will detail the concept of
otherwise acquiring spares defense-in-depth and how Thank You RMEL Transmission
for a HILF event. the Critical Infrastructure
Protection Standards apply
Committee
Ameren Transmission defense-in-depth to protect
Sparing and Resiliency CHAIR Chris Koch
the Bulk Electric System.
Strategy Angela Piner Manager, Substation
Panelist #2: Luke Wollin, VP Engineering
Director, Transmission Wednesday, HDR, Inc. Kansas City Power & Light
Design, Ameren
The presentation will focus January 31, VICE CHAIR Keith Nix
on assessing the physical
threats and other potential
2018 Ana Bustamante
Director, T&D Engineering
VP, Technical Services and
System Reliability
risks to the transmission UNS Energy Corporation Texas New Mexico Power
system. We will discuss 8:00 a.m. - 8:15 a.m.
the security measures and Welcome Back and Scott Bayer Mike Pfeister
spare equipment neces- Opening Remarks Director, Transmission & Manager of Scheduling &
sary to provide resiliency Substation Engineering and Reliability Services
and properly mitigate Construction SRP
these risks. The discussion Austin Energy
will primarily concentrate John Quintana
on physical security at Jedd Fischer Transmission Asset
the substation and criti- Senior Project Manager Maintenance Manager
cal structures, hardened Nebraska Public Power Western Area Power
District Administration
Randy Harlas
Manager, Substation & Relay
El Paso Electric Company
Opening Presentation
Michael Sparks
Director, Office of Security Assistance
U.S. Department of Energy
Mark Hojnacke
Physical Security Specialist
Office of Security Policy
U.S. Department of Energy
Carl Pocratsky
Engineer, Office of Security Assistance
U.S. Department of Energy
SECURITY,
RISK,
AND
INVESTMENT DECISIONS
A DOE Office of Security Approach
For
Securing Power Marketing Administration Transmission Assets
Michael H. Sparks
Director, Office of Security Assistance
U.S. Department of Energy
January 30, 2018
The Office of Security Assistance
The Office of Security Assistance manages and integrates multiple programs to improve the
security posture of the Department of Energy and protect the Department’s assets, facilities and
personnel.
• The Department’s lead for the identification, evaluation, and modification of the
Departmental security risk management policies; the modification and deployment of
vulnerability assessment tools and processes needed to meet site security mission
requirements.
• The Department’s primary resource for developing security performance data for the
analysis and assessment of protection systems, weapons and equipment.
• The Department’s corporate resource for the assessment and integration of safeguards and
security technologies necessary for countering current and emerging threats while
systematically reducing operating costs, enhancing protective force safety and survivability,
and improving overall security effectiveness.
AU‐52 Physical Security Emphasis
Field Assistance... To Protect... Against... To Prevent…
Loss of Grid
Physical Damage Resilience
Copper
Copper
Threats Theft
Cyber Attacks
Vulnerabilities
Sustained
Weather Events Catastrophic
Outage
Military
Non-Violent
Assault
Protests
Consequences
Physical Attacks
(…Of Danger or Money) Geomagnetic Storm
Why Physical?
Quadrennial Energy Review Published April 2015
Figure 2‐3. Left figure: Electric Disturbance
Events, January 2011 – August 2014
6
Pay Attention to the Bottom Line
First known hacker-caused power outage signals troubling escalation
- Highly destructive malware creates "destructive events" at 3
Ukrainian substations.
by Dan Goodin - Jan 4, 2016 3:36pm EST
Ars Technica UK
SANS NewsBites Vol. 18 Num. 001 : Cyber Attack Takes Out Power in Ukraine;
The ICS Team at SANS has been researching this
one since Dec 24th (an unplanned Holiday challenge and it was not Ed
Skoudis - this we know). A big unknown remains: how the electric
service was actually disrupted? A file wiper function can certainly
disrupt the SCADA system, but that alone does not account for the
outage. The SSH capability is probably a "tell" here as we suspect an
attacker manually interacted with an infected machine, like an HMI
(human machine interface), to command breakers to open (just a theory
at this point). The wiper function could then have been used to extend
the outage by denying the SCADA system, but the impacted Ukrainian
utility was still capable of resorting to manual operations to (re-close
7
breakers) and energize their system.
Next:
David Batz
Sr. Director, Cyber & Infrastructure Security
Edison Electric Institute
How the IoT (Internet of Things)
Wants to Ruin Your Life and
Disrupt Implementation of New
Physical Security Solutions”
David Batz
Senior Director, Cyber & Infrastructure Security
Edison Electric Institute
[email protected]
3
(Potential) Adversaries
Script Kiddies
_ Business Network
Hacktivists
Irregular Actors
Disgruntled Insider
Nation State/State Sponsored
4
Why
5
Distributed Denial of Service
Attacks
6
July 2010 Stuxnet
“Isolated Network”
7
The Internet Arms Bazaar
8
Threats
9
Not Just Cyber
Power Grid Preparedness Falls Short, How Safe and Reliable is America’s
Report Says Electric Grid?
10
Components of Interest
11
Externalities – The Insider Threat
12
Cryptowall / Ransomware
13
Not Just Individuals
14
Externalities
OPM Breach
21.5
1.1
15
Ukraine December 2015
December 2016
What Happened
How did it happen
Could it happen again? Could it happen here?
16
Ukraine December 2016
Transmission Substation
A message?
17
CRASHOVERRIDE
18
Externalities
19
Major Cyber Events Ramping Up
Wanna Cry
- Shadow Brokers
- Eternal Blue
NotPetya
- Supply Chain
attack
Ransomware
-or- Wiper
20
Recent Events
21
Internet of Things (IoT)
1/1/2018
6.4B
5.5 M/day
2020
20.8B
22
Or Faster
23
IoT Devices
24
Architectural Failures
Meltdown
Spectre
25
Industry Leadership on Physical
Security
26
Cyber Mutual Assistance
is a program sponsored by the ESCC.
ESCC Purpose & Scope
Purpose: The ESCC is the principal liaison between
the electric sector and the federal
government for coordinating efforts to
prepare for, and respond to, national-level
disasters or threats to critical infrastructure.
Scope: The ESCC facilitates and supports policy and
public affairs-related activities and
initiatives designed to enhance the reliability
and resilience of the electric grid. The ESCC
is not operational.
Photo by Karla Marshall
32
33
Spare Transformer Equipment Program - 2006
SpareConnect – 2014
Transformer Transportation
SpareConnect offers an
online tool to communicate a
utility’s bulk power system
equipment needs and
provides point of contact
information for people and
equipment across the North
American electric utility
industry for specific equipment
categories and classes.
39
Utility, Asset Owner Response
40
DOE/DHS Electricity Subsector
Cybersecurity Capability Maturity Model
Approximately 20 companies (Investor Owned
Utilities, Coops and Munis) participated in the
pilot.
Domains in maturity model in which companies
are evaluated:
1. Asset, Change, and Configuration Management
2. Workforce Management
3. Identity and Access Management
4. Risk Management
5. Supply Chain and External Dependencies
Management
6. Threat and Vulnerability Management
7. Event and Incident Response, Continuity of
Operations
8. Situational Awareness
9. Information Sharing and Communications
10. Cybersecurity Program Management
41
NIST Cyber Security Framework
Core Functions
• Identify- Develop the organizational understanding to manage
cybersecurity risk to systems, assets, data, and capabilities
44
Responding
45
Guiding Principle
Sam Sharwarko
Security Specialist
Western Area Power Administration
Physical Intrusion Detection
Sam Sharwarko, CPP, PSP
January 30, 2018
RMEL Physical & Cyber Security Conference
Tempe, AZ
1
Western Area Power Administration
• 1 of 4 PMAs in the DOE
• Transmit and market wholesale power throughout
15 states in Western U.S.
• HQ in Denver; 5 Regions
• Sam Sharwarko, CPP, PSP
• Office of Security & EM
• Physical Security countermeasures
• Security integration and coordination
• BPA with Williams Electric Co.
Delay
Deter
Property Line
Semi-Public Semi-Private
Public Private
Intruder Authorized
Behaviors User
Behaviors
Behavior Parameter
9
Behavior Patterns
Probability
2 1
Intruder Authorized
Behaviors User
Behaviors
3
4
Behavior Parameter
10
Behavior Patterns
Intrusion
Detection
Probability
Strategy
Intruder Authorized
Behaviors User
Behaviors
Behavior Parameter
11
Detection Zones
12
Detection Technology
• Perimeter Disturbance
• Fence line
• Buried cable
• Visual
• Cameras
• People
• Volumetric
• RADAR
• MW
• IR
Intrusion Detection – RMEL Physical & Cyber Security Conference 13
Detection Zones
14
RADAR
18
Microwave & IR Beam
21
Interior Technologies
• Passive Motion Detection
• Detect energy
• PIR
Laurie Williams
Sr. Manager, Ops & Planning Reliability
Compliance
PNM Resources, Inc.
January
2018
2
PNM Resources Snapshot
New Mexico and Texas Service Territories
Registered Functions
Transmission
Transmission
Transmission
Transmission
Distribution
Generator
Generator
Balancing
Authority
Authority
Resource
Operator
Operator
Planning
Provider
Provider
Planner
Planner
Service
Owner
Owner
Entity
Name Region BA DP GO GOP PA RP TO TOP TP TSP
PNM WECC X X X X X X X X X X
TNMP TRE X X X X
3
Background on PNM
5
Audit Schedules
WECC TRE
• Notice of Audit Dates email: 5/6/16 • Notice of Audit Dates email: formal
• IRA Request: 5/6/16 Due: 8/20/16 4/27/17 (informal email 4/13/17 &
• Draft IRA Report: 11/28/16 via phone 4/12/17)
• Final IRA: 12/22/16* • Draft IRA Report: 3/17 and 4/16
• Notice of Compliance Audit: 2/7/17 • Final IRA : 4/11/17
• Pre-Audit Conference Call: 3/29/17 • Notice of Compliance Audit: 5/2/17
• Pre‐Audit Survey: 3/9/17 • Pre-Audit Conference Call: 4/12/17
• Audit Package: 4/7/17 • Pre‐Audit Survey/TOP Information
Request: 5/26/17
• First DR: 4/10/17
• Audit Package: 6/1/17
• Audit Dates: 5/8/17 – 5/19/17
• First DR: 6/26/17
• CIP and O&P Audit period: 2/19/14
– 2/7/17 • Audit Dates: 7/31/17-8/11/17
• Final Audit Report: 7/18/17 • CIP Audit Period: 2/26/14-10/11/17
• O&P Audit Period: 8/28/14-10/11/17
WECC formally publishes audit schedule w/ • Closing Meeting: 10/11/17
dates in late summer for full year following 6
PNM Audit Team
CIP AUDIT TEAM (11 total) OPS AND PLANNING AUDIT TEAM (7 total)
• John Graminski, Senior Auditor, Audit Team • Mike Wells, Senior Auditor, Audit Team Lead
Lead • Roger Cummins, Senior Auditor
• Morgan King, Senior Auditor • Jay Loock, Senior Auditor
• Gary King, Senior Auditor • Mark Christensen, Auditor
• Zach Trublood, Senior Auditor • Patrick Van Guilder, Senior Risk Engineer
• Lisa Wood, Auditor • Jim Terpening, WECC Consultant
• Eric Weston, Auditor • Mike Brock, WECC Consultant
• Carl Bench, Auditor
• Mark Lemery, Auditor FERC Observers
• Holly Eddy, Auditor • Bayard Koch, Electrical Engineer (did not
• Jennifer Salisbury, Associate Auditor attend)
• Stacia Carron, Associate Auditor • Robert Clark, Electrical Engineer
WECC Observers
• Darren Nielsen, Manager, CIP Audits NERC Observers
• Katie Iversen, Associate Engineer • Ryan Mauldin, Compliance Assurance Advisor
7
TNMP Audit Team
O&P (3 total)
• Frank Vick, Sr Compliance Analyst, Audit Team Leader-O&P Auditor since Jun 2005
– CenterPoint Senior Electrical Engineering Specialist in the Substation Engineering Division,
also relaying and planning experience
• Jens Steinborn, Compliance Engr III- O&P Auditor since Aug 2010
– Semi-conductor product development, certified System Operator in ERCOT,
• Michael Dillard, Compliance Analyst II- O&P Auditor since Feb 2015
– Navy nuclear plant operator, QA inspector, Operator Southern Company, C.o. Tallahassee,
NERC Certified RC
CIP (3 total)
• Kenath Carver, Compliance Team Lead- CIP Auditor since Feb 2012
– IT Business Solutions Analyst and Sr IT Security Administrator
• Benjamin Gregson, CIP Security Analyst I - CIP Auditor since Sept 2016
– System Admin Texas A&M System Turbomachinery Laboratory (5 yrs) &IT banking
environment
• Paul Hopson, CIP Security Analyst III - CIP Auditor since Aug 2016
– Information Security Officer and Senior Network Systems Engineer for the PUCT and TX AG
Office and the Texas Rehabilitation Commission as a LAN/WAN Network Manager
8
Audit Scope
PNM TNMP
2014
2014
• 32 O&P requirements • 19 O&P requirements
• 29 CIP requirements • 15 CIP requirements
• No findings by WECC beyond • No findings by TRE beyond
those self-reported by PNM those self-reported by TNMP
• Audit package -> • Audit package ->
– 564 O&P and 316 CIP – 167 O&P and 286 CIP
documents (955 MB) documents (335 MB)
• 59 Data Requests or “DRs” • 93 Data Requests
• 13 interviews • 12 interviews
• 5 Facility tours • 4 Facility tours
• 14 Technical Feasibility • No TFE reviews
Exception (TFE) Reviews • 4 CIP “live” demonstrations
– ~1.5 days
9
CIP Audit Scope
PNM TNMP
1. CIP-002-5.1 R1 16. CIP-007-6 R3 1. CIP-002-5.1a R1.
2. CIP-002-5.1 R2 17. CIP-007-6 R4 2. CIP-002-5.1a R2.
3. CIP-003-6 R1 18. CIP-007-6 R5 3. CIP-005-5 R1.
4. CIP-003-6 R3 19. CIP-008-5 R1 4. CIP-005-5 R2.
5. CIP-004-6 R1 20. CIP-009-6 R1 5. CIP-006-6 R1.
6. CIP-004-6 R2 21. CIP-010-2 R1 6. CIP-006-6 R2.
7. CIP-004-6 R3 22. CIP-010-2 R2 7. CIP-007-6 R1.
8. CIP-004-6 R4 23. CIP-010-2 R3 8. CIP-007-6 R2.
9. CIP-004-6 R5 24. CIP-011-2 R1 9. CIP-007-6 R3.
10. CIP-005-5 R1 25. CIP-011-2 R2 10. CIP-008-5 R1.
11. CIP-005-5 R2 26. CIP-014-2 R1 11. CIP-009-6 R1.
12. CIP-006-6 R1 27. CIP-014-2 R2 12. CIP-010-2 R1.
13. CIP-006-6 R2 28. CIP-014-2 R3 13. CIP-010-2 R2.
14. CIP-007-6 R1 29. CIP-006-6 R3 14. CIP-014-2 R1.
15. CIP-007-6 R2 15. CIP-014-2 R2.
10
O&P Audit Scope
PNM TNMP
1. COM-002-4 R1 17. PER-005-2 R5 1. COM-002-4 R1. 17. PRC-005-6 R3.
2. COM-002-4 R2 18. PER-005-2 R6 2. COM-002-4 R2. 18. PRC-023-4 R1.
3. COM-002-4 R3 19. PRC-001-1.1(ii) R3 3. COM-002-4 R4. 19. TOP-001-3 R9.
4. EOP-001-2.1b R3 20. PRC-001-1.1(ii) R5 4. EOP-004-3 R3.
5. EOP-005-2 R10 21. PRC-004-4(i) R1 5. EOP-008-1 R5.
6. EOP-005-2 R11 22. PRC-004-4(i) R2 6. EOP-008-1 R7.
7. EOP-008-1 R6 23. PRC-004-WECC-1 R1 7. FAC-008-3 R3.
8. FAC-003-4 R1 24. PRC-005-6 R1 8. FAC-008-3 R6.
9. FAC-003-4 R2 25. PRC-005-6 R3 9. FAC-008-3 R8.
10. FAC-003-4 R6 26. PRC-006-2 R9 10. IRO-017-1 R2.
11. FAC-003-4 R7 27. TOP-002-2.1b R4 11. PER-005-2 R3.
12. FAC-008-3 R6 28. TOP-002-2.1b R6 12. PRC-001-1.1(ii) R3.
13. FAC-014-2 R5 29. TOP-002-2.1b R11 13. PRC-004-5(i) R1.
14. FAC-014-2 R6 30. TOP-006-2 R2 14. PRC-004-5(i) R2.
15. FAC-501-WECC-1 R3 31. TOP-007-0 R1 15. PRC-004-5(i) R3.
16. PER-005-2 R3 32. VAR-002-4 R1 16. PRC-005-6 R1.
11
PNM Site Visits and Interviews
• Interviews
– CIP-002 R1 (1st DR), -004, -005, -006 R1, -007, -010
– FAC-003 and -008
– PER-002-5
– PRC-005 and FAC-501-WECC
• Site Visits
– Primary, Backup and Generation Dispatch Control Centers (CIP and
O&P)
– 2 ‘Medium’ Stations
• CIP-014 In-person “hand-off” – via hard copy
• TPL-001-4 informal Q&A session with SME
12
TNMP Site Visits and Interviews
• CIP “Live” Demonstrations • Interviews
– CIP-014-2 R1, R2
– CIP-005-5 R1 (Part 1.5), R2
– COM-002-4 R1, R2, R4
– CIP-007-6 R1, R2, R3, R4* – EOP-004-3 R3
– CIP-009-6 R1 (Parts 1.3 and 1.4) – EOP-008-1 R5, R7
– CIP-010-2 R1, R2 – FAC-008-3 R3, R6, R8
– IRO-017-1 R2
• Site Visits – PER-005-2 R3
– Primary and Backup Control – PRC-001-1.1 (ii) R3
Centers for CIP and O&P – PRC-004-5(i) R1, R2, R3
• CIP-014 In-person “hand-off” – PRC-005-6 R1, R3
– PRC-023-4 R1
via encrypted drive – TOP-001-3 R9
13
General Observations
WECC TRE
• O&P Audit team more • CIP Audit team more CIP
utility experience (due to technical background
large team size) • No final Audit report to
• Many newer CIP Auditors date
• More extensive IRA • >Administrative effort –
evaluation evidence inventory/
certifications
Often differing Audit Approaches
But teams reached same conclusions and were open
to discussion/explanation
14
General Observations Con’t.
17
CIP - TRE
• TRE - extensive “live” demonstrations
– Validated EAP firewall rules, Anti-virus configurations, Intrusion detection,
Remote access (no IRA), etc.
18
CIP – TRE, con’t.
• Requested laptop with CIP access
• Requested inventory of all low impact BES Cyber
Systems
• Significant volume to certain DRs – large sample
sizes
• Use of NP-view software- network path
connectivity and validation of access control lists
19
CIP - WECC
• CIP Interviews followed templates published by others
on WICF
• Less extensive “live” demonstrations
– More top-down approach to each standard
• Interviews included SMEs reiterating procedures “in
own words”
• ‘CIP Data Set’ and ‘CIP Request For Information’
differed from TRE/NERC version
• WECC walk-thru included workstation logins
– Ping corporate network from device w/I ESP to validate
network segregation and to ensure antivirus up-to-date
• Focused on validation of self-reported items
20
O&P - TRE
• TOP Information Request - not posted on TexasRE.org
– Narrative responses regarding CC displays and operations capabilities such as
voltage, load shed, real-time status, alarming, etc.
– Photos of comm capabilities, TOAPs, screen shots of available procedures
– Summary of TNMP’s disaster recovery and fail over plans / capability
– Delegation agreements
• ‘PRC-005 Spreadsheet’ – posted on TexasRE.org
• Comparison of evidence against prior/current ERCOT
documentation
– e.g. FAC-008 MLSE vs. the SSWG data from ERCOT
• Significant and in-depth DRs
– TRE indicated generally issue 150 CIP questions and another 150 O&P
on average
21
O&P - WECC
• O&P System Operator Interviews followed
templates published by WICF
– Additional event-related questions
• Protection Systems Maintenance Summary
spreadsheet – replaced ‘Attachment G’
• Requested explanations of FAC-008 changes
against prior (2014) audit materials
• Completed and reported results of on-site
validation of self-report scope and
remediation/mitigation
22
ERCOT Operating Guides and Protocols Audits
• TRE reviews compliance on behalf of Public Utility
Commission of Texas or “PUCT”
– PUCT handles enforcement at recommendations of TRE
– 90 calendar day notification and 45 calendar day submittal (August 22,
2014 notification for November 11, 2014 start date)
• Separate TRE group dedicated to ERCOT OG & P
• Not on NERC Audit cycles – undefined frequency
• Questionnaires rather than RSAWs
• Less formal and defined than NERC Audits but generally
follows enforcement process – self-reporting, mitigation
plans, etc.
23
Q&A
Laurie Williams
Senior Manager, Ops and Planning Reliability Compliance
Phone: 505-241-0641
[email protected]
24
PANEL: Sparing Strategies for
Transformers
Sparing Strategy at NPPD
Art Wiese
Transmission & Distribution Manager
Nebraska Public Power District
Transformer Spare Analysis
RMEL Meeting
January 30, 2018
Background
The electric power industry is coming under increasing
scrutiny/pressure to ensure the reliability of the grid under new
kinds of threats. Threats include both physical and cyber attacks,
or even naturally occurring events (severe storms, solar activity),
that could cause extensive damage. Examples are:
Recent attacks on the electrical system, including the rifle attacks disabling
transformers at Metcalf in California,
The use of cyber-warfare to disrupt the power supply in the Ukraine,
Super Storm Sandy highlighted the growing risk to the utility industry.
NPPD is considering stocking additional equipment to respond to
these events
2
Background
Spare Transformers In Stock:
345/115 kV
230/115kV
Additional Spare Transformer NPPD is Considering:
345/230 kV
3
Third Party Proposal
Members nominate what they need, and third party keeps on hand a
quantity that would meet any one members needs during a qualifying
event.
Here is an example:
Members nominated 2 identical transformers.
Third party determined it will store 1
Each member pays 1/2 of the carrying costs to hold 1 in
inventory.
This is like paying the carrying cost of 1/2 transformer
The material will have full vendor warranty upon delivery to member
Upon a “Qualifying Event”, members purchase the items at the original
cost, and are responsible for the delivery-to-site costs as well.
4
NPPD Position
Based on preliminary pricing, the annual cost to store this same
“nominated” inventory at NPPD would be the lower cost option
NPPD will continue to monitor industry activity to see if the
business case changes.
Other Considerations
To take advantage of Warranty, NPPD would likely install a new
transformer and placed a “used” transformer in inventory
5
Questions?
Ameren Transmission Sparing and
Resiliency Strategy
Luke Wollin
Director, Transmission Design
Ameren
...........................................................................................................................................................................................................................................................................................................................................................................................................................................................
...............................................................................
Luke Wollin, PE
Director, Transmission Design
Ameren Corporate
Fully rate-regulated electric and gas utility
• 2.4 million electric and 0.9 million gas customers
• 10,200 megawatts (MW) of regulated electric generation capability
• Over 7,900 miles of regulated electric transmission
• 2016 Ameren rate base: $13.4 B
Ameren Corporation
• Vertically integrated electric • Electric and gas delivery and • Invests in regional projects
generation, transmission and electric transmission • Regulated by FERC
delivery and gas delivery business
business • Serves 1.2 million electric
• Serves 1.2 million electric and 0.8 million gas
and 0.1 million gas customers
customers • Invests in local reliability
• 10,200 MW of total projects
generation capability • Regulated by ICC2
• Regulated by MoPSC1
2
...........................................................................................................................................................................................................................................................................................................................................................................................................................................................
...............................................................................
3
Substation Security Strategy
...........................................................................................................................................................................................................................................................................................................................................................................................................................................................
...............................................................................
• Maintain a 99% availability of EHV units due to manufacturing lead time for replacement EHV
transformers of 15-18 months
• Inventory of 4 extra high voltage (EHV/345 kV) spare transformers is currently maintained
– Spare transformers are strategically located at sites with convenient railroad access
– Spare for all voltage classes and MVA sizes
– A spare EHV transformer could be moved and installed in place of a failed or failing unit in 6 weeks
– Specialized railcar purchased for hauling EHV transformers across our system
• Member of EEI STEP, a sharing program for the catastrophic loss of multiple transformers
– Presidential declaration of emergency due to an act of terrorism required to trigger
• On October 2, 2017 joined the RESTORE transformer sharing agreement with 24 other utilities
– Triggered by a major event due to an act of terror, natural disaster, or other events creating an
urgent grid need
– Regional sharing agreement with utilities primarily located in the southeast
– Can be expanded to include other long lead items such as breakers and transmission line towers
• Power Circuit breakers and protection devices (relays) required to backup most
transmission substation devices are currently maintained.
– In an emergency, substation equipment would be removed from operation at
unaffected substation or taken from in-progress projects.
• Spare structures, conductor and other materials necessary to restore about 5 miles of
transmission line are maintained
– Lines have hardened storm structures every 5-10 miles to stop a cascading failure
– Rebuilding 5 line miles takes 3-4 weeks, which is the lead time on additional
structures/hardware
– In the event of a larger emergency, other utilities could supply material and manpower
• Special line towers and river crossings are not spared due to the large design variation
• Overall strategy is to identify and maintain spare equipment to mitigate the risk of extended
outages due to long lead times of certain high voltage equipment
• Long-term resiliency plan includes the following goals to mitigate potential risks
– Replace any piece of substation equipment in 6 weeks
– Restore load to metropolitan areas in 2 weeks after a coordinated physical attack. All
lines back in
– Replace equipment to operate the system normally within 6 months due to the total
loss of a substation
– Rebuild 5 miles of transmission line within 1 month
– Replace a key transmission structure within 6 months
• Replace aging transformers at the most critical locations with hardened units
– Transformer tank capable of withstanding a .50 caliber shot from 15’
– Radiators equipped with valves actuated if a bullet is detected
– Dry-type polymer bushings which do not fail catastrophically if punctured
• Design and develop spare transmission structures for special structures on critical lines
– Special long span structures (river crossings) take 12-18 months to design, procure and install
– Install physical security at key structures to detect and prevent physical attack
11
Western Area Power
Administration. Transformer Risk
Strategy for High Impact Low
Frequency Events
Chris Lyles
Asset Management Specialist
Western Area Power Administration
Transformer Risk Strategy
High Impact Low Frequency Events
Subject, Office or event
Topics
Overview of WAPA
Transformer Spare Need
Objectives
High Impact Low Frequency Events (HILF) and Quantities
Alternatives for Evaluation
WAPA Stored Inventory
WAPA In‐Service Spares
Grid Assurance
Project Timeline
Subject, Office or event 2
Overview of WAPA
• Part of the Department of Energy. One of four
Power Marketing Administrations (PMAs)
• 15 State Footprint
• 1400 employees. Four regions, one HQ
• Over 700 wholesale customers
• 17,000 miles of high voltage transmission
• 319 Substations
• Approximately 440 large power transformers
Subject, Office or event 3
Operational Spares Vs. HILF Spares
Subject, Office or event 4
Why are Spare Transformers Needed?
WAPA has a reliability requirement to its customers, and the bulk
electric system, to be prepared for the loss of transformers.
The unanticipated loss of a transformer in the bulk electric system
directly impacts the resiliency/reliability of the system.
Large Power Transformers typically have an acquisition lead time
of 18‐24 months.
Unique characteristics of transformers require custom build.
Transformer can weigh up to 100 tons and contain 25,000 gallons of oil.
Transportation requires multiple specialized vehicles, road permits, and task
specific crews.
Subject, Office or event 5
Typical WAPA Large Power Transformer
Subject, Office or event 6
Specialized Transportation
Subject, Office or event 7
Analysis Objectives
Develop and analyze alternatives for response to high impact low
frequency events resulting in loss of multiple transformers.
Solicit feedback from WAPA subject matter experts (SMEs) and
WAPA Customers.
Develop final recommendation incorporating customer feedback
and regional expertise.
Analysis and subsequent report is a WAPA‐wide strategy intended
to produce a recommendation that most efficiently uses WAPA’s
entire fleet of transformer options.
Subject, Office or event 8
Event Definition
WAPA Analysis
Define three high impact low frequency event types.
Assume loss of all large power transformers within impacted site.
Local Event
Event center at WAPA facility.
Impacted sites are within 5 mile radius of center.
All WAPA facilities could be center of event.
Seismic Event
Event center at Tracy or Mead facility.
Impacted sites are within 80 mile radius of center.
Targeted Event
Event center at major population centers and military installations – 9 sites identified.
Impacted sites are within 50 mile radius of center.
Subject, Office or event 9
Spares Needed
Quantities
Why Now?
Threat of physical, calculated attacks on electrical infrastructure.
Potential vulnerabilities in the electrical utility industry have been identified.
Power systems are being operated closer to their operating limits.
Subject, Office or event 10
Alternatives Currently Under Evaluation
WAPA Warehoused Alternative
WAPA System Enhancement Alternative
Grid Assurance Alternative
Hybrid Alternative
No Action – Status Quo
Subject, Office or event 11
Project Timeline
Develop draft transformer strategy analysis for HILF
events
Present strategy to customer groups and solicit
feedback (stakeholder outreach)
• Modify strategy based on feedback and submit to WAPA
senior managers
• Finalize strategy, recommendations, and develop funding
proposal
• Final stakeholder outreach and strategy execution
Subject, Office or event 12
Questions???
Chris Lyles
Asset Management Specialist
[email protected]
720‐962‐7249
Subject, Office or event
Critical Infrastructure Protection
Through Defense-in-Depth
Ruben Robles
Manager, Critical Infrastructure Protection
SRP
NERC CIP AND DEFENSE-
IN-DEPTH
Ruben Robles – January 30, 2018
Presenter’s Bio
Ruben Robles, Manager – SRP ERC CIP Team
[email protected]
602-236-8910
@RubenRobles18
https://www.linkedin.com/in/roblesruben/
Bachelor of Science in Electrical Engineering
Master of Science in Information Management
Master of Science in Cyber Security and
Information Assurance
Arizona Professional Engineer Registration
Detect
React
Balanced focus
People
Technology
Operations
RMEL, 01/30/2018 CIP Through Defense-in-Depth, R. Robles
Defense-in-Depth - DHS
Assumption
No ”silver bullet” to solve cybersecurity vulnerabilities
Multiple layers of defense mechanisms
Protect
Detect
Respond
Personnel
Technologies
RMEL, 01/30/2018 CIP Through Defense-in-Depth, R. Robles
Defense-in-Depth - SANS
Assumption
No single measure can adequately protect a network
Protecting through a series of defense mechanisms
Effective
security plan
Layers of defense:
Operations
Personnel
Technologies
R5 Access Revocation
R3 Vulnerability assessments
Gilbert D. Flores
Emergency Management Specialist
Western Area Power Administration
Tiffani DeFore
Emergency Management Program Manager
Western Area Power Administration
Organizational preparedness for
an active threat incident
Tiffani DeFore & Gilbert Flores
WAPA Emergency Management Program
Who
What
Why
Where
Ruben Robles
Manager, Critical Infrastructure Protection
SRP
Electric Reliability Compliance
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 1
Presenter Bio
Electric Reliability Compliance
About Me
• Ruben Robles, Manager – SRP ERC CIP Team
• [email protected]
• 602-236-8910
• @RubenRobles18
• https://www.linkedin.com/in/roblesruben/
• Bachelor of Science in Electrical Engineering
• Master of Science in Information Management
• Master of Science in Cyber Security and Information Assurance
• Arizona Professional Engineer Registration
• 11yrs in Telecommunications
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R 2
Robles, 01/30/18
Objectives
Electric Reliability Compliance
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 3
About Salt River Project
Electric Reliability Compliance
Statistics
Authorized under the National Reclamation Act
Serving Central Arizona since 1903
Provide generation, transmission, and distribution services
Nation’s third-largest public power utility
BA, DP, GO, GOP, PA/PC, RP, TO, TOP, TP, & TSP within WECC
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 6
CIP-010 R3
Electric Reliability Compliance
NERC Guidance
• “…models the baseline configuration of the BES Cyber
System”
– Guidance and Technical Basis
• “…the requirement is to ‘model’ the baseline configuration and
not duplicate it exactly.”
• “…BES Cyber System at a Control Center to be modeled that may
not otherwise be able to be replicated or duplicated exactly.”
– Red Seal
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 7
Active Vulnerability Assessment
Electric Reliability Compliance
Plan of Action
• Determine purpose
• Determine objective
• Determine capabilities
• Determine resources
• Create sustainable processes
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 8
CIP-010 R3
Electric Reliability Compliance
NERC Guidance
• Active Vulnerability Assessment
– Network Discovery
– Network Port and Service Identification
– Vulnerability Scanning
– Wireless Scanning
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 9
Active Vulnerability Assessment
Electric Reliability Compliance
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 10
Active Vulnerability Assessment
Electric Reliability Compliance
Vulnerability Scanning
• CIP-010 R1
– Document logical network accessible ports for the baseline
• Tools
– Tenable – Nessus
– Rapid7 – Nexpose
– Tripwire – IP360
Courtesy: https://www.gartner.com/reviews/market/vulnerability-assessment
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 13
Active Vulnerability Assessment
Electric Reliability Compliance
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 18
Active Vulnerability Assessment
Electric Reliability Compliance
Determine Objective
• How do you address the new device/network?
• Possible objectives
– Ensure the signal strength is weaker
– Identify the device/SSID/BSSID
– Ensure the device is not on the BCS
– Ensure the device complies with org’s policy
– Ensure SSID is not broadcast
– Update documentation
– Confirm wireless isn’t being used for the BCS
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 19
Active Vulnerability Assessment
Electric Reliability Compliance
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 22
Active Vulnerability Assessment
Electric Reliability Compliance
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 23
Active Vulnerability Assessment
Electric Reliability Compliance
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 25
Active Vulnerability Assessment
Electric Reliability Compliance
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 26
Active Vulnerability Assessment
Electric Reliability Compliance
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 27
Active Vulnerability Assessment
Electric Reliability Compliance
• Netsurveyor
• Kismet
• Acrylic
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 29
RMEL, Active Vulnerability Assessments for High Impact BES Cyber Systems, R Robles, 01/30/18 30