nyiso excellence in execution: a report on results filenyiso excellence in execution: a report on...
TRANSCRIPT
NYISO Excellence in Execution:
A Report on Results
Management Committee MeetingAugust 26, 2009
Wayne BaileyVice President & Chief Compliance OfficerNew York Independent System Operator
2
Overview• In 2006, the NYISO undertook a series of Excellence in Execution initiatives to improve its performance and reduce errors. Steps included:
Information Technology Infrastructure UpgradesContinuous Process Improvement (LSS)Process & Controls DevelopmentOrganizational Enhancements
• Multi-year data show improvements across the enterprise.• Initiatives are now part of the NYISO’s on-going business practices.
3
What Do the Data Show?
Data indicate errors are down and performance is improved in an array of areas:
SettlementsPrice CorrectionsIT DeploymentsMarket System AvailabilityTariff WaiversCustomer Inquiry Responses
4
Settlement Process Data
Reduced time to process bills & improved accuracy of billing system
Reduction in Active Billing Issues
0
5
10
15
20
25
30
Mar-06 Jul-06 Nov-06 Mar-07 Jul-07 Nov-07 Mar-08 Jul-08 Nov-08
Monthly Six-Month Moving Average
Reduction in Open Billing Months
05
101520253035
Mar-06 Jul-06 Nov-06 Mar-07 Jul-07 Nov-07 Mar-08 Jul-08 Nov-08
Quantity Six-Month Moving Average
5
Thunderstorm Alert Data
Problem In 2006, 53 Thunderstorm Alerts (TSA) were initiated, resulting in charges of $27,871,363 that Market Participants were unable to verify. TSA charges did not appear on a Market Participant’s invoice until the 4-month true up. For 2006, Market Participant’s were charged $1,474,496 in interest at the 4-month true-up.
ImprovementsCycle time of the TSA calculations improved to ensure that a greater number of TSA events were reflected on the Initial invoice.Efficiencies gained by eliminating non-value added steps and interactions between NYISO and LECG.From April 2008 through April 2009, there were 42 TSA events resulting in charges of $54,282,612.
All 42 events appeared on the initial invoice.The new process saved Market Participants $1,849,707 in interest charges.
6
Price Correction Data
Automation of the manual process and process improvements of price corrections are improving the accuracy of pricing and the timeliness of the correction processMonth
Perc
enta
ge
DecNovOctSepAugJulJunMayAprMarFebJan
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
2006200720082009
Variable
Percentage of hours corrected year-to-date
7
Software Deployment & System Availability Data
0
20
40
60
80
100
120
2005 2006 2007 2008 2009
System Availability Deployment Defects
Rate of Change Deployments2005 - 5512006 - 6002007 - 6102008 - 6252009 - 532 (through 7/31/09)
8
DSS Availability DataProblem • The Data mart was not
consistently meeting the SLA of availability at 5 a.m. MP’s expect to start running their CSV files at 5am.
Our Service Level Agreement (SLA) for DSS availability states that DSS will be unavailable each night from 2AM to 5AM while we load new data from the previous day. At the start of this project in the Spring of 2007 we were hitting this SLA 67% of the time. The mean available time was 5:50 AM ranging from 4:26AM to 8:17PM. When the available time for DSS is delayed past 5AM MP's cannot access either their CSV files, which includes their daily bill, or run any other reports
ImprovementsWith a redesign of our nightly load process this project successfully achieved its target improvement of 90% availability by 5AM in September 2007. We have continued to build on this success and for 2009 YTD (through 8/10/09) we are hitting our SLA 98.2% of the time. The mean available time is 3:15AM ranging from 2:47 AM to 5:45AM.
9
Tariff Waiver Data
0
1
2
3
4
5
6
7
8
pre 2005 2005 2006 2007 2008 2009
Error Made Error Found
Within the last 18 months, the NYISO has filed a number of tariff waivers
Some were errors in implementationIn other cases, tariff language needed to match intent
The vast majority of these waivers are to address errors that were made prior to the impact of Excellence in Execution initiativesNYISO has addressed errors with process and control improvements in each case
10
Customer Inquiry Responsiveness Data
Through process enhancements and organizational changes, the timeliness of customer inquiry response has consistently improved
Time period for responding / engaging SME support to Customer Inquiries
1.31% 1.43%
87.21%
8.88%2.22% 1.69%
93.99%
3.26%
97.17%
1.79% 0.51% 0.53%0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
120.00%
1 2 3 >3
Days
200720082009 YTD
11
Steps Taken to Improve Execution
Settlement System Replacement (SSR) system --deployed in 2007Enhanced Price Validation Application -- deployed in 2007Quality Assurance Improvements
Improved testing process by providing multiple layers of testing up to and including a full test cycle utilizing a production quality environment exercising full regression testing, load testing and performance testingQA Test environments built out from one integrated system to multiple functional test systems, one fully integrated test system and one fully integrated Staging System that mirrors production with full failover and redundancy capabilities
12
Production ReleaseMost Production releases impacting Market Participants were moved to "off hour" deploymentsStructured Software Release Planning was established to allow for testing processes to detect errors that result from unplanned interaction between changes.Detailed release notes on the impacts of changes were enhanced and provide to the MPs for review, assuring that all changes made to the systems were known and understood by the MPs prior to deployment.Enhanced testing process to include automated regression testing and performance and load testing.Improved controls around test activities within internal processes
Steps Taken to Improve Execution
13
Dedicated Configuration and Change Management group in IT established
Allows the Quality Assurance Department to focus on developing Integrated Product Release level Test Plans and executing them through a full test cycleConfiguration and Change Management group focuses on improving the change management policies to manage change, facilitate release planning and manage the configurations of the NYISO production systems
Steps Taken to Improve Execution
14
Dedicated Security group in IT to focus on Cyber SecurityAddress security vulnerabilities and compliance with established industry standards
Addition of dedicated platform support groupsAdminister and address new and existing products, thus assuring thorough support of the Market and Operational Systems
State-of-the-art Network Operating CenterDedicated to 24-7 proactive monitoring of the Production Systems on both the Market and Operational sides.
Alignment of the IT Development Organization to the Project and Product Management Organization at the Business Unit Level
Align development efforts with the needs of the business both internally and externally
Steps Taken to Improve Execution
15
Established continuous process improvement group and a process & controls group (Merged groups in 2009)
415 mapped processes799 identified controls140 Realized process improvement efforts49 Active efforts across key processes58% of organization trained in process improvement tools and methods
Process improvement efforts have resulted in improved quality, increased efficiencies, and cost savings across key business processes.
Additional benefits include fostering a culture of process excellence in day-to-day operations.
Steps Taken to Improve Execution
16
Formalized control testing plans
Centralized controls repository
Documented processes & controls
Used as part of the continuous process improvement cycle and the reasonable assurance services cycle to proactively
improve processes and controls
Steps Taken to Improve Execution
17
BillingQuick Start Reserve Settlements on the Initial Invoice
Reduce time to invoice from 4 month invoice to initialThunderstorm Alert Settlements on Initial Invoice
Reduce time to invoice from 4 month to initialReduce The Number of Rejected Start-up/Shut-Down Requests
Streamlined process for approving economic reliefReduce The Number of Curtailed Wheel-through Transactions where one of the Balancing Authorities are Not Notified of the Curtailment
Reduced manual adjustments & customer inquiries for transaction curtailments.Increase the number of Invoices Issued before 10am on the Date Due
Improve the timeliness of the invoiceOperations
Improve VSS/DMNC Reporting Process to Transmission OwnersReduced time of report from three weeks to five days
Improve Alarm Notifications in RangerImprovement for Operators to highest priority alarms
Improve New/Modified Tie-Line Setup ProcessReduced incorrect set-ups to zero thus reducing billing data errors
Reducing One-Line Display Update Cycle Time Cycle time reduced in halfOne benefit is the reduced time on data verification conversations with Transmission Owners
Customer ServiceReduce cycle time in Customer Registration Process
Improvement of Registration DocumentationImprove Customer Inquiry Resolution Process
Reduce the cycle time of responses
Sample of 140 LSS Projects
18
Going Forward: Continuous Improvement Cycle
Identify& prio
ritize
new opportunitie
s
Monitor processperformance
The continuous improvement cycle
is a key component of the
quality management
system
19
Going Forward: Internal Control
Internal control is broadly defined as a process, effected by an entity's board of directors, management, and other personnel, designed to provide reasonable assurance regarding the achievement of objectives in the following categories:
a) Effectiveness and efficiency of operations;
b) Reliability of financial reporting; and
c) Compliance with laws and regulations.
Internal Control-Integrated Framework document page 5Committee of Sponsoring Organizations of the Treadway Commission (COSO)
20
Going Forward: Reasonable Assurance
21
Reasonable Assurance Services Cycle
Policy, Process and ControlChange Management
(Impact/Risk Analysis)
Controls Testing Continuous Validation
•Process and requirements Analysis •Control Self-Assessment
•Risk based methodology to select controls to be tested
•Tested for design and operational effectiveness
Our repositoryof informationNot effective?
Update required
This is key
Upd
ate
•Allows us to monitor changes to the systems•Assess impact to system•Update system artifacts
Provides key stakeholders reasonable
assurance that our processes are producing
expected results.
These services provide the mechanism to
prevent, detect and remediate
The New York Independent System Operator (NYISO) is a notThe New York Independent System Operator (NYISO) is a not--forfor--profit profit corporation that began operations in 1999. The NYISO operates Necorporation that began operations in 1999. The NYISO operates New Yorkw York’’s bulk s bulk electricity grid, administers the stateelectricity grid, administers the state’’s wholesale electricity markets, and provides s wholesale electricity markets, and provides
comprehensive reliability planning for the statecomprehensive reliability planning for the state’’s bulk electricity system.s bulk electricity system.____________________________________________________________________________________________________________________
www.nyiso.comwww.nyiso.com