IEEE 1588/PTP: The Future of Time Synchronization in the Electric Power Industry

Authors: Bernhard Baumgartner, Christian Riesch, Manfred Rudigier Authors’ affiliation: OMICRON electronics GmbH, Oberes Ried 1, 6833 Klaus, Austria

Contact: bernhard.baumgartner@omicron.at

1 Introduction

At the latest, when the reasons for power failures or blackouts in which more than one power utility is involved must be analysed, the importance of accurate and precise time stamps of events becomes visible. Experiences from the past show, that aligning data from different utilities can often take much longer than analysing the data itself. But also for protection, automation, and control of electric power systems accurate time references are needed. Radio-based time sources like global satellite navigation systems are commonly used and provide precise time. To avoid the requirement of dedicated GPS receivers for each individual device, time distribution systems such as the Network Time Protocol (NTP) [1] or IRIG-B are used. Whereas NTP was already specified in IEC 61850 [2] as time distribution mechanism, it is only suitable in applications for which a time synchronization accuracy in the milliseconds range is sufficient. In contrast, time codes like IRIG-B provide much better performance, but they require the installation of a separate cabling for the distribution of the timing signals. With the implementation of the Precision Time Protocol (PTP), defined in the IEEE 1588 standard [3][4], for the first time a very accurate and safe way to provide time references throughout the power station’s Ethernet network is made available. PTP allows distributing reference time information in a local area network like NTP does, but due to the innovative protocol it allows to reach accuracies in the sub-microsecond range. This paper presents a general introduction to the PTP standard and its novelties in comparison to existing time synchronization and distribution. It continues with implementation and application scenarios in power utilities including infrastructural requirements to allow a successful deployment of PTP time synchronisation in such environments. We focus on the advantages of the PTP power profile for integrating PTP time synchronisation into a modern IEC 61850 based station control network. The paper ends with an outlook on the implementation and transition scenarios with regard to time synchronization in the electric power industry.

2 Time Synchronization in a Smart Substation

Before touching base on the basic functionalities and advantages of IEEE 1588 PTP, in this section we are providing a short overview on the current and emerging applications for time synchronized measurements and data in the modern power grid.

As long as all processes and events in a facility like a substation are controlled from one singular central point, the absolute accuracy of the stations system time is not really important. But as soon as time synchronized switching events involving more than one substation have to be performed the absolute accuracy of each station’s time reference gains significant importance.

The North American Blackout back in August 2003 visualized how painful and time consuming it can be to align data, whose time stamps are derived from inaccurate time references. As a result the task force investigating the blackout demanded a regulation that ensures a minimum absolute accuracy for time stamped disturbance event data. With the adoption of the NERC

1 Standard PRC018-1 [5] in

2006 it is now a legal obligation that all recorded data has to have an accuracy of 2 ms or better in relation to UTC

2.

1 North American Electric Reliability Cooperation

2 Universal Coordinated Time Scale

Nowadays, a lot of measurement and control data in the power grid have to have an absolute accuracy of approximately 1 ms:

SCADA3 Data

Data from Event and Disturbance Recorders

Time stamped data from Protection Relays (IEDs)

Lightning Strike Correlation

A time accuracy of 1 ms is relatively easy to reach. But some current and emerging future measurement applications require a much higher accuracy. The applications mentioned below for example require an absolute accuracy of 1 µs or better:

Sampled Values

Synchrophasor4 Measurements

Travelling Wave Fault Location

In the IEC 61850-90-1 several performance classes for IED synchronizing have been defined. The accuracies for these classes range from 1 ms to 1 µs and are shown in Figure 1 and Figure 2.

Time Performance Class Accuracy Purpose

T1 ± 1 ms Time tagging of events

T2 ± 100 µs Time tagging of zero crossings and of data for the distributed synchrocheck. Time tags to support point on wave switching

Figure 1 – Time performance classes for time tagging of events according to IEC 61850 90-1

Time Performance Class Accuracy T3 ± 25 µs

T4 ± 4 µs

T5 ± 1 µs

Figure 2 - Time performance classes for instrument transformer synchronization

To time synchronize all devices involved in the processes and measurements mentioned above usually GPS disciplined time references, commonly called substation clocks are used. In central Europe alternatively substation clocks synchronized to the DCF 77 long-wave time signal and standard-frequency radio station in Mainflingen, Germany, are in use. The substation clock provides a time synchronization signal, which is distributed to the IEDs throughout the substation using a variety of time distribution methods. Most classic distribution methods require a separate time distribution infrastructure as shown in Figure 3.

3 Supervisory Control & Data Acquisition

4 Synchronized phasor measurements of sinusoidal quantities synchronized in time and expressed as

phasors.

Figure 3 – Simplified block diagram of a time synchronization signal distribution via a separate time distribution network

The following methods are commonly used for the distribution of time synchronization signals in substations [6]:

IRIG-B The IRIG

5 Time Codes were originally developed by a working group of the US Airforce to allow

standardized time coding of measurement data originating from different locations. Today mostly the IRIG-B Code is used for civilian applications including the electric power industry. The IRIG-B Code transports the time synchronization signal with 100 bits/s and depending on the used distribution method

6 synchronization accuracy between 1 ms and 10 µs can be reached. For the distribution of

the IRIG-B code either twisted pair wires or coaxial cables are used. The original IRIG-B Code did only support UTC. To take into account leap seconds, local time zones and daylight saving time the IRIG-B time code was extended in IEEE 1344-1995

7. However, this protocol extension is only

supported by some IEDs on the market, while many are still only compliant to the standard IRIG-B time code.

One Pulse per Second (1 PPS) The digital 1 PPS

8 signal is a widely used reference signal for time synchronization and is provided

nearly by every substation clock. The signal is a simple rectangular 1 Hz pulse, whose rising or falling edge marks the beginning of a new second. The synchronization accuracy of the pulse is in the range of a few nano seconds. If the cable delays occurring during the distribution of the signal are taken into consideration the achievable overall accuracy is about 1 µs. The 1 PPS signal itself does not contain any additional time information which allows linking the edge of the pulse to a specific absolute time. As a result, additional time information needs to be transported to the IEDs via a separate system (e.g. NTP

9). Due to this fact the importance of 1 PPS for time synchronization in substations is

constantly declining.

Serial (ASCII) Broadcast Time Codes These time codes are just mentioned for completeness. Due to better alternatives they are very seldom used in the electric power industry. In such systems the reference clock distributes the actual time in ASCII string format via a serial interface. The achievable synchronization accuracy is strongly depending on the used bit rate plus firmware and software latencies. For bitrates of 19200 Baud or higher, a typical system accuracy of approximately 1 ms can be reached.

5 Inter Range Instrumentation Group

6 Un-modulated Code (0/+5 V Shift) or modulated Code (1 kHz Carrier)

7 IEEE Standard for Synchrophasors for Power Systems

8 1 Pulse Per Second

9 Network Time Protocol. For further information see the following section.

With the introduction of IEC 61850 the use of Ethernet networks in substations is constantly increasing. Therefore time synchronization systems which make use of the station network (see Figure 4) are constantly gaining importance. An example for such a system is the network time protocol (NTP).

Figure 4 – Simplified block diagram of time synchronization signal distribution via the station network

NTP The Network Time Protocol (NTP) is used to synchronize clocks in computer networks and was especially developed to allow reliable time synchronization in networks with variable packet runtime, such as the Internet. The achievable synchronization accuracy is mainly a function of the network traffic and the latencies in the used operating systems. Special algorithms are used to estimate the average delay time between the NTP server and each individual client in the network. In the Internet a time accuracy of approximately 10 ms can typically be reached. In local area networks like used in substations the achievable accuracy is in the range of a few milliseconds. This accuracy is sufficient to assign a certain absolute point of time to the rising edge of a 1 PPS signal. However, since two separate time reference signals (NTP & 1 PPS), distributed via two separate cablings are required, such combined solutions are very seldom implemented.

System Typical accuracy Separate cabling needed Ambiguity

IRIG-B 10 µs - 1 ms10 yes 1 year

1PPS 1 µs yes 1 second

ser. ASCII 1 ms yes none

NTP 1 ms - 10 ms no none

Figure 5 – Overview on the so far covered time synchronisation systems

The target for further developments was to create a time synchronization system, which allows to achieve high synchronization accuracy (1µs or better) while using the existing Ethernet network infrastructure in the smart grid. This target was achieved with the precision time protocol (PTP).

10 Depending on the chosen distribution method (modulated or unmodulated)

3 Precision Time Protocol (PTP)

With the Precision Time Protocol, the IEEE 1588 standard [3] defines a solution for synchronizing clocks in a computer network, for example via Ethernet. Like for NTP a common cabling for data communication and time synchronization is used which results in reduced cabling requirements and allows the use of the existing network cabling. Contrary to systems with separate distribution infrastructures the cable distribution delays cannot be simply calculated by measuring the used cable lengths during installation. The route a data packet takes in a computer network can change dynamically and the network infrastructure can be changed very easily, which requires dynamic correction of the network time delay for each data packet. Network components such as switches can cause additional delays for data packets, which can be significantly higher than the delay caused by the cabling. The Precision Time Protocol offers a method that allows determining and compensating the above mentioned dynamic time delays automatically.

Time Synchronization with PTP The basic principle of the method is shown in Figure 6. Two clocks, a Master Clock M (e.g. a GPS disciplined time reference) and a Slave Clock S (e.g. of a protection relay or an intelligent merging unit) are connected via a computer network. The target is to synchronize the Slave S to the Master M with the result that both clocks synchronously provide exactly the same time information.

Figure 6 – Determination of the time between master clock and slave clock using two data packets which are sent in opposite directions.

The time deviation between the two clocks is expressed by the value . In Figure 6 this deviation is also shown by the shifted zeros on the time axes. The target is to measure . To achieve this, a data packet A is sent from the Clock M via the Ethernet network to the Clock S. Thereby the Master M notes the point of time , at which the data packet was sent. Thus, is a time stamp containing the absolute point in time the master’s internal clock showed when the data packet was sent. The data packet requires some time to travel through the computer network before it reaches the Slave Clock S. This time delay is called in Figure 6 and is the sum of all delays caused by the cabling and the

network components such as switches. After this time delay the data packet arrives at the Slave S which now generates the time stamp .

This means that the Slave S notes the time his inner clock shows at the time of arrival of the data packet A. Thus, the relationship between and is given by

. (1)

Subsequently the Slave S sends back the data packet B to the Master M. The times at which the packet has been sent out by the slave ( ) and at which it has been received by the master ( ) are recorded for the further calculation. If both data packets take the same path through the network we can assume an identical network time delay and get

. (2)

At each clock two timestamp values are now available, and at the clock M as well as und at clock S. As soon as the master forwards his time stamps ( and ) to the Slave in a data packet the Slave is able to solve the system consisting of equations (1) and (2) to calculate . It obtains the deviation of his time from the master’s time as

. (3)

The Slave S can now use this information to correct its inner clock [7]. By continuous repetition of this measurement (typically once every second) and correction of the slave clock the deviation between the master and the slave clock can be reduced to typically 100 ns.

For the delay measurement method described above it is crucial that both data packets A and B take the same path through the network. In case of substation networks which include redundant paths to increase the reliability of the system this requirement cannot be fulfilled.

Figure 7 – Different packet routing for data packet A and B

Figure 7 shows an example where the data packet A takes a different path through the network than packet B. Consequently, the propagation delay time of both packets will not be equal and equation (3) does not hold anymore. Therefore, another delay measurement mechanism called the peer delay mechanism (or peer-to-peer delay mechanism, P2P) was introduced in IEEE 1588-2008. In contrast to the delay request-response mechanism (or end-to-end, E2E), in a network of clocks using the P2P mechanism, the propagation delay of each link in the network is determined in a separate measurement. Figure 8 depicts an example.

Figure 8 – Principle of the peer-to-peer delay mechanism

All PTP clocks that are configured to use the P2P delay mechanism measure the propagation delays of each network link using the principle described in Figure 6. For each network link the

corresponding propagation delay time is known. To synchronize the slave clock to the master

clock, the master clock periodically sends Sync packets which are received by the slave clocks. Again, this sync message can take one of two different paths (labeled Path 1 and Path 2 in Figure 9). But no matter which one it takes, the network devices in between add correction information based on the measured propagation delay times to the Sync packet. On its arrival at the slave clock, the sync packet contains the accumulated propagation delay times, allowing again the calculation of the time offset between master and slave.

Figure 9 - Slave C2 synchronizing on data packets arriving via different routes.

From the description of the two synchronization methods (E2E and P2P) it becomes obvious that special demands for the network infrastructure apply to ensure a proper function of the time synchronization via PTP. These infrastructural requirements are addressed in the following section.

Network Infrastructure for PTP The above mentioned methods make the following requirements for the network infrastructure necessary:

The propagation delay for the data packets used for synchronization needs to be symmetric, network traffic is not allowed to influence the synchronization accuracy;

All time stamps used for synchronization need to be recorded with high accuracy.

The first requirement is easy to fulfill as long as just a simple network cable is used to interconnect the two clocks, since in this case the time delay differences between the two signal directions can be ignored. In a complex computer network with network switches this requirement is not necessarily fulfilled. Especially, since network components cause different packet propagation times, depending on the level of data traffic in the network. As a result, conventional network switches are causing significant measurement errors with the described synchronization methods. To ensure proper functioning of PTP it is necessary to use special network components, so-called Transparent Clocks [3]. Such clocks are network switches which determine the signal propagation time for each data packet through the switch. They forward the corresponding correction information to the clocks in the network. If the routing of the data packets is changing inside the network – for example due to redundant network components – special mechanisms like the described peer-to-peer mechanism have to be supported by the transparent clocks. In such a case transparent clocks have to support the peer delay measurements and add the resulting delay information to the synchronization packets.

The second requirement for accurate time stamps is usually fulfilled by doing the time stamping in dedicated hardware in the network interface of the clock. This hardware time stamping eliminates software induced latencies and results in time stamp resolutions of 10 ns and less. In principle it is also possible to time-stamp the PTP data packets in software if no hardware with PTP support is

available11

. However, to utilize the high accuracy that can be achieved with the Precision Time Protocol, network components must be equipped with PTP capable network interfaces.

Automatic Configuration Another special feature of the protocol defined in IEEE 1588-2008 is the Best Master Clock Algorithm (BMCA). This algorithm is used to automatically determine the current best clock, which is the one to provide the reference time in the network. This clock becomes the grandmaster clock in the network and all other clocks synchronize their time to the grandmaster clock’s time. A special manual configuration to define which clock takes over the grandmaster role in the network is therefore not needed. Backup solutions are also covered by the Best Master Clock Algorithm. If the grandmaster clock of the network has a malfunction the second best clock in the network becomes the new grandmaster clock without the need of any special interaction. The protocol ensures that also in complex network infrastructures a unique grandmaster is determined and used for further synchronization.

Figure 10 – Best Master Clock Algorithm (BMCA) in a two-part network with six clocks (C1 to C6).

Figure 10 shows a network consisting of 6 clocks (C1 to C6), which are connected to each other via two switches (S1 and S2). Among the six clocks the clock C4 has an outstanding characteristic. This clock has a GPS receiver and is therefore in the position to obtain highly accurate time information from the satellite navigation system. Since this clock possesses the most accurate time in the network the BMCA ensures that clock C4 becomes the grandmaster clock in the network. All other clocks in the network (C1, C2 … which for example can be included in IEDs or intelligent merging units) are synchronizing their time to the time provided by C4.

C3 is a special clock. This clock has two network ports and is therefore in the position to interconnect the two networks around the two switches S1 and S2 with each other. Due to the BMCA the network port of Clock C3 connected to Switch S1 is configured as slave (indicated in Figure 10 by the letter S) and as a result C3 receives its time from the grandmaster clock C4. In the network around Switch S2 the clock C3 takes over the master role and forwards the time received from C4 to the slaves C5 and C6. In the IEEE 1588 terminology a clock like C3 is called a Boundary Clock. Such a clock allows for example to synchronize the time in two fully isolated networks to one common grandmaster.

The described configuration of the clocks is automatically performed by the BMCA. If a clock removed, added or replaced, the network timing structure is reconfigured automatically.

11 This method is called software time stamping.

Power Profile The IEEE 1588 is a very comprehensive standard which defines a number of operation modes to cover a high variety of applications for measurement applications, telecommunication applications, automated test equipment, and last but not least also the electric power industry with a special focus on the smart grid. Application specific adaptations of the protocol are done in profiles. All protocol adaptations relevant for the electric power industry are taken care of in the so-called Power Profile IEEE C37.238-2011 (IEEE Standard Profile for Use of IEEE 1588 Precision Time Protocol in Power System Applications [4]). This profile defines which parts of the IEEE 1588-2008 standard need to be implemented for electric power applications and which settings should be applied. The profile was developed with a special focus on the interoperability with other protocols used in the Smart Grid [8] such as the protocols defined in the IEC 61850.

The last remaining cable delay It has been shown that with the implementation of PTP all infrastructure related delays between the clocks in the network are automatically compensated. Only the time delay caused by the cable that connects the GPS Antenna with the grandmaster clock still needs to be compensated manually. Further on, even special HF antenna cables have high cable attenuation at the GPS reception frequency

12, which limits the maximum cable length to a range from 50 m to 100 m. If difficult

reception conditions or other local circumstances (e.g. in a cavern power plant) require to cover longer distances, additional measures, such as the use of in-line amplifiers or signal mixing to an IF frequency, need to be taken.

Figure 11 – OTMC 100p Antenna-integrated PTP Grandmaster Clock by OMICRON Lab

By directly integrating the PTP grandmaster clock into the Antenna as shown in Figure 11, the use of a coaxial antenna cable is no longer needed. The connection to the clocks which shall be synchronized to the master can be simply established via Ethernet. The power supply of the grandmaster clock is also done via the Ethernet cable using PoE

13. With standard Ethernet cables

already a distance of 100 m can be covered. By using optical Ethernet the distance between the outdoor mounted antenna-integrated grandmaster clock and the network clocks to be synchronized can be extended up to two kilometers. Due to the omission of the antenna cable there is no need to manually compensate its cable delay anymore.

12 1.57542 GHz

13 Power over Ethernet

4 Implementation & Transition Scenarios for the smart substation

The comprehensive nature of the IEEE 1588 standard allows developing individual, tailor-made time synchronization concepts for each facility.

Figure 12 - PTP implementation example for a smart substation

Figure 12 shows a simplified implementation scenario for an IEC 61850 substation. For simplicity the IEC 61850 station bus and the IEC 61850 process bus are symbolized by the two transparent clocks S1 & S2

14. To each bus a GPS disciplined ordinary clock (C1 and C2) is connected. In our example

the clock C1 delivers the best time and therefore it was automatically selected as grandmaster clock by the BMCA. All clocks in the system including clock C2 are synchronized to C1. To avoid a direct connection between the IEC 61850 station bus and the IEC 61850 process bus a boundary clock is used to time synchronize both buses. This ensures that both buses are not directly connected and therefore malfunctions or problems in one bus do not cause problems in the other bus. The shown setup also provides full redundancy. If the clock C1 loses its accuracy the next best clock in the system, which is very likely C2, will become the grandmaster of the entire substation. Also a malfunction of the boundary clock is covered. In this case the clock C2 will become grandmaster of the IEC process bus while C1 remains grandmaster of the IEC 61850 station bus.

The clock C3 has a special function since it is synchronized to the grandmaster clock and locally provides time synchronization signals like IRIG-B or 1 PPS to non-PTP capable IEDs. These IEDs (colored grey in Figure 12) communicate via the IEC 61850 process bus and receive their time synchronization signal from clock C3 via a separate wiring.

For existing substations a transition to PTP can be done step-by-step, since existing time synchronization systems and infrastructure can remain in use until the PTP infrastructure is fully

14 In reality a number of switches will be used to provide the bus infrastructure.

deployed. Even NTP and PTP can be operated in parallel over the same network infrastructure. While the existing network cabling can be re-used all Ethernet switches have to be replaced by PTP capable switches (transparent clocks).

In principle it is possible to upgrade the firmware of a modern IED to allow PTP time synchronization. By using software-only time stamping an overall accuracy in the range from 20 µs to 100 µs can be achieved [8]. This is sufficient for general time stamping purposes outlined in IEC 61850-90-1 (see Figure 1) but insufficient for the accuracy classes T3 to T5 (see Figure 2). As a result IEDs that need to meet the accuracy classes T3 to T5 either need to be upgraded to allow hardware time stamping or exchanged if they are to be synchronized via PTP. Alternatively local ordinary clocks (like C3 in Figure 12) can be used to derive previously used time synchronization standards like IRIG-B to synchronize legacy equipment.

5 Summary

If the entire infrastructure of a network is IEEE 1588 compliant all propagation delays inside the network are automatically compensated. Thus the protocol ensures that the time reference received by every device in the network is the same. The IEEE 1588 standard defines an entire system of different PTP clocks, such as master clocks, transparent clocks, boundary clocks and ordinary clocks, which ensures that all timing needs in a complex network can be addressed. Further on, redundancy and back up requirements for safe 24/7 operation are taken care of by innovative procedures like the Best Master Clock Algorithm. To ensure that PTP fully works in a modern power utility the power profile for PTP clocks was defined in IEEE C37.238, which especially addresses the needs and requirements of the electric power industry and integrates with IEC 61850. Therefore the precision time protocol is an optimum and flexible technology for the time synchronisation in the smart grid.

Based on the outlined facts it can be justifiably claimed that the future of time synchronization in the smart grid will be defined by the IEEE 1588 Precision Time Protocol.

Literature

[1] RFC 5905, Network Time Protocol Version 4: Protocol and Algorithms Specification

[2] IEC 61850: 2003 Communication networks and systems in substations

[3] IEEE 1588:2008 IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems

[4] IEEE C37.238-2011 IEEE Standard Profile for Use of IEEE 1588 Precision Time Protocol in Power System Applications

[5] PRC-018-1, NERC Standard (NERC short for North American Electric Reliability Corporation)

[6] Dickson, B.: Substation time Synchronisation PAC World Magazine, Issue: Summer 2007

[7] Eidson, J. C.; Measurement, Control and Communication Using IEEE 1588. 2006; London: Springer-Verlag

[8] Steinhauser, F., Riesch, C., Rudigier, M.: IEEE 1588 for time synchronization of devices in the electric power industry. ISPCS 2010; Portsmouth, NH, USA.

The Autors:

Bernhard Baumgartner obtained his engineering degree in 1990 from the technical college for electronics and telecommunications in Rankweil, Austria. In the following years he worked as development engineer and product manager for major companies in the digital broadcast industry. During his time as hardware developer he was intensively engaged with time synchronizing technologies for digital TV and radio transmitters in single frequency networks. Since 2006 he works for OMICRON electronics in Klaus, Austria, where he is responsible for the business segment OMICRON Lab.

Christian Riesch studied Electrical Engineering at the Vienna University of Technology (VUT), Austria and received the Dipl.-Ing. (M.Sc.) degree and the Ph.D. degree in 2005 and 2009, respectively. From 2005 to 2009 he was a Research Assistant with the Institute of Sensor and Actuator Systems, VUT, performing research in the field of miniaturized sensor technology. Since 2009 he is with the hardware development team of OMICRON electronics in Klaus, Austria, and works on solutions for the time synchronization of measurement systems.

Manfred Rudigier studied Computer Science at the University of Applied Sciences Vorarlberg and obtained a Master's degree in 2009. Since 2007 he works as an embedded software engineer at OMICRON electronics in Klaus, Austria. His responsibilities include, among other things, the development of the IEEE 1588 protocol software.