Synchronizing Networks with IEEE 1588
Hi, my name is Alex Tan from National Semiconductor. Today I am going to talk to you about synchronizing networks with IEEE 1588. So the objectives of this webinar are twofold. First we're going to introduce the IEEE 1588 Precision Timing Protocol for synchronizing time over an Ethernet network.
Objectives
We'll talk about how it works and what you do in order to get time synchronization on a Ethernet network. And second we'll go into detail on different methods to develop products with IEEE 1588 PTP support. So the first question is how does IEEE 1588 PTP synchronize a network? So the answer is to look at what IEEE 1588 PTP is.
How Does IEEE 1588 PTP Synchronize a Network?
PTP means Precision Time Protocol.
What is IEEE 1588 Precision Time Protocol?
It's an IEEE standard that's designed to synchronize the time between nodes of an Ethernet network. Now that's key because what we're really about is synchronizing the actual time at each node of the network. And from that, once you have synchronization on the time, then you could provide frequency synchronization or even phase synchronization depending on the requirements of your application. The IEEE 1588 PTP is an upper layer protocol. It's typically contained in a UDP/IP packet. That means at the onset it's a software based protocol that lives up at the top in the application space in your system. And finally, it's an algorithm to calculate the time differences between the nodes and then update the time at the slave node to match the master clock. And we'll go into detail on that algorithm and how it works. But essentially the idea is it's kind of like in those old spy movies where there's a couple of spies that need to go off and do things at the same time. So before they go out, one of the spies say I'm going to set my watch and then he looks at his watch and he says one, two, three, mark, and everybody sets their watch to that time. So the IEEE 1588 algorithm is essentially what those spies did, but on a recurring basis so that the time is always synchronized between all of the nodes on the system. So a key question that you might ask is how is this different from older technologies? Well, one of the differences is in synchronization technique.
How is IEEE 1588 PTP synchronization different from older technologies?
Older control and communication standards use synchronous connections that recover the frequency from the control link. So they actually are recovering the frequency off of the link. Now that was a very effective method of providing synchronization but it cannot be used with Ethernet since Ethernet does not provide deterministic links through the network. Also the IEEE 1588 PTP algorithm calculates the time by using that algorithm and then derives the appropriate frequency. So although you end up with the same eventual result, the approach to it is very different. And that will be something to consider when you're developing systems that used to use the older control synchronization technique and are now using IEEE 1588 PTP. One of the key advantages of the 1588 is flexibility. The synchronous connections such as that were used in the older systems need to follow a strict hierarchy, whereas the IEEE 1588 can adapt to the system. The hierarchy is established by the quality of the clock source, not the location of the clock in the control network. So here is a typical IEEE 1588 network topology very simplified but to give you that idea.
A typical IEEE 1588 network topology
So first you have the primary clock source which comes from the GPS in this case. But could also be a Rubidium clock or some other atomic clock or even just a trusted source that is acting as the leader for the network. The 1588 creates a master slave hierarchy of clocks in the network. And the grand master clock source acts like that lead spy that we were talking about in that he is the master in this 1588 clock synchronization operation. In this case, all of the different devices are connected through a switch. And each of the slave devices at the bottom have an ordinary clock on them, and they might be any type of node on a network depending on what you're doing. But they have an Ethernet import, sorry, an Ethernet port and they act as the slave. So all the packets that contain the time information are sent from the grand master clock source through the switch to each of these ordinary clocks at the bottom of the network. And then synchronization occurs between each master and slave pair. By extension, each slave node is effectively synchronized with all of the other nodes in the network through this process. So let's go into a little bit more detail on that. So the first thing that happens is the packets are exchanged over an Ethernet network.
How IEEE 1588 PTP works
The time that the packets are sent and received at the master and at each slave device is recorded. And the precision of that time stamp really is the fundamental, sets the fundamental level of precision for the time synchronization operation. The algorithm calculates the difference in time and the differences in frequency and then adjusts the slave clock time and frequency so that it matches the master. And this is repeated on a periodic basis. It's actually a fairly straightforward algorithm. And if you think it through, it is not very different than what you would do if you were trying to synchronize your watches or your time with other people. But this is now in an algorithm format. So let's now look in even more detail on exactly what is being sent. So on the right hand side of this slide, you'll see what we call a time ladder, and on the left hand bar of that ladder is the master clock time, and on the right hand is the slave clock time.
Adjusting Time with IEEE 1588 PTP
Now although these are both moving forward in time, there's no relationship between the master clock and the slave clock at the beginning, neither in time nor in frequency. The first step is that this, a sync message, so a synchronization message, was sent from the master to the slave. And when it arrives at the slave, the time that it's arrived is recorded by the slave. And when it's sent from the master, the time that it was sent is recorded by the master. The slave is actually doing the calculations so it's the slave that needs to have the time. So the first piece of time information that the slave have is the T2 time. The T2 time is the time that the sync message arrives at the slave. The time that it was sent is sent in the followup message by the master. So that message isn't timed, but it contains that it was sent. And that gives it enough information to calculate the delay between when the slave received that message and when the master sent the message. An important point of IEEE 1588 is it also takes into account the time that the packet was sent going back from the slave to the master. And that's done using the delay request message. In that case, the time that it's sent is recorded as it's sent on T3 by the slave, and the time it's received at the master which is T4 is then sent back in the delay response message. So it's a simple set of essentially four packets that allow the slave to have enough information to calculate the time correction. There are ways to simplify the number of packets that are sent, but this is the basic exchange of packets required to do the time synchronization. Now the time correction is given by the two calculations on the left. The first one is the calculation of the mean path delay which is the time that it took from the sync message to be received at slave from when it was sent in the master. And then also the time from the delay request going back added together and divided by two. The important significance of that is that the basic algorithm now essentially is expecting that the time that it takes in actual time from the master to the slave. Is the same as the time it takes from the slave to the master, in other words a synchronous path. The other calculation then is what is the actual offset from the master. And the offset of the master is to take the time that the first sync message was received at the slave minus the time that was sent from the master. And then from that which is the absolute time on the slave clock scale, and from that you subtract the mean path delay we just calculated. And that is the method by which the time on the slave clock is adjusted to match the master time. But it's not sufficient just to match the times because if you match the time at one point in time without changing the rates, it will quickly move out of synchronization.
1588 Rate Correction
So the 1588 algorithm also has a rate correction. And the rate is really done by looking at multiple sync messages over time to see how each of those sync messages vary. So you look at the difference on the master time scale, which is in this case is T1N - T1, and the difference on the slave time scale, which is T2N - T2, to determine what the rate ratio is. And that's simply obtained by dividing the difference on the slave time scale by the difference on the master time scale. And then you can adjust the slave clock by this factor to match the master clock. Now you have adjusted both the time and the frequency and you now have a slave node that is synchronized to the master node. So here's some general information on the IEEE 1588 spec. Version one was ratified in 2002, and it essentially focused on the needs of companies developing factory automation and test and measurement products.
General Info on the IEEE 1588 Spec
Version two was approved in March of 2008. And it included input from telecommunications, audio-video bridging applications and other applications that could benefit from Ethernet time synchronization. Some of the changes in version two included allowing the synchronization to happen at a much faster rate and different methods of providing higher levels of synchronization throughout the network. So let's take a few moments here to look specifically at how you can add PTP synchronization into a system that's designed to work on an Ethernet network.
Adding PTP Synchronization to System Design
So there's several things you need to consider when you're developing these systems. One is what level of synchronization precision do you need?
Evaluating Implementation Options
Do you need synchronized inputs and outputs? And what's the development impact? How much of an impact is there going to be on your existing hardware and software? Or if you're starting from scratch, how much of an impact will there be on the overall development when you add in IEEE 1588 support? So the steps to implement IEEE 1588 support is that there is a software component because it's a software algorithm, and the IEEE 1588 stack sits in software. And also for nearly all applications that use IEEE 1588, there's a hardware acceleration component to get to the precision required by these applications. So first I'll start off by talking about the pros and cons of the different approaches, and then I'll go into detail on how those different approaches are actually implemented.
Comparing IEEE 1588 PTP Solutions
But this way you have kind of an overview at the front, and you know what to look for when we're discussing this. So first, the most basic approach is the software only approach. From a development standpoint, it requires software development, but essentially no change to hardware if you already have an Ethernet PHY. But there's a big caveat to that, and that's that the precision performance is too low for most applications. And typically you will be looking at a system that has greater than ten microseconds on a single link. There's a lot that happens between when a packet is received at a node and when the software is able to determine the time that it was received. And that's why software has a lower precision. Another step is to do hardware assist in the FPGA. This often requires significant hardware change because it requires either the addition of a high end FPGA or oftentimes a change in the FPGA. And both software and FPGA IP development are required. Now in terms of performance, the FPGA approach will time stamp at the Ethernet MAC level. So at the Ethernet MAC level, you will get fairly good time stamp, and typically you can be looking at around 30 nanoseconds or worse on a single link. Hardware assist in a microcontroller is a similar approach, but it does not require the amount of development that the FPGA does. But it may require you to change to a new microcontroller. And your existing software applications may need to be customized for that new microcontroller. And of course, all development further on will be based on that microcontroller platform. So from a precision standpoint, you're going to get similar time stamping at the Ethernet MAC level, and so similar performance can be achieved on the FPGA. It really depends on the specifics of the microcontroller you choose. Another approach, and one that we've here at National been working very hard to develop and introduce to the market, is the hardware assist in the Ethernet PHY. Such as you can find in the DP83640 Precision PHYTER. This allows a simple hardware implementation because it really just requires replacing an existing PHY with the new DP83640 Precision PHYTER. And it does have software customization required. Similar to a software only approach, it needs a stack plus it needs to be able to communicate the time stamp from the PHY to that stack. So this provides the tightest time synchronization that's commercially available. And typically you'll look at less than ten nanoseconds on a single link. And oftentimes we can show systems based on the DP83640 that will be in the three and a half to say two, two to three nanosecond on a single link performance level. So let's go into more detail on these different implementations. From a software implementation standpoint, what I have at the top is a diagram showing the path of the time information, and really the packet as it's coming into the system.
IEEE 1588 Software Implementation
From the left, which is the network, it goes into the RJ45 or fiber connection into the PHY, and that's really where you have the first opportunity to time stamp. And then from the PHY it will go through an MII RMI interface to the Ethernet MAC, which may be part of a microcontroller, an FPGA or an ASIC. Or it may stand alone and be connected through a PCI or local bus. That microcontroller FPGA or ASIC will then have to analyze the time stamp, and that's done on the right hand side of the bar in the software domain. So first it's handled by the OS which will include the TCP/IP stack or UDP/IP stack and then the applications base. The little icons here show where the time critical actions of the IEEE 1588 protocol are being handled. The clock is representing the 1588 clock, so that's the clock that's being synchronized to the master clock on the other side of the network. The yellow box is the PTP software stack, and the orange box is where the time stamp occurs. And the white box with the blue IO is where the 1588 GPIOs or the time synchronized GPIOs sit. So in this implementation, all of these functions are handled in software. So anything in terms of contact switching, handling interrupts, all of these things can impact the precision that the time stamp is taken in the software. This is the simplest implementation. There are no hardware changes. And the PTP packets travel through all of the hardware from the wire and can incur delays in every step from the PHY to the MAC, the microcontroller and OS, during the software processing. And this level of synchronization is not sufficient for many applications. The next step we talked about was implementing the 1588 at the MAC level with an FPGA. So you could see I've got the similar system diagram on the hardware and software, but we've added an FPGA.
IEEE 1588 Implementation with Hardware Support at the MAC in an FPGA
And this FPGA is actually snooping on the MII RMII bus. It's looking at the packets that are going over that bus. And then it has a connection to the microcontroller, maybe through PCI or a local bus or some other connection, to provide the information that it obtains. So in this case, in the FPGA is located the time stamping and also the synchronized IOs and the 1588 clock. So really, it offers a much higher level of precision than the time stamping in the software because it's not impacted by the OS or microcontroller delays. But it can be impacted by delays at the PHY and on the MII RMII. So in this scenario, the software is still, the PTP stack itself is still being handled in the software on the microcontroller. This requires packet processing and PTP functions to be developed in the FPGA. That can be a significant task and requires a significant amount of specialized IP development. And the precision depends on the details of the FPGA IP and the delays within the Ethernet PHY. The third method is to do the implementation inside of the microcontroller. Again, this allows for time stamping at the MAC layer.
IEEE 1588 Implementation with Hardware Support in a Microcontroller
In this case, oftentimes the MAC will be part of the microcontroller or FPGA. So you could think of this as everything inside of that dashed box as being on one chip. But the time stamp is occurring at the MAC level, and then the 1588 clock and IOs are often handled at the microcontroller. The software is still out, sorry, the PTP stack is still handled in the software. And the precision in this implementation depends on the details of the microcontroller design and delays within the Ethernet PHY. A big consideration in this scenario is that this may require software to be ported over to a different microcontroller. And then will be dependent on that microcontroller's development path going forward. The fourth approach we talk about uses the 1588 time critical functions implemented in the PHY.
IEEE 1588 Implementation with Hardware Support at the Wire in the Ethernet PHY
And the real advantage to this is that it allows the time stamp to be recorded right at the wire, as soon as it goes into the system, for the highest level of precision. And the more precise those time stamps are, the more precise the overall result of the PTP algorithm's calculations are, which results in a more precise time synchronization. Also this approach integrates the clock discipline straight into a single chip. So that it can easily provide a very precise 1588 synchronized clock source independent of the microcontroller and the overall system. So in summary, IEEE 1588 PTP is a key component to providing synchronization to Ethernet networks. It provides both frequency and time synchronization through an exchange of packets over an Ethernet network.
Summary
There are four main approaches to implement IEEE 1588 PTP, software only, development in an FPGA, as a feature built into specific microcontrollers. Or as a feature in an IEEE 1588 PTP Ethernet PHY. For further reading on this topic, please feel free to visit the different links that are called out on this web page.
Further Reading
For both information on the DP83640 time stamping and the PHY approach to solving the IEEE 1588 solution and also to understand the IEEE 1588 protocol itself in the 1588 links.
For more information
And for more information on these products, please go to our product folder at the URL that's shown on this website. Thank you very much for your time. And I appreciate your attendance, and I hope that this webinar has been helpful to you.
Digg
Del.icio.us
