40-Gbps and 100-Gbps Ethernet will bring new test challenges
By Martin Rowe, Senior Technical Editor -- Test & Measurement World, 3/1/2009 2:00:00 AM
Internet and data-center users always demand higher bandwidth to carry voice, data, and especially video. Because of that demand, today’s 10-Gbps optical and electrical links are running out of capacity. Data centers and core networks need faster links.
| ALSO SEE: Test equipment for 40-Gbps and 100-Gbps Ethernet Links to more information about 40/100-Gbps Ethernet A shorter version of this article appeared in our March 2009 Test Digest. ADDED APRIL 1, 2009: Information about 40G/100G test equipment announced at OFCNFOEC 2009. |
That was the conclusion expressed at OFCNFOEC in March 2008 (Ref. 1). By that time, IEEE P802.3ba, a standard that will define an architecture for 40-Gbps and 100-Gbps Ethernet, was already in development (Ref. 2). Although IEEE P802.3ba is still in the works, engineers around the world are beginning to develop products that employ these links, and those products will need testing. While much of the testing for IEEE P802.3ba will leverage 10-Gbps technology, some tests will require new equipment and new techniques.
Flexible architecture
A team of engineers, led by John D’Ambrosia of Force10 Networks, is working out the details of IEEE P802.3ba. While details are still forthcoming, the IEEE P802.3ba task force has defined a general architecture. “40 Gigabit Ethernet and 100 Gigabit Ethernet Technology Overview” explains the architecture’s protocol sublayers and how they function (Ref. 3).
Figure 1 shows a simplified diagram of the architectures for 40-Gbps and 100-Gbps optical networks. Although the architectures differ slightly, IC manufacturers could implement both in the same part.
  Figure 1. The IEEE P802.3ba architecture defines three sublayers (shaded boxes) for optical transmission that support four data lanes to achieve 40 Gbps (top) or four or ten data lanes to achieve 100 Gbps (bottom). |
For optical networks, the IEEE P802.3ba standard defines three sublayers: PCS (Physical Coding Sublayer), PMA (Physical Medium attachment) and PMD (Physical Medium Dependent). Both 40-Gbps and 100-Gbps implementations can use existing 10-Gbps fiber PHYs (physical links). In the future, the same architecture will support 100 Gbps links using four 25-Gbps lanes. (IEEE P802.3ba also defines two additional sublayers for copper connections. Figure 1 shows the architecture for optical only).
The PCS takes an aggregate 40-Gbps or 100-Gbps data stream and performs 64b/66b encoding to produce 66-byte blocks. It then sends the blocks across four lanes for 40-Gbps transmission or 20 lanes for 100-Gbps transmission. “We chose 20 lanes because it's divisible by 1, 2, 4, 5, and 10,” explained D’Ambrosia.
So far, it appears that PCS will be used with four or 10 lanes. Each of the 20 PCS lanes will include a lane marker that identifies a lane and provides timing information for each data block. “Trends and Issues in Ultra-High-Speed Transmission Technologies” includes a table that details the values of the alignment markers for each of the 20 PCS lanes (Ref. 4). Figure 2 shows how the 66-bit blocks will travel within the PCS lanes. Blocks will be distributed on a “round robin” fashion across the PCS lanes. The process of dividing the blocks across PCS lanes is called “striping.”
 Figure 2. The PCS sublayer places 66-bit blocks into 20 lanes that include lane markers. Courtesy of Ethernet Alliance. |
The PMA sublayer matches the number of PCS lanes to the number of lanes that a physical layer requires. For 40-Gbps transmission, the PMA sublayer maintains four lanes. For 100-Gbps transmission, the PMA sublayer converts 20 lanes to 10 lanes. These sets of lanes are called XLAUI for 40-Gbps and CAUI for 100-Gbps links. (The XL and C stand for 40 and 100 using Roman numerals; AUI stands for “attachment unit interface.”)
The PMD sublayer provides the final interface to a physical medium. The table describes these physical media and their respective transmission distances. In general, 40-Gbps links will be used in data centers and 100-Gbps links will be used in core networks.
IEEE P802.3ba’s flexible architecture supports LR (long reach) and ER (extended reach) optical links. These links are based on WDM (wavelength-division multiplexing) across SMF (single-mode fiber). SR (short reach) links use multiple fibers, each carrying a different lane.
Because optical and electrical components that send and receive 10-Gbps signals are already available, initial implementations of 40-Gbps and 100-Gbps transmission systems will use four and ten 10-Gbps lanes, respectively. Four-lane implementations using 25-Gbps links (actually 25.78125 Gbps because of encoding) will take some time to appear, but communications carriers are already planning to use WDM, which has four wavelengths, on single-mode fiber for 40-km transmissions (Ref. 5).
Testing P802.3ba
Because 10-Gbps lanes will appear first, you’ll be able to use your existing 10-Gbps optical test equipment to measure parameters such as timing jitter, amplitude, and BER (bit-error rate). But you’ll have to test one transmission path at a time until equipment that can test four or 10 lanes becomes available. “Tests for the individual optical lanes will be very similar to tests for existing 10-Gbps technology,” said Edward Nakamoto, director of hardware for Spirent Communications.
Testing multilane Ethernet will require you to start by testing each lane individually for signal integrity and BER. When testing four or ten 10-Gbps lanes, you must illuminate all lanes, preferably with data, and perform BER measurements on each lane.
You’ll then have to modulate the laser with the data. “Trends and Issues in Ultra-High-Speed Transmission Technologies” (Ref. 4) also describes modulation techniques that are in development for use with single-mode fibers for long-haul transmissions.
Michael Fleischer-Reumann of Agilent Technologies noted that you can test an optical medium by generating PRBS (pseudorandom bit sequence) test patterns with a BER tester. When testing an implementation that uses four or 10 fibers or wires in each direction, you’ll have to test each path individually. “When testing a WDM multimode fiber with four or 10 wavelengths, you’ll need a tunable laser,” he said.
Testing individual lanes is a good start, but you’ll need to test lane timing and skew as well. When using WDM fiber, each data stream will use a unique wavelength, but each wavelength has a different propagation speed. Optical receivers must compensate for timing differences in transmissions.
To do that, IEEE P802.3ba will define alignment blocks at the PCS sublayer that convey timing information. Alignment blocks appear once every 16,384 blocks in a data stream (Ref. 5). Receivers will use those blocks to realign the lane blocks before reconstructing the data stream. “The standard is very insensitive to skew,” said Jeff Lapak, 10 Gigabit Ethernet Consortium manager at UNH-IOL (University of New Hampshire Interoperability Lab). “It will be difficult to break alignment as part of our testing, but we’ll do it.” Lapak intends to test for skew-induced errors by removing alignment blocks from where they belong and by inserting them where they don’t belong in a data stream.
Unfortunately, detecting errors is complicated because testers may not have access to each sublayer. Because IC designers will try to integrate the sublayers into as few devices as possible, it’s likely that the PCS, PMA, and PMD sublayers won’t reside in three ICs. At least two, and perhaps all three, may reside in a single IC.
Lapak said that he’s waiting to see how IC designers package the sublayers before UNH-IOL engineers and students develop test tools. “IOL wants to test where sublayer interfaces will be the most consistent among manufacturers” he said. Companies such as Sarance Technologies have begun implementing 40-Gbps and 100-Gbps Ethernet Cores into Xilinx FPGAs (field-programmable gate arrays). This implementation puts the PCS and PMA sublayers together and it adds MAC (media access control), a layer above PCS.
Because the sublayers will reside in ICs, the interfaces between the devices will be electrical regardless of the physical transport medium. Whether it’s four or 20 lanes, electrical crosstalk will be a significant challenge.
“Interference and crosstalk may occur on the electrical side,” said Toshihiro Suzuki of Anritsu, a member of the IEEE P802.3ba task force. “At the design and verification stage, engineers must test for crosstalk and interference by using a multichannel pattern generator.” Thus, you’ll then need an oscilloscope to look at adjacent lanes—where you can get access to them.
At the system level, you will need to test the multiplexing and demultiplexing functions. “Tests will need to be developed to test the breaking up of the traffic into multiple lanes, said Spirent's Nakamoto.”
Lapak of the UNH-IOL went a step further, saying that he expects to test block encoding and decoding by reversing the order of the data on PCS lanes. “You don’t know which lane will carry a block. Every lane has to be able to carry blocks from any other lane.” Thus, he expects to develop test tools to reverse the order of the lanes and test whether a receiver will receive alignment information and reconstruct the data.
References
1. Rowe, Martin, “
,” Test & Measurement World online, February 27, 2008.
2. Rowe, Martin, “
,” Test & Measurement World, December 1, 2007.
3. D’Ambrosia, John, David Law, and Mark Nowell, “40 Gigabit Ethernet and 100 Gigabit Ethernet Technology Overview.” Ethernet Alliance, Beaverton, OR, November 2008.
4. Technical Note: “Trends and Issues in Ultra-High-Speed Transmission Technologies,” November 2008, Anritsu, Morgan Hill, CA.
5. Suzuki, Toshihiro, “XLAUI/CAUI Jitter Tolerance Test Requirement Proposal,” Comment #199 from the November 2008 Plenary Meeting, IEEE PP802.3ba 40Gb/s and 100Gb/s Ethernet Task Force, November 11–13, 2008.
Links to more information about 40/100-Gbps Ethernet
“40G/100G Implementations with 10G FPGA,” Altera, July 2008.
“High Speed Ethernet (HSE): 100 GE Proof of Concept Demonstration,” from Ixia contains links to white papers “Enabling 100 Gigabit Ethernet: Implementing Multilane Distribution in the PCS Layer” and “An Overview of Next-Generation 100 and 40 Gigabit Ethernet Technologies.”
“The Road to 100G Networking,” Ciena, December 2008.
Shafal, Farhad, “Technical Feasibility of 100G/40G MLD,” Sarance Technologies, March 2008.
Video, “World's first 100GbE over 100G Wavelength Demo - by Nortel”
|
Test equipment for 40-Gbps and 100-Gbps Ethernet Because testing the 10-Gbps physical lanes that comprise 40G and 100 links is similar to testing 10-Gbps Ethernet, you can test the physical layer of 40-Gbps and 100-Gbps Ethernet with oscilloscopes, BER testers, and optical spectrum analyzers.
Agilent Technologies’ E4899A serial BER tester, designed for research labs and standards bodies, lets you perform BER tests at 40 Gbps and 100 Gbps. Anritsu recently announced I/O cards for its MP1800A signal analyzer that let you modulate and analyze 40-Gbps and 100-Gbps optical signals. Centellax recently introduced a 40-Gbps clock and data multiplexer. You can use the instrument to perform BER measurements on physical-layer links. PicoSolve, which was acquired by EXFO on February 9, 2009, manufactures PC-based optical sampling oscilloscopes that can characterize and monitor high-speed transmissions at 40 Gbps. What’s still needed?
Protocol analyzers that decode the data blocks into Ethernet packets will also help. In addition, testers that inject unexpected alignment blocks and remove expected alignment blocks will help you test your network link under stressed conditions. IC manufacturers are currently working with test-equipment makers to develop the components that will carry the protocol sublayers. One example is Altera, which has announced that its high-end FPGAs are available with 24 transceivers capable of communicating at 11.3 Gbps. Thus, one device can handle ten to eleven 10-Gbps lanes with additional transceivers available for generating errors and monitoring traffic. The FPGA can connect directly to optical modules, and it can hold the PCS and PMA sublayers in the device. But if designers use the Altera device, then test engineers won’t have access to the interface between the PCS and PMA sublayers for testing. It’s an engineering tradeoff that you may have to make. What about 25 Gbps lanes? The architecture that IEEE P802.3ba will define provides for implementations that use four 25-Gbps physical lanes to achieve 100-Gbps throughput. These implementation will need new technology in the form of ICs and optical components that will run at those speeds.
The higher data rate will require engineer to develop electrical and optical components capable of reaching that speed, and test equipment will need to keep up. For example, real-time oscilloscopes will need even more bandwidth than the current 30-GHz state-of-the-art available in LeCroy's WaveMaster 8 Zi series. You’ll also need BER testers, clock-recovery units, optical spectrum analyzers, and other equipment capable of working with signals at those speeds. Stressed-eye testers will also let you test optical receivers for the added signal distortion that will occur at the higher bit rate. |
|
40G/100G test equipment announced at OFCNFOEC 2009 The 2009 OFCNFOEC conference and exhibition in San Diego (March 22–26) brought additional 40G/100G test equipment to the market. Agilent Technologies introduced the N4391A optical modulation analyzer. The N4391A consists of a wide-bandwidth, polarization-diverse coherent optical receiver, vector-signal analysis software, and an Infiniium Series 90000 oscilloscope. The system produces optical I/Q diagrams and optical constellation diagrams. Anritsu’s MP180000A now has option 001 (pre-code) and option 002 (decode) that supports the DP-QPSK (100G) and DQPSK (40G) optical modulations. The options let you evaluate optical modulators. You can use the MP180000A to both generate and analyze singals using a PRBS31 data pattern. Centallax introduced a multichannel signal integrity test system that uses up to five 10-Gbps pods to produce PRBS (pseudorandom bit sequence) patterns for performing BER (bit-error rate) tests. The system consists of a Model ST10ME controller and TG5P1A PRBS pods. The system lets you run one channel with a bit sequence of interest and use the other four channels as “aggressors” that add interference to the primary channel. For example, you can set each of the four aggressors with a preprogrammed amount of sinusoidal jitter. Picosecond Pulse Labs added the Model 8020 optical splitter for its line of pattern generators. With the Model 8020, you can send a bit pattern from one source to four receivers. Thus, you can test a 40-Gbps data stream from a 10-Gbps stream. Tektronix added optical sampling modules to its DSA8200 Digital Serial Analyzer series. With 80 GHz of bandwidth, the 80C10B module can handle 40-Gbps data streams for standards such as OC-768, 40Gbase-LR, and 40x10G LAN PHY. The 80C25GBE supports 100GBase-LR4 and 100GBase-ER4, with and without FEC. |
100-Gbps Ethernet is coming
12/21/2007100-Gbps Ethernet is coming
12/06/2007Analyze the highest speed signals
01/31/2009Give the jitters to IEEE 1394b receivers
09/01/2001
-
FLIR offers IR camera for under $3000
-
Don't let the economy compromise quality (Guest commentary)
-
Danaher speeds up restructuring, acquires life-sciences businesses
-
Agilent’s Cover-Extend technology eliminates need for physical test points for in-circuit test
-
So many combinations: Testing a switch-matrix board
View All Webcasts
