IPTV: Video's latest test frontier

  
If a transmission lacked I-frames, it would quickly become garbled, because the MPEG decoder would never receive a reference frame to restore the picture. Likewise, if a viewer turned on a channel in mid-program, the lack of a complete reference frame would leave the picture undecipherable until one arrived from the service provider. That's where the I-frames are essential. An I-frame containing complete image information must be inserted every few frames (about two or three times per second) to serve as a reference for the P-frames and B-frames.

If an IP packet containing part of an I-frame is lost, the MPEG decoder in the receiver (TV set or set-top box) loses crucial information that affects preceding and following frames. "A set-top box may crash if it misses an I-frame" noted Paul Robinson, MPEG segment manager at Tektronix. So, a network must deliver nearly lossless IPTV streams. Figure 1 shows the results of lost video information. (To see this scene in video format, you can download a video of these images.)

Figure 1.  Lost video information from a bad I-frame produces poor quality video in three of the four frames. A bad I-frame can result from lost packets. (a) A good video frame without errors. (b) This figure has five bad MPEG slices that result from the B-frame forward-referencing a bad I-frame. (c) Here, forward-referencing a bad I-frame results in poor motion-vector placement. (d) This poor image results from reverse-referencing a bad I-frame. View the entire video. Courtesy of Tektronix.

Other network problems such as packet jitter and delay can reveal the health of a network. Packet jitter refers to difference in arrival times from one packet to another. "Jitter doesn't necessarily affect video quality," said Marc Todd, CEO of IneoQuest, "but jitter is a good indicator of potential packet loss." Buffers in network components, particularly in set-top boxes, store incoming packets and usually deliver them smoothly. Cumulative packet jitter across an IP network can result in packet loss if a buffer overfills. A gap in delivery that's too long can cause a buffer to empty, resulting in no program delivery. Figure 2 shows the results of lost or delayed IP packets on an MPEG video stream.

The larger the buffer, the less likely it will overflow and cause packet loss, but there's a tradeoff. Each TV channel requires a unique MPEG stream. When you switch channels, your TV or set-top box must request the new channel from a server. If you have several TVs all on the same channel, they will share the same stream. But different channels require different streams, which taxes your access network.

Figure 2.  Jitter and long delays in delivery of Ethernet (IP) packets can cause an MPEG buffer to empty, resulting in picture loss. Courtesy of IneoQuest.

Another problem occurs when you change channels. "When you switch channels, the decoder must clear the buffer," said Ofir Michael, director of product management for VoIP and IPTV at Radcom. "In addition, the server must check that the subscriber should get a requested channel." IP packets from the new channel must flow into the buffer and an MPEG decoder must wait for an I-frame from the new channel before it can strip the MPEG video stream from the IP packets and decode the MPEG stream.

Some members of the industry claim that the problem has been solved. "Channel change time isn't an issue," said Patrick Pfeffer, chief network architect at Detecon, during a press conference on the status of IPTV at this year's OFC/NFOEC conference in Anaheim. "Picture quality is better than analog."

Using most protocol analyzers, you can measure the packet jitter and loss in IP networks. The measurements will tell you if the IP network is healthy, but they provide no indication of the final product—video quality. To measure video quality, you need to test the MPEG transport stream as well. An MPEG protocol analyzer can provide a detailed look at a video stream's headers and payloads. "How do you test MPEG-2 transport streams?" describes how MPEG testers analyze the 188-byte packets that carry encoded video and audio (Ref. 5).

The final leg in testing IPTV comes in testing the video quality. Some test companies offer video quality test equipment, but the industry has yet to settle on a standard measurement scale. Several makers of IPTV test equipment use a mean opinion score (MOS) algorithm to measure video quality. MOS is based on human perceptions of video quality, using a scale from 0 to 5. MOS is an subjective test based on people's opinions of video quality. ITU-T Recommendation P.800 defines the procedure for conducting MOS evaluations (Ref. 6). Some companies have developed their own algorithms to produce a consistent method of scoring perceived video quality. But ultimately, the consumer decides.

Brix networks, for example, uses the Brix Video Quality Index. The company's tester measures IP network parameters such as jitter and packet loss, but it also analyzes MPEG transport streams carried in IP packets. The Brix tester doesn't decode the MPEG streams of live video because of copyright issues, noted CTO Hedayet. It can, however, inject its own video streams into the network and compare decoded video against the original.

IneoQuest calls its measure Media Deliver Index (MDI). As the name implies, MDI looks at how well a network delivers MPEG video over an IP network. It measures delay factor and media loss rate. Delay factor is a computed arrival time of each packet. Media loss rate measures the difference between packets received and expected per second. IneoQuest also measures video-stream characteristics such as flow rate in megabits per second.

Video-quality measures attempt to eliminate human subjectivity and provide consistent test results. The industry will eventually need to standardize on one video-quality measure. Regardless of that measure, the success of IPTV depends on the viewing public's perception and its tolerance for errors and delays.