Solder-joint study shows defect levels remain above targets

In 1999, Stig Oresjo, then of Agilent Technologies, conducted a major study of solder-joint defects on printed-circuit boards (PCBs). The study, which at the time provided the most definitive data on the subject, involved 15 companies and more than 1 billion solder joints. Oresjo concluded that although companies claimed defect levels in the range of 75 to 150 defects per million opportunities (DPMO), the reality was 5 to 10 times that high.

Now retired, Oresjo has returned to the well one more time to see if the situation has changed. In a new analysis, he tabulated 3.7 billion joints—almost four times as many as in the previous study—from 1100 different board types at 14 companies, again including both OEMs and contract manufacturers.

Whereas the first study included companies only from the US and Western Europe, this time Oresjo studied two companies in the US, four in Europe, and eight in Asia. His results suggest that although board quality has improved in the past eight years despite increased complexity, defect levels still do not live up to manufacturers’ claims.

Oresjo described the new study this way: “To reduce the number of random variables, we restricted the sample to companies using [Agilent’s] 5DX x-ray inspection systems, as well as datalogging software that we could read easily, thereby facilitating the analysis. To encourage participation, I agreed not to reveal company names. Boards under test ranged from medium to high complexity.

“We excluded low-complexity boards because we wanted boards that routinely underwent x-ray inspection, often not practical at low complexities. Production volumes covered the entire range, as did product mix.

“We considered every solder joint that was inspected to be a defect opportunity, ignoring any joint that bypassed that step. Every defect that the 5DX caught and that was later verified as a defect counted toward the total. I did exclude data from any board that went through the inspection process several times in quick succession and any board that was inspected during program development rather than production, and limited each company to the most recent 500,000,000 joints in the data they provided. I developed software in Visual Basic to read the data and convert it to a Microsoft Access database.

“If the inspection step flagged a defect, but it was never verified as real and repaired, we removed the entire board from the analysis. Only about 1% of the boards fell into a category where the data were not used. We didn’t look at false calls, only verified defects in relation to the total number of defect opportunities.

“Even achieving defect levels similar to those in the earlier study would have been an accomplishment. Today’s boards are much more complex. Also, manufacturers are much more cost-conscious, and when belts tighten, test tends to take a big hit. But the situation has improved more than that.

“In the first study, we found an aggregate average of 1083 DPMO. This time, defects totaled 572 DPMO. We broke the data down further into 327 DPMO for actual defects that underwent repair and 245 DPMO for process indicators.

“We differentiated between the two based on the repair operator’s judgment on whether to fix a problem before sending the board on to the next step. Process indicators might not require immediate action, but if there are enough of them, they should trigger adjustments to the process.

“Insufficient solder, for example, can be a process indicator or a defect. It may not cause a board to fail, but it can still compromise the board’s reliability in the field. Expansion and contraction caused by powering up and powering down the board can make the joint fail, as can excessive handling. Reducing or eliminating this problem might mean cleaning the solder mask. Another reason for tracking process indicators is that, in our experience, manufacturers who experience a lot of them also tend to have a higher rate of repairable defects.”

Although Oresjo expressed no doubt that board quality has improved in recent years, he pointed out that the processes have changed somewhat, making the data more difficult to compare directly with the earlier study.

“More companies are using visual inspection than did in 1999,” he commented. “In Asia, that inspection might be more manual than in the US and Western Europe where AOI [automated optical inspection] is more common, but either way, some defects that would have shown up before would not have survived as far as the x-ray system this time. Visual inspection catches the lowest-hanging fruit, so to speak.”

When asked what surprising results showed up in the data, Oresjo responded, “We found defects on all kinds of components and solder joints—gull wings, BGAs [ball-grid arrays], connectors, resistors, and so on. Some of them were naturally hidden from optical inspection, but others should have been visible to optical inspection. We would have expected those to be caught at the earlier step.”

Despite the data generated by the study, manufacturers commonly still quote much lower defect rates. Contract manufacturers often stick with the 75 DPMO that they have been touting for years. When asked about the discrepancy, Oresjo explained, “The lower numbers can be supported if you realize that they are calculated under best-case conditions. Companies use the most stable boards in production, the best-controlled processes, the best production days, and so on. If all contract manufacturers signed up to the higher numbers, they would show up more frequently. But companies look at the others’ claims and realize that they have to quote numbers obtained under similar conditions in order to compete. You can probably find the more realistic numbers internally within OEMs, because there they are used to guide engineers and managers to product and process improvements.”

How do companies react when they see the study results? Oresjo put it this way: “Not as many people have seen this data as saw the previous study. But the response is generally the same—a room full of smiles and nodding heads. Industry people have a very good idea about the real-world situation. They use the data during manufacturing and during development to improve both processes and products. Awareness of real defect levels is very important. If defects escape to customers, they cause warranty and other field failures, incurring both high costs for correction and a loss of goodwill.”