Subscribe Now Subscribe Today
Research Article
 

The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method



Min-Chie Chiu, Long-Jyi Yeh and Che-Jung Hsu
 
Facebook Twitter Digg Reddit Linkedin StumbleUpon E-mail
ABSTRACT

To improve the precision in the recognition process, a new algorithm (the image division method-IDM) is proposed. Recently, to meet various client requisitions, a flexible manufacturing process applied to various products of smaller quantity has become the trend. Even though the above new method can improve deficiency recognition in PCBAs, a huge quantity of samples used in off-line training is still obligatory. Unfortunately, the method is not suitable for a process that includes various products of smaller quantity. Moreover, not all deficiencies can be fully recognized by a single algorithm. To overcome the above drawbacks and increase the recognition rate, a combination of these algorithms in conjunction with a neural network system, which will increase the recognition rate with fewer samples, is proposed. Consequently, results reveal that deficiency recognition can be improved when the IDM in conjunction with other AOI algorithms are linked with a neural network.

Services
Related Articles in ASCI
Search in Google Scholar
View Citation
Report Citation

 
  How to cite this article:

Min-Chie Chiu, Long-Jyi Yeh and Che-Jung Hsu, 2008. The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method. Journal of Applied Sciences, 8: 2814-2824.

DOI: 10.3923/jas.2008.2814.2824

URL: https://scialert.net/abstract/?doi=jas.2008.2814.2824
 

INTRODUCTION

The Printed Circuit Board Assembly (PCBA) is one of the most important components installed inside an electronic product. To promote electronic performance, many electronic elements have been installed onto the compact board; therefore, the Surface Mount Technology (SMT) used to fasten and assemble the Surface Mount Device (SMD) on the printed circuit board`s surface becomes complicated. As expected, because of improper process, various deficiencies often occur. Therefore, quality control which can lower the cost of manufacturing is crucial.

The traditional inspection of PCBA is performed by humans and is time-consuming. This will result in fatigue. Therefore, the Automatic Optical Inspection (AOI), which lower labor costs while maintaining a higher inspection level, is widely used. To successfully identify the deficiencies inside a PCBA, the superiority of the inspection algorithm is important. To improve inspection efficiency with respect to various deficiencies, the AOI is equipped with different algorithms.

The white point statistic method is mainly used to identify the printed character as well as the related print. The image has classified as two values (black and white) by a threshold value which is determined by a specified region with contra colors. By summing up the total number of white points, the deficiency of the opposite element can be picked up. It is found that the white point statistic method is superior in identifying the deficiency of the opposite element due to the white color of the opposite surface element, However, precision will decrease when the ratios of the white point in the testing image are similar to those of the standard image (qualified image). Similarly, when a component is missing, the accuracy recognition will also decrease. To overcome this drawback, a manipulating selection of the image zone is required.

Yeh and Perng (2004) proposed the coefficient correlation method to recognize the deficiency in PCBA`s images. By comparing the averaged gray value and variation between the standard image and the testing image, calculating their relationship and determining a threshold, the deficiency can then be distinguished. The coefficient correlation method is easy to use without presetting a threshold value; in addition, precision will not be influenced by various testing images. However, it will be highly influenced when the light intensity is changed or misalignment occurs.

A total gray error index method is calculated by subtracting all the gray values of the testing image with respect to the standard image and then summing up the absolute variation. The deficiency can be distinguished by using the above index. This method has the advantage of minimal influence with respect to the various images of the PCAB; however, it will be highly influenced when the light intensity is slightly changed or misalignment occurs.

The gray zone division/statistic method divides the gray zones of the standard image and the testing image into five regions-0~49, 50~99, 100~149, 150~199 and 200~255. By using a bar chart to analyze the number of pixels with respect to the five regions, the bar with biggest deviation will be selected as the characteristic zone. A selected threshold value is taken to evaluate the deficiency of the testing image. The gray zone division/statistic method is superior when the light intensity is slightly changed or the location of image is slightly shifted.

The high gray variation/pixel ratio method (T1 method) is similar to the total gray error index method. The threshold (T1) is calculated by dividing the total gray error by the total image points and multiplying it by 1.5. By investigating the number (N1) of pixels in which the gray value is greater than T1, the new indicator used to identify the deficiency is obtained by dividing N1 by total image points.

As investigated above, not all deficiencies can be recognized by using a single algorithm. To overcome this drawback, a new and efficient algorithm-an image division method (IDM)-is proposed. Moreover, for manufacturing processes with various products of smaller quantity, fewer samples are used in off-line training resulting in a drop in precision. Therefore, a more efficient recognition system-a neural network in conjunction with several AOI algorithms-is required and proposed in the research.

CLASSIFICATION OF DEFICIENCIES IN A PCBA

Because of technical improvements in semi-conductor, many electronic elements attached to the PCBA are miniature; therefore, several deficiencies often exist when the PCBA is completed. In order to assure product quality after the re-flow of the PCBA, an AOI is adopted to find the deficient electronic elements which are fastened to the surface of the printed circuit board. The manufacturing process of PCBAs which is shown in Fig. 1 includes a PCB loader, a printer machine, a mount machine, a re-flow and a PCB un-loader.

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 1: The manufacturing process of a PCBA

The general deficiencies which occasionally occurred in AOI are classified as follows:

Wrong element: The misplacement of the electronic element in the PCBA is possible during an incorrect assembly process and will result in a tremendous rise in cost.

Missing element: Because of collision and vibration, a missing electronic element can happen during the assembly process. This will ruin the PCBA`s performance. The related deficient images after the graying process are shown in Fig. 2.

Misalignment: The incorrect placement of elements happens when machine`s precise allocation is insufficient. The deficient images after the graying process are shown in Fig. 3.

Reverse: The influence of reverse is huge for directional electronic elements such as capacitors and integrated circuits.

Opposite: If the bottom of electronic element is turned up, the result will be an incorrect performance in the PCBA. The deficient images after the graying process are shown in Fig. 4.

No solder: The solder for the electronic element is insufficient or terminated when the soldering process is incomplete. The deficient images after the graying process are shown in Fig. 5.

Bridge: An overflow in the soldering process will result in an unwanted connection between electronic elements. The related deficient images after the graying process are shown in Fig. 6.

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 2: The deficiency of missing elements in a PCBA

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 3: The deficiency of misalignment in a PCBA

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 4: The opposite deficiency in a PCBA

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 5: The no solder deficiency in a PCBA

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 6: The bridge deficiency (solder overflow) in a PCBA

AOI INDEX IN A PCBA

In order to evaluate the availability of the AOI algorithm, four kinds of AOI indexes including (1) false-alarm rate, (2) fault-miss rate, (3) incorrect-flaw-classification rate and (4) inspection time are considered. The false-alarm rate is a mis-judgment made by mistaking the qualified product as the unqualified product. A higher false-alarm rate will increase the loading of the product`s inspection and maintenance.

The fault-miss rate is a mis-judgment made by mistaking the unqualified product as the qualified product. A higher fault-miss rate will influence the quality of the product; in addition, the unqualified product which has not been picked up will be sent to the next manufacturing process which may consequently cause a product void. This will lead to a rise in the cost of the product.

The incorrect flaw-classification rate is the mis-classification of a deficiency. For example, a missing deficiency is regarded as a misalignment. A higher incorrect-flaw-classification rate which happens because of an inappropriate inspection algorithm will misrepresent the deficiency`s condition and influence the improvement strategy during the manufacturing and soldering process.

The inspection time in an AOI system is essential. The maximum allowable time is no more than the operation time of the previous equipment.

In this research, the above AOI indexes in conjunction with various algorithms are programmed by JAVA.

IMAGE DIVISION METHOD

When the ratio of the full image to deficient element is small enough, the gray value which is lower than that of the threshold value will lead to an incorrect recognition. In order to improve this drawback, the image division method is adopted by dividing the testing image into several regions. Subsequently, an individual image comparison for each region will be carried out to identify the deficiency by using the specified threshold value. Obviously, the required inspection time will be increased if the number of regions increases. The f, a maximal common factor of the image`s length (m) and width (n), is adopted for dividing the full image. Here, the standard image and inspection image are divided as (Is1, Is2 ,..., Isn) and (It1, It2 ,..., Itn).

The accumulated variations (E1, E2 ,..., En) of the gray value between the standard images and the inspection images at each divided region are:

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(1a)

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(1b)

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(1c)

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(1d)

To find the location of the primary deficiency, the maximum total variation Emax is selected.

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(2)

Under the circumstance of the deficiency located along the edge of the region, to shift the deficiency to the center of the region, information of the deficiency`s center is required in advance. With the coordinates of the deficiency at the down/left corner and the upper/right corner-(xmin, ymin) and (xmax, ymax)- the center (xc, yc) of the deficiency can be obtained.

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(3)

Where:
xmax, ymax = The corresponding (x,y) that causes the maximum variation of (Is(x, y)-It(x, y)) at the zone with Emax
xmin, ymin = The corresponding (x,y) that causes the minimum variation of (Is(x, y)-It(x, y)) at the zone with Emax

After shifting the center of the specified region to the center of the primary deficiency, the mean gray values (µs, µT) and variances Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Methodwith respect to both the standard images and the inspection images are calculated as:

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(4)

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(5)

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(6)

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(7)

By using Eq. 4-7, a new index (I) for the IDM (image division method) is defined as:

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(8)

NEURAL NETWORK MODEL

The concept of the neural network (NN) was formalized by McCulloch and Pitts (1943) who proposed a mathematical model of neural cells (MP model) and develop the original digital model of neural cells in 1943. Hebb (1949) proposed a learning algorithm in the neural network which initiated much research work in learning algorithms. Rosenblatt (1958) proposed the first neural network-the perception model. By using a double-layer perception neural network, human vision can be imitated successfully. Twenty years later, because of the improper usage of neural network on XOR problem, neural network has been criticized by Minsky and Papert (1969). Therefore, further research on the NN was temporarily stopped and transferred to artificial intelligent (AI). Hopfied (1982) proposed both the hopfied neural network and the back propagation network. These are able to overcome many of the problems never solved before. Thereafter, the NN has been widely developed and applied in various fields.

Back-propagation network: BPN is one of the most popular models applied in various fields. The theory and algorithm has been clearly defined by the propagation rule (i.e., generalized delta learning rule) proposed by Rumelhart et al. (1986).

BPNs are composed of an input layer, multiple hidden layers and an output layer. The structure of a BPN is shown in Fig. 7.

The mathematic form is expressed as:

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(9a)

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(9b)

where, F is the activation function that is non-linear.

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 7: The structure of a BPN

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 8: The gradient descent method

Xj is the jth input,Yi is the ith output, Wij is the weight and bj is the bias value.

The principle of the BPN is to use the Gradient Descent Method (GDM) to minimize the error function (E) during the NN`s learning process.

As indicated in Fig. 8, by using the GDM, the delta rule can be deduced which minimizes the difference between the real value and the predicted value using the consecutive correction. The related error function is:

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(10)

where, dk is the targeted value at the kth neural cell (output layer) and yk is the output value at the kth neural cell (output layer).

The weights of the BPN will be self-adjusted when each of the training is inputted into the system. The span of adjustment illustrated below is proportional to the first derivation of an error function with respect to weights.

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 9: The sigmoid function

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 10: The hyper-tangent function

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
(11)

where, ? is the learning rate of the BPN.

The BPN includes two processes-the forwarding propagation and the backward propagation. By inputting a given sample into the BPN, the weights (Wij) and related bias (bj) can be self-adjusted using errors that occur during the backward propagation process.

After the BPN training process is completed, the weights and bias will be stabilized. The predicted value can thus be calculated using the forwarding propagation.

The more training samples inputted into the BPN, the higher the precision in the BPN system. The learning rate ? is essential during training process. A higher learning rate ? resulting in a faster convergence speed will cause a loss of precision. On the other hand, a lower learning rate ? resulting in increased precision will slow down the convergence speed. The number of hidden layers depends on the designer`s experience. In this study, a simple one hidden layer is adopted for the recognition of deficiencies in an AOI system.

The activation function (F) located inside the hidden layers plays an essential role that a discontinuity point can approach after the transformation of the activation function (F). As indicated in Fig. 9 and 10, either the sigmoid function or the hyper-tangent function is adopted as the activation function (F). The latter, which has a higher variety of slope rate and is more efficient in convergence, is thus adopted in this research.

Processing of back-propagation network: To establish a relationship between the BPN and the AOI algorithm, a commercial package-NeuroSolutions 5.0- is adopted in this research.

As indicated in Fig. 11, the neural builder is selected.

To simplify the BPN model, one-layer is selected in the hidden layer option as shown in Fig. 12.

The resultant figure run in NeuroSolutions 5.0 is shown in Fig. 13.

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 11: A commercial package - NeuroSolutions 5.0

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 12: Selection of the hidden layer

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 13: Result of BPN in NeuroSolutions 5.0

RESULTS

Results: In this research, two hundred inspection pictures used in practical PCBA`s inspection process have been adopted. Here, one hundred and seventy-four pictures are qualified. Ten pictures are missing components; eight are misaligned; eight are opposite. The related images and various AOI algorithms can be obtained and assigned using the interface window programmed by the JAVA program run in a notebook (Intel Pentium 1.5 GHz and 768 MB RAM). The selected range of images is 640x48 pixels. The flow diagram of the AOI is shown in Fig. 14.

Recognition with respect to individual AOI algorithms: After using various AOI algorithms, the related results with respect to each algorithm are shown in Table 1-6.

Recognition by the BPN in conjunction with various AOI algorithms
Method I: BPN in conjunction with five kinds of AOI algorithms: Five kinds of AOI algorithms, including (1) the coefficient correlation method, (2) the total gray error index method, (3) the gray zone division/statistic method, (4) the white point statistic method and (5) the high gray variation/pixel ratio method (T1 method), are adopted in conjunction with the BPN. The result of recognized deficiencies is shown in Table 7.

Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method
Fig. 14: The flow diagram of AOI in PCBA

Table 1: Recognition result for the coefficient correlation method
Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method

Method II: BPN in conjunction with six kinds of AOI algorithms: To improve the recognition precision, the image division method is added to the above BPN system. Therefore, six kinds of AOI algorithms, including (1) the coefficient correlation method, (2) the total gray error index method, (3) the gray zone division/statistic method, (4) the white point statistic method, (5) the high gray variation/pixel ratio method (T1 method) and (6) the image division method, are grouped in conjunction with the BPN.

Table 2: Recognition result for the total gray error index method
Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method

Table 3: Recognition result for the gray zone division/statistic method
Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method

Table 4: Recognition result for the white point statistic method
Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method

Table 5: Recognition result for the high gray variation/pixel ratio method (T1 method)
Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method

Table 6: Recognition result for the IDM
Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method

Table 7: Recognition result using the BPN in conjunction with five kinds of algorithms (Method I)
Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method

Table 8: Recognition result using a BPN in conjunction with six kinds of algorithms (Method II)
Image for - The Deficiency Recognition in PCBA`s Automatic Optical Inspection System by Using Back-Propagation Network Method

The result of recognized deficiencies is shown in Table 8.

DISCUSSION

Recognition with respect to individual AOI algorithm: As indicated in Table 1, most of the qualified images have index values which are larger than 0.9970 when the coefficient correlation method is used; therefore, an assumption is made that the image will be qualified when the index is larger than 0.9970. It is obvious that the coefficient correlation method has an efficient false-alarm rate. However, three kinds of deficiencies are diversely allocated. The recognition ability of the incorrect-flaw-classification rate is insufficient.

As indicated in Table 2, both of the qualified and unqualified images are diversely distributed along the index axis when the total gray error index method is applied in the AOI. It is possible that the number of samples is insufficient.

As indicated in Table 3, the character of the qualified image can be roughly identified when the gray zone division/statistic method is used. However, the efficiency of the incorrect-flaw-classification rate is insufficient because of the diverse distribution of the deficiencies on the index`s axis.

As indicated in Table 4, the distinction between the qualified and unqualified images is not very clear when using the white point statistic method. Moreover, the efficiency of the incorrect-flaw-classification rate is insufficient because of the diverse distribution of the deficiencies on index`s axis.

As indicated in Table 5, the deficiency of misalignment is grouped at the index 0.1~0.12 when using the high gray variation/pixel ratio method. However, the distinction between the qualified and unqualified images is not clear. Therefore, the efficiency of the false-alarm rate is insufficient.

As indicated in Table 6, most of the qualified images have index values which are larger than 0.96 when the IDM is used. It is obvious that the efficiency of the false-alarm rate in the IDM is superior to the coefficient correlation method. Consequently, the image division method proposed in this paper will promote the efficiency of the false-alarm rate during the AOI process.

Recognition by the BPN in conjunction with various AOI algorithms: As discussed above, not all kinds of deficiencies can be fully recognized by a single algorithm. Therefore, several algorithms are grouped in conjunction with the BPN to recognize the deficiencies in the PCBA`s image. As indicated in Table 7 and 8, the false-alarm rate of the Method II (with six AOI algorithms) is higher than the Method I (with five AOI algorithms).

CONCLUSION

Even though traditional AOI methods (with the coefficient correlation method, the total gray error index method, the gray zone division/statistic method, the white point statistic method and the high gray variation/pixel ratio method) have been substantially applied in the AOI process, the deficiency recognition is still insufficient. To improve the efficiency, a new algorithm (IDM) is adopted in the AOI process.

As the deficiency recognition by using a single AOI algorithm is still inadequate, several algorithms are grouped in conjunction with the BPN to recognize the deficiency of the PCBA`s image. Experimental results reveal that the false-alarm rate of Method II (with the coefficient correlation method, the total gray error index method, the gray zone division/statistic method, the white point statistic method, the high gray variation/pixel ratio method and the image division method) is higher than them Method I (with the coefficient correlation method, the total gray error index method, the gray zone division/statistic method, the white point statistic method and the high gray variation/pixel ratio method). This means if more numbers and algorithms are grouped in the BPN, a higher deficiency recognition can be reached.

Consequently, the BPN system in conjunction with various AOI algorithms can efficiently and quickly improve the precision of deficiency recognition in the PCBA`s inspection.

REFERENCES

1:  Hebb, D.O., 1949. The Organization of Behavior:A Neuropsychological Theory. 1st Edn., Wiley, New York, ISBN: 0805843000
Direct Link  |  

2:  Hopfield, J.J., 1982. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA., 79: 2554-2558.
Direct Link  |  

3:  McCulloch, W.S. and W. Pitts, 1943. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys., 5: 115-133.
CrossRef  |  Direct Link  |  

4:  Minsky, M.L. and S.A. Papert, 1969. Perceptrons. 1st Edn., MIT Press, Cambridge, MA, ISBN:026263113

5:  Yeh, C. and D.B. Perng, 2004. A reference standard of defect compensation for leather transactions. Int. J. Adv. Manuf. Technol., 25: 1197-1204.
CrossRef  |  Direct Link  |  

6:  Rosenblatt, F., 1958. The perceptron: A probabilistic model for information storage and organization in the brain. Psych. Rev., 65: 386-408.
CrossRef  |  PubMed  |  Direct Link  |  

7:  Rumelhart, D.E., J.L. McMclelland and C. Asanuma, 1986. Parallel Distributed Processing: Explorations in the Microstructure of Cognition. 1st Edn., MIT Press, Cambridge, MA, ISBN:0262181231

©  2021 Science Alert. All Rights Reserved