The Printed Circuit Board Assembly (PCBA) is one of the most important
components installed inside an electronic product. To promote electronic
performance, many electronic elements have been installed onto the compact
board; therefore, the Surface Mount Technology (SMT) used to fasten and
assemble the Surface Mount Device (SMD) on the printed circuit board`s
surface becomes complicated. As expected, because of improper process,
various deficiencies often occur. Therefore, quality control which can
lower the cost of manufacturing is crucial.
The traditional inspection of PCBA is performed by humans and is time-consuming.
This will result in fatigue. Therefore, the Automatic Optical Inspection
(AOI), which lower labor costs while maintaining a higher inspection level,
is widely used. To successfully identify the deficiencies inside a PCBA,
the superiority of the inspection algorithm is important. To improve inspection
efficiency with respect to various deficiencies, the AOI is equipped with
The white point statistic method is mainly used to identify the printed
character as well as the related print. The image has classified as two
values (black and white) by a threshold value which is determined by a
specified region with contra colors. By summing up the total number of
white points, the deficiency of the opposite element can be picked up.
It is found that the white point statistic method is superior in identifying
the deficiency of the opposite element due to the white color of the opposite
surface element, However, precision will decrease when the ratios of the
white point in the testing image are similar to those of the standard
image (qualified image). Similarly, when a component is missing, the accuracy
recognition will also decrease. To overcome this drawback, a manipulating
selection of the image zone is required.
Yeh and Perng (2004) proposed the coefficient correlation method to recognize
the deficiency in PCBA`s images. By comparing the averaged gray value
and variation between the standard image and the testing image, calculating
their relationship and determining a threshold, the deficiency can then
be distinguished. The coefficient correlation method is easy to use without
presetting a threshold value; in addition, precision will not be influenced
by various testing images. However, it will be highly influenced when
the light intensity is changed or misalignment occurs.
A total gray error index method is calculated by subtracting all the
gray values of the testing image with respect to the standard image and
then summing up the absolute variation. The deficiency can be distinguished
by using the above index. This method has the advantage of minimal influence
with respect to the various images of the PCAB; however, it will be highly
influenced when the light intensity is slightly changed or misalignment
The gray zone division/statistic method divides the gray zones of the
standard image and the testing image into five regions-0~49, 50~99, 100~149,
150~199 and 200~255. By using a bar chart to analyze the number of pixels
with respect to the five regions, the bar with biggest deviation will
be selected as the characteristic zone. A selected threshold value is
taken to evaluate the deficiency of the testing image. The gray zone division/statistic
method is superior when the light intensity is slightly changed or the
location of image is slightly shifted.
The high gray variation/pixel ratio method (T1 method) is similar to
the total gray error index method. The threshold (T1) is calculated by
dividing the total gray error by the total image points and multiplying
it by 1.5. By investigating the number (N1) of pixels in which the gray
value is greater than T1, the new indicator used to identify the deficiency
is obtained by dividing N1 by total image points.
As investigated above, not all deficiencies can be recognized by using
a single algorithm. To overcome this drawback, a new and efficient algorithm-an
image division method (IDM)-is proposed. Moreover, for manufacturing processes
with various products of smaller quantity, fewer samples are used in off-line
training resulting in a drop in precision. Therefore, a more efficient
recognition system-a neural network in conjunction with several AOI algorithms-is
required and proposed in the research.
CLASSIFICATION OF DEFICIENCIES IN A PCBA
Because of technical improvements in semi-conductor, many electronic
elements attached to the PCBA are miniature; therefore, several deficiencies
often exist when the PCBA is completed. In order to assure product quality
after the re-flow of the PCBA, an AOI is adopted to find the deficient
electronic elements which are fastened to the surface of the printed circuit
board. The manufacturing process of PCBAs which is shown in Fig.
1 includes a PCB loader, a printer machine, a mount machine, a re-flow
and a PCB un-loader.
||The manufacturing process of a PCBA
The general deficiencies which occasionally occurred in AOI are classified
Wrong element: The misplacement of the electronic element in the
PCBA is possible during an incorrect assembly process and will result
in a tremendous rise in cost.
Missing element: Because of collision and vibration, a missing
electronic element can happen during the assembly process. This will ruin
the PCBA`s performance. The related deficient images after the graying
process are shown in Fig. 2.
Misalignment: The incorrect placement of elements happens when
machine`s precise allocation is insufficient. The deficient images after
the graying process are shown in Fig. 3.
Reverse: The influence of reverse is huge for directional electronic
elements such as capacitors and integrated circuits.
Opposite: If the bottom of electronic element is turned up, the
result will be an incorrect performance in the PCBA. The deficient images
after the graying process are shown in Fig. 4.
No solder: The solder for the electronic element is insufficient
or terminated when the soldering process is incomplete. The deficient
images after the graying process are shown in Fig. 5.
Bridge: An overflow in the soldering process will result in an
unwanted connection between electronic elements. The related deficient
images after the graying process are shown in Fig. 6.
||The deficiency of missing elements in a PCBA
||The deficiency of misalignment in a PCBA
||The opposite deficiency in a PCBA
||The no solder deficiency in a PCBA
||The bridge deficiency (solder overflow) in a PCBA
AOI INDEX IN A PCBA
In order to evaluate the availability of the AOI algorithm, four kinds
of AOI indexes including (1) false-alarm rate, (2) fault-miss rate, (3)
incorrect-flaw-classification rate and (4) inspection time are considered. The false-alarm rate is a mis-judgment made by mistaking the qualified
product as the unqualified product. A higher false-alarm rate will increase
the loading of the product`s inspection and maintenance.
The fault-miss rate is a mis-judgment made by mistaking the unqualified
product as the qualified product. A higher fault-miss rate will influence
the quality of the product; in addition, the unqualified product which
has not been picked up will be sent to the next manufacturing process
which may consequently cause a product void. This will lead to a rise
in the cost of the product.
The incorrect flaw-classification rate is the mis-classification of a
deficiency. For example, a missing deficiency is regarded as a misalignment.
A higher incorrect-flaw-classification rate which happens because of an
inappropriate inspection algorithm will misrepresent the deficiency`s
condition and influence the improvement strategy during the manufacturing
and soldering process.
The inspection time in an AOI system is essential. The maximum allowable
time is no more than the operation time of the previous equipment.
In this research, the above AOI indexes in conjunction with various algorithms
are programmed by JAVA.
IMAGE DIVISION METHOD
When the ratio of the full image to deficient element is small enough,
the gray value which is lower than that of the threshold value will lead
to an incorrect recognition. In order to improve this drawback, the image
division method is adopted by dividing the testing image into several
regions. Subsequently, an individual image comparison for each region
will be carried out to identify the deficiency by using the specified
threshold value. Obviously, the required inspection time will be increased
if the number of regions increases. The f, a maximal common factor of
the image`s length (m) and width (n), is adopted for dividing the full
image. Here, the standard image and inspection image are divided as (Is1,
Is2 ,..., Isn) and (It1, It2
The accumulated variations (E1, E2 ,..., En)
of the gray value between the standard images and the inspection images
at each divided region are:
To find the location of the primary deficiency, the maximum total variation
Emax is selected.
Under the circumstance of the deficiency located along the edge of the
region, to shift the deficiency to the center of the region, information
of the deficiency`s center is required in advance. With the coordinates
of the deficiency at the down/left corner and the upper/right corner-(xmin,
ymin) and (xmax, ymax)- the center (xc,
yc) of the deficiency can be obtained.
||The corresponding (x,y) that causes the maximum variation
of (Is(x, y)-It(x, y)) at the zone with Emax
||The corresponding (x,y) that causes the minimum variation of (Is(x,
y)-It(x, y)) at the zone with Emax
After shifting the center of the specified region to the center of the
primary deficiency, the mean gray values (µs, µT)
and variances with
respect to both the standard images and the inspection images are calculated
By using Eq. 4-7, a new index (I) for the IDM (image
division method) is defined as:
NEURAL NETWORK MODEL
The concept of the neural network (NN) was formalized by McCulloch and Pitts
(1943) who proposed a mathematical model of neural cells (MP model) and develop
the original digital model of neural cells in 1943. Hebb (1949) proposed a learning
algorithm in the neural network which initiated much research work in learning
algorithms. Rosenblatt (1958) proposed the first neural network-the perception
model. By using a double-layer perception neural network, human vision can be
imitated successfully. Twenty years later, because of the improper usage of
neural network on XOR problem, neural network has been criticized by Minsky
and Papert (1969). Therefore, further research on the NN was temporarily stopped
and transferred to artificial intelligent (AI). Hopfied (1982) proposed both
the hopfied neural network and the back propagation network. These are able
to overcome many of the problems never solved before. Thereafter, the NN has
been widely developed and applied in various fields.
Back-propagation network: BPN is one of the most popular models
applied in various fields. The theory and algorithm has been clearly defined
by the propagation rule (i.e., generalized delta learning rule) proposed
by Rumelhart et al. (1986).
BPNs are composed of an input layer, multiple hidden layers and an output
layer. The structure of a BPN is shown in Fig. 7.
The mathematic form is expressed as:
where, F is the activation function that is non-linear.
||The structure of a BPN
|| The gradient descent method
Xj is the jth input,Yi is the ith output, Wij
is the weight and bj is the bias value.
The principle of the BPN is to use the Gradient Descent Method (GDM)
to minimize the error function (E) during the NN`s learning process.
As indicated in Fig. 8, by using the GDM, the delta
rule can be deduced which minimizes the difference between the real value
and the predicted value using the consecutive correction. The related
error function is:
where, dk is the targeted value at the kth neural cell (output
layer) and yk is the output value at the kth neural cell (output
The weights of the BPN will be self-adjusted when each of the training
is inputted into the system. The span of adjustment illustrated below
is proportional to the first derivation of an error function with respect
|| The sigmoid function
|| The hyper-tangent function
where, ? is the learning rate of the BPN.
The BPN includes two processes-the forwarding propagation and the backward
propagation. By inputting a given sample into the BPN, the weights (Wij)
and related bias (bj) can be self-adjusted using errors that
occur during the backward propagation process.
After the BPN training process is completed, the weights and bias will
be stabilized. The predicted value can thus be calculated using the forwarding
The more training samples inputted into the BPN, the higher the precision
in the BPN system. The learning rate ? is essential during training
process. A higher learning rate ? resulting in a faster convergence
speed will cause a loss of precision. On the other hand, a lower learning
rate ? resulting in increased precision will slow down the convergence
speed. The number of hidden layers depends on the designer`s experience.
In this study, a simple one hidden layer is adopted for the recognition
of deficiencies in an AOI system.
The activation function (F) located inside the hidden layers plays
an essential role that a discontinuity point can approach after the transformation
of the activation function (F). As indicated in Fig.
9 and 10, either the sigmoid function or the hyper-tangent
function is adopted as the activation function (F). The latter, which
has a higher variety of slope rate and is more efficient in convergence,
is thus adopted in this research.
Processing of back-propagation network: To establish a relationship
between the BPN and the AOI algorithm, a commercial package-NeuroSolutions
5.0- is adopted in this research.
As indicated in Fig. 11, the neural builder is selected.
To simplify the BPN model, one-layer is selected in the hidden layer
option as shown in Fig. 12.
The resultant figure run in NeuroSolutions 5.0 is shown in Fig.
|| A commercial package - NeuroSolutions 5.0
|| Selection of the hidden layer
|| Result of BPN in NeuroSolutions 5.0
Results: In this research, two hundred inspection pictures used
in practical PCBA`s inspection process have been adopted. Here, one hundred
and seventy-four pictures are qualified. Ten pictures are missing components;
eight are misaligned; eight are opposite. The related images and various
AOI algorithms can be obtained and assigned using the interface window
programmed by the JAVA program run in a notebook (Intel Pentium 1.5 GHz
and 768 MB RAM). The selected range of images is 640x48 pixels. The flow
diagram of the AOI is shown in Fig. 14.
Recognition with respect to individual AOI algorithms: After using
various AOI algorithms, the related results with respect to each algorithm
are shown in Table 1-6.
Recognition by the BPN in conjunction with various AOI algorithms
Method I: BPN in conjunction with five kinds of AOI algorithms:
Five kinds of AOI algorithms, including (1) the coefficient correlation
method, (2) the total gray error index method, (3) the gray zone division/statistic
method, (4) the white point statistic method and (5) the high gray variation/pixel
ratio method (T1 method), are adopted in conjunction with the BPN. The
result of recognized deficiencies is shown in Table 7.
|| The flow diagram of AOI in PCBA
||Recognition result for the coefficient correlation method
Method II: BPN in conjunction with six kinds of AOI algorithms:
To improve the recognition precision, the image division method is added
to the above BPN system. Therefore, six kinds of AOI algorithms, including
(1) the coefficient correlation method, (2) the total gray error index
method, (3) the gray zone division/statistic method, (4) the white point
statistic method, (5) the high gray variation/pixel ratio method (T1 method)
and (6) the image division method, are grouped in conjunction with the
||Recognition result for the total gray error index method
||Recognition result for the gray zone division/statistic
||Recognition result for the white point statistic method
||Recognition result for the high gray variation/pixel
ratio method (T1 method)
||Recognition result for the IDM
||Recognition result using the BPN in conjunction with
five kinds of algorithms (Method I)
||Recognition result using a BPN in conjunction with six
kinds of algorithms (Method II)
The result of recognized deficiencies is shown in Table
Recognition with respect to individual AOI algorithm: As indicated
in Table 1, most of the qualified images have index
values which are larger than 0.9970 when the coefficient correlation method
is used; therefore, an assumption is made that the image will be qualified
when the index is larger than 0.9970. It is obvious that the coefficient
correlation method has an efficient false-alarm rate. However, three kinds
of deficiencies are diversely allocated. The recognition ability of the
incorrect-flaw-classification rate is insufficient.
As indicated in Table 2, both of the qualified and
unqualified images are diversely distributed along the index axis when
the total gray error index method is applied in the AOI. It is possible
that the number of samples is insufficient.
As indicated in Table 3, the character of the qualified
image can be roughly identified when the gray zone division/statistic
method is used. However, the efficiency of the incorrect-flaw-classification
rate is insufficient because of the diverse distribution of the deficiencies
on the index`s axis.
As indicated in Table 4, the distinction between the
qualified and unqualified images is not very clear when using the white
point statistic method. Moreover, the efficiency of the incorrect-flaw-classification
rate is insufficient because of the diverse distribution of the deficiencies
on index`s axis.
As indicated in Table 5, the deficiency of misalignment
is grouped at the index 0.1~0.12 when using the high gray variation/pixel
ratio method. However, the distinction between the qualified and unqualified
images is not clear. Therefore, the efficiency of the false-alarm rate
As indicated in Table 6, most of the qualified images
have index values which are larger than 0.96 when the IDM is used. It
is obvious that the efficiency of the false-alarm rate in the IDM is superior
to the coefficient correlation method. Consequently, the image division
method proposed in this paper will promote the efficiency of the false-alarm
rate during the AOI process.
Recognition by the BPN in conjunction with various AOI algorithms: As
discussed above, not all kinds of deficiencies can be fully recognized by a
single algorithm. Therefore, several algorithms are grouped in conjunction with
the BPN to recognize the deficiencies in the PCBA`s image. As indicated in Table
7 and 8, the false-alarm rate of the Method II (with six
AOI algorithms) is higher than the Method I (with five AOI algorithms).
Even though traditional AOI methods (with the coefficient correlation
method, the total gray error index method, the gray zone division/statistic
method, the white point statistic method and the high gray variation/pixel
ratio method) have been substantially applied in the AOI process, the
deficiency recognition is still insufficient. To improve the efficiency,
a new algorithm (IDM) is adopted in the AOI process.
As the deficiency recognition by using a single AOI algorithm is still
inadequate, several algorithms are grouped in conjunction with the BPN
to recognize the deficiency of the PCBA`s image. Experimental results
reveal that the false-alarm rate of Method II (with the coefficient correlation
method, the total gray error index method, the gray zone division/statistic
method, the white point statistic method, the high gray variation/pixel
ratio method and the image division method) is higher than them Method
I (with the coefficient correlation method, the total gray error index
method, the gray zone division/statistic method, the white point statistic
method and the high gray variation/pixel ratio method). This means if
more numbers and algorithms are grouped in the BPN, a higher deficiency
recognition can be reached.
Consequently, the BPN system in conjunction with various AOI algorithms
can efficiently and quickly improve the precision of deficiency recognition
in the PCBA`s inspection.