The geometric characteristics of a TV set can be only measured and adjusted
by using a visual closed-loop control structure. We propose to automate this
procedure in the production line at the manufacturing plants. Traditional methods
used employ an expert human operator for tuning the geometric parameters of
the television set. Naturally this is both time consuming and cannot be performed
in real-time. We have already proposed and published a novel fuzzy test pattern
(Toosizadeh and Peiravi, 2005) and an adaptive alignment
algorithm for the displays (Peiravi and Toosizadeh, 2008)
which we have used in this research and have implemented the present system
in real time.
Advances made in the technology behind the manufacturing of television sets
have made it possible to change the screen characteristics by modifying some
digital values instead of tuning a few potentiometers. These digital values
are stored in an EEPROM and can be changed simply by a service remote controller
or the internal serial bus of the television set that is referred to as the
I2C Bus (Lindahl et al., 1994). Some
of the advantages of this new technology are as follows:
||Reducing the number of assembling components to be assembled resulting
in an increased reliability for the overall system
||Reducing the time needed to perform adjustments of the screen parameters
||Enhancing the quality of the television display
Therefore, these factors increase the production efficiency and reduce the
overall cost of television manufacturing. A description on how this digital
technology works internally in television sets and the definitions of some of
the geometric parameters has been presented by Suckle (1988).
The geometric adjustment of a CRT display is usually done in two steps. In
the first step that is known as the Integrated Tube Component (ITC) process,
an operator has to adjust the colour purity rings and deflection yokes on the
CRT. In the second step, an operator has to adjust the display circuitry using
various potentiometers on the displays circuit board (Yerem,
2001). In this study we consider the second step for adjustment of the geometric
parameters of the television set. The adjustment of geometric parameters by
human operators seems to be an easy task. However, it is a very tedious task
if it should be performed continuously over time. In addition, constantly looking
at a television screen from a short distance is very harmful for the human eyes.
Moreover, a few of the screen parameters cannot be accurately adjusted by a
human operator and such adjustments are highly dependent on the operators experience
and personal judgment (e.g., S-Correction which is vertical linearity and RGB
parameters). Continuous exposure to the X-rays produced in the vicinity of the
CRT is also very dangerous for the human operators.
In this study, we present a novel approach for the automation of this calibration
procedure in order to increase the accuracy and the uniformity of the adjustment
of the parameters. We shall refer to this automation system as the auto-alignment
system. Such an automation can increase the rate of production and the quality
of the displays coming out of a television manufacturing factory. A discussion
about the image quality model and the concept of the Image Quality Circle (IQC)
has been represented by Engeldrum (1999). An auto-alignment
system can be used to apply objective image quality and subjective image quality
(Engeldrum, 1999) to adjust physical image parameters
of the television set.
Earlier study carried out in order to simplify and automate the adjustment
of the geometric characteristics of the CRT display are as follows: 2D and 3D
models of the CRT have been used in systems presented by Webb
et al. (1993) and Webb and Kern (1996) for
transforming coordinate systems in an automated video monitor alignment system.
These systems need the CRT model and a high-precision assembly line conveyor
system with low tolerance fixturing. In another patented study, an inspection
system that uses multiple high-resolution inspection cameras, in conjunction
with a single stereoscopic reference camera have been introduced by Fridge
(1997). A more recent work on the ITC measurement of the CRT display has
been proposed by Chuang et al. (1999). A method
for adjusting image geometry in video display monitor by the use of a human
operator feedback through an input device has been patented by Devine
(1999). A complicated system with multiple cameras and photodiodes for testing
and aligning a CRT which can perform a series of required tests has been presented
by Buckley et al. (1999, 2001).
Even more recently, Webb and Simpson (2001, 2002)
patented an apparatus and a method by using a host computer processor and the
memory associated with the video graphics controller to dynamically adjust video
images on the CRT screen in order to reduce the costs of CRT monitors manufacturing
without the limitation on dynamic alignment techniques.
All of the aforementioned systems have complexities such as the need
to use multiple cameras or one camera with a fixture and some predefined
information such as 3D model of the CRT. Therefore, they are computationally
intensive. We have presented a novel approach to reduce these complexities.
THE PROPOSED SCHEME
In this study we present the results of the implementation of a machine-vision
based control system by an image-based structure (Hutchinson
et al., 1996; Shen, 2000) through relative
measurements that require no camera calibration. This visual control system
can be used as an auto-alignment system for television sets (Fig.
1). Here, we will discuss the construction of the present novel auto-alignment
system for the television.
In this system the pattern generator generates a suitable pattern on
the TV screen to measure each geometric attribute. A vision system placed
in the feedback loop transfers the images of the TV screen through a frame
grabber. This is input to a measuring algorithm to gauge the geometric
characteristics of the television screen being adjusted. A digital controller
compares the measured values with corresponding set points to form the
control signal that is used as an appropriate adjustment signal. This
process is repeated over and over until the error between the measured
and the desired values becomes less than a predefined tolerable value.
Block diagram of the closed-loop system to automate the alignment
of TV screen, d: Set points of the geometric parameters of the TV
screen, m: Measured values of geometric parameters of the TV screen,
u: Control signal to adjust parameters, g: Visual geometric characteristics
of the TV screen, p: Generated pattern to measure TV screen attribute,
n1: Ambient light noise, n2: CCD noise, lens
disturbance, CCD resolution and so on and n3: Analog to
digital converter noise
Our television auto-alignment system contains a Panasonic colour video
camera of 768×576 resolution and a 450 MHz AMD-K6II PC with two display
graphics cards, one used as a pattern generator and the other one used
as a user interface. Also, a Pinnacle PC-TV video capture card with the
maximum resolution of 768x576 has been installed in the PC as a frame
grabber. The PC produces a suitable pattern on the television screen being
adjusted through the specified graphics card and then the video camera
transfers the image of the television screen through the frame grabber
to the measurement algorithm that runs on the PC. The pattern used and
the adjustment parameters are properly considered in the measurement algorithm.
The digital controller that is also running on the PC uses the measured
values and respective set points to generate the adjustment control signal.
An infrared transmitter that has been connected to the PCs parallel port
sends the specified adjustment signal to the television being adjusted.
This adjustment process is repeated over and over until a tolerable adjustment
of the geometric characteristics is achieved. This process of repeated
operations usually takes around four iterations to be completed that is
around 90 sec on our platform.
DIFFICULTIES IN SYSTEM DESIGN
Some of the difficulties that we encountered in the design of the proposed
automatic adjustment system for the geometric parameters of a CRT are
||The mutual effect of some of the geometric characteristics
yields a multi-input multi-output plant that is difficult to control
||The non-linear relationship between the input signals and the measured
outputs that represent the geometric characteristics of the CRT due to the
screen not being flat
||The existence of noise due to ambient light, noise of the cameras CCD,
the noise in the video capture card, etc. introduce difficulties in the
||The possible variations of the relative pose between the camera and the
CRT with the actual attributes of the camera lens (Motai
Kosaka, 2001) introduce some distortions in the closed loop control
In addition, the synchronization of the camera frames by the frames of
the television being adjusted must be considered. An auto alignment system
must be capable of adjusting the geometric attributes of the CRT screen
with a high degree of accuracy at a very short time. This places stringent
requirements on processing time and demands the development and application
of real-time algorithms to produce control signals.
THE DEFINITION OF ADJUSTMENT PARAMETERS
A critical issue in the design of an auto-alignment system is to define
a suitable set of adjustment parameters based on the geometric characteristics
of the television screen. Suppose that the geometric parameters of the
television screen form a space such that each dimension in this space
corresponds to one of the geometric parameters. The selection of the adjustment
parameters must be such that they are fairly insensitive to the variation
of television-camera pose and establish a unique point in the geometric
parameters space of the television screen. In other words, there must
be a one-to-one correspondence between the adjustment parameters and the
geometric parameters of the television screen.
The present characteristics of the television production line in the
manufacturing house are such that if we use a single fixed camera the
variations of the television-camera pose are only possible in the horizontal
direction. Any variations in the vertical direction would be minimal and
can be ignored. The major types of variations in the television-camera
pose consist of angular variations, depth variations and horizontal displacement
as shown in Fig. 2.
Moreover, the definition of the adjustment parameters and the design
of adjustment algorithm must be robust to displacement disturbances. Given
the fact that in consecutive adjustments of the televisions running down
the production line, the relative position of the camera with respect
to the television set is variable, we decided to use an image-based control
approach and selected the adjustment parameters proportional to the measured
size of the TV screen being adjusted.
||All kinds of pose variations between a fixed camera
and television on production line
||Some vertical adjustment parameters actually used in
Definitions of several typical vertical parameters and the corresponding
generated patterns on the TV screen are shown in Fig. 3.
It is clear that the displacement errors do not considerably affect the
defined adjustment parameters. Note that the defined parameters are only
related to the vertical geometric characteristics of the screen. Since
the number of horizontal geometric parameters in television is usually
more limited than the vertical ones, a similar but smaller set of parameters
can also be considered for the horizontal direction. The defined adjustment
parameters are related to the size and the relative location of the television
screen being adjusted. Therefore, we need to develop an algorithm to detect
the television screen in the captured image.
Assuming that the relative camera-television pose is fixed during the
adjustment of a given television set, we need to measure the precise location
and the exact dimensions of the television screen in the captured image.
This is obviously a critical factor in the estimation of the defined adjustment
parameters. In order to achieve this, we brighten all the points of the
television CRT screen by applying a white pattern first. The brightness
of the television screen area is thus clearly separated from the surrounding
area and can be used in the following captured images by applying an appropriate
threshold to create suitable binary images. The next step would be to
identify the screen boundary and eliminate the parasitic information outside
the edges of the screen in the captured images. To achieve this we have
developed a novel fast algorithm using a weighted window to follow the
edge of the screen in the binary image.
Edge following algorithm: Present edge detection algorithm employs
a weighted 3×3 window whose variants are shown in Fig. 4.
On the figure each template relates to a case of the boundary pixels of
convex shapes and its assigned value is unique.
||Weighted 3×3 window and its significant variants
Of the templates shown,
only template 15 does not relate to the boundary. It is related to the
inside of the shape. For concave shapes, other templates must be added.
The value assigned to each template is simply calculated as follows:
four of the pixels are assigned weights of 2n as indicated
in Fig. 4. If the corresponding pixels are on, the associated
weights are counted. The sum of the weighted numbers related to the pixels
in the window is computed in each location that the window is placed.
The template matches and their corresponding location on the image of
the CRT screen are shown in Fig. 5.
If we place the central point of the above window on any inner point
of the screen boundary, a template match 15 occurs. Now if we move the
window in any direction and proceed so far that we cross the boundary,
a template match different from 15 occurs. In this situation, the central
point of the window represents an edge pixel.
TV screen and the detail of present edge following algorithm.
Expected template matches are shown in the various locations
After finding an edge point,
we move the window in either clockwise or counterclockwise direction on
the screen boundary in an attempt to follow the edge of the CRTs image
on the screen. We continue this procedure until the first edge point is
encountered again. In this manner, all the edge pixels are found and are
stored in another image in memory.
To illustrate how the edge of the CRTs image on the screen is followed
in present algorithm, suppose we have an image of the TV screen as shown
in Fig. 5. The original window placed inside the boundary
of the screen is moved horizontally to the right until a template match
other than 15 occurs. If the template match is 3, then we have reached
the upper right corner; if it is 11, then we have reached the right side
and if it is 9, we have reaced the lower right corner of the CRTs image
on the screen.
Assume, for example, that we have encountered a template match 11 which
indicates that we have reached the right side of the image. To follow
the edge of the screen, we move the window down by one pixel.
|| Flowchart of the edge following algorithm
this move, the right pixel in the bottom of the window is tested to see
if it is located on the image of the screen. If it is, the windows center
is placed on it as we move downwards. Otherwise the window is moved straight
downwards. This process is continued until a template match other than
11 occurs. In this case it must be 9 indicating that we have reached the
lower right corner of the CRTs image on the screen. Now before we move
the window towards the left, we must test the lower left pixel of the
window to see if it is located on the image of the screen or not. If it
is, we move the window such that its center is located on that pixel.
Otherwise, the window is moved straight to the left by one pixel. This
process is continued in a similar manner until we reach the lower left
corner. Then we move upwards to reach the upper left corner. Then we move
to the right until we reach the upper right corner. Then we move downwards
until we reach the boundary point at which we started our movement on
the boundary. Obviously the template matches that occur as we reach the
various corners-the right side, the left side, the bottom and the top
of the screen-are all different from each other as indicated in Fig.
4 and 5. The flowchart of this algorithm is represented
in Fig. 6. In order to save CPU time, we only use the
basic shift operation rather than the computationally intensive multiplication
operation in the computation of the weights for the window as depicted
in the upper right corner of Fig. 6.
We compared the performance of present algorithm with that of Sobels edge
detection algorithm (Parker, 1997) since other edge detection
algorithms are more difficult to code and much slower than Sobels algorithm.
||Results of applying present proposed edge following
and Sobel edge detection algorithms
The results are shown in Fig. 7 where, we can see the
advantages of present proposed edge following algorithm in rejecting noise
of ambient light on the front panel of the television. The speed of our
algorithm to follow the edge of the TV screen in captured images is approximately
nine times the speed of the Sobel edge detection algorithm. Both algorithms
were efficiently programmed in C++ and were run on the same platform to
make this comparison.
Another advantage of present suggested algorithm is that it can be used
to quickly measure the dimensions of the TV screen, whereas existing algorithms
lack this potential.
Detecting the screens border: During the detection of the screens
border, the coordinates of some of the templates used can help us determine
the dimensions and the exact position of the screen in the captured image.
For example, since the template No. 11 identifies the right side of the
screens border, the horizontal mean value of the template No. 11 estimates
the rightmost position of the screen. Likewise, this technique can be
used for the other sides of the screen with templates No. 14, 7 and 13.
Obtaining vertical adjustment parameters: We can obtain the values of
the adjustment parameters using the dimensional and positional characteristics
of the TV screen in the image. To measure the adjustment parameters, first we
generate an appropriate pattern on the TV screen. The pattern generated depends
on the proposed adjustment parameter that we want to obtain. The position of
the applied pattern on the screen is then determined using the maximum gradient
edge detection algorithm (Parker, 1997;
Meade and Webb, 1998).
For example suppose we generate the pattern of vertical size adjustment
parameter as shown in Fig.8. Starting from the top side
of the screen, the maximum gradient edge detection algorithm is applied
on several parallel vertical lines of pixels that are limited to the width
of the TV screen. Averaging the maximum gradient position of the negative
edge-from bright pixels to the dark ones-over these parallel lines results
in the location of the used pattern. The obtained pattern location with
the top and down side positions of the TV screen is used to calculate
the value of the vertical size adjustment parameter (Fig.
3a). The other adjustment parameters are calculated in a similar manner.
||V-Size pattern and vertical lines of pixels
Obtaining Horizontal Adjustment Parameters: The possible variations
in the television-camera relative pose on the production line are such that
they can influence the measurement of the horizontal geometric characteristics
more than the vertical ones. Therefore, in order to be able to properly measure
the horizontal geometric characteristics a few optical points must be carefully
considered. Lets assume that we want to measure the deviation of the horizontal
centre. We must therefore define the horizontal centre adjustment parameter.
Suckle (1988) suggested to generate a vertical line pattern
at the horizontal centre. He defined the horizontal centre adjustment parameter
similar to the vertical case as shown in Fig. 9.
Since Suckle (1988) did not present any experimental
study on this approach, the practical problems with this suggestion were not
considered by him. In a practical experimental setup, the defined adjustment
parameter would only be correctly measured if the viewing direction of the measurement
camera is exactly perpendicular to the surface of the TV screen. Any deviations
from this orthogonality condition due to possible television angular displacement
moving along on the production line until it reaches the front of the camera
where it is temporarily stopped for a few moments would introduce error in measurement.
The optical nature of the effect of the angular displacement of the TV screen
from the orthogonality condition with the camera viewing direction from the
top view is shown in Fig. 10.
Suppose that the point A shows the leftmost side of the screen and the point
B represents the rightmost side of the screen. Any angular displacement θ°
of the TV screen from the exact orthogonal direction, results in new states
for the rightmost and leftmost sides of the screen that are shown by the points
A′ and B′, respectively.
||Preliminary horizontal center pattern and its adjustment
|| Optical essence for preliminary H-Center pattern
The images of the points
A and B are the Ac and Bc, respectively that are created
on the image plane (Hutchinson et al., 1996).
For the rotated TV screen case the images of the points A′ and
B′ on the image plane are the A′c and B′c,
To estimate the measurement error due to angular displacement of the
screen we consider corresponding imaginary points for the A′
and B′ on the exact orthogonal direction in such a manner
that their images on the image plane are the same as the original ones.
One of the imaginary points A′p is located
at the intersection of the ray directed from the pixel at point A′
towards the optical centre-point C-with the line AB. The other point B′p
is located at the intersection of the imaginary line from the optical
centre towards B′ with the line AB. The middle point
of the line AB and the line A′ B′ are
coincident. Therefore, this point is the correct horizontal centre of
the TV screen for both the exact orthogonal case and the tilted case.
The measurement error of the defined horizontal centre adjustment parameter
due to angular displacement of the screen from the perpendicularity condition
equals to the distance between the middle point of the line A′pB′p
and the correct centre. The following equations show the calculation of
the measurement error of the defined adjustment parameter Eh1
(θ) by accumulating half of the difference between the length of
OA and OA′p and the difference between the
length of OB and OB′p.
In order to minimize the error due to any deviations from orthogonality
we define an appropriate horizontal centre adjustment parameter by considering
a pattern with a pair of vertical parallel lines near the sides of the
screen. The corresponding adjustment parameter ΔHcenter
is described in Fig. 11.
Figure 12 shows the optical nature that is used to
calculate the measurement error of the suggested adjustment parameter.
The points M and N represent the two vertical parallel lines on the pattern
that have the same distance from the horizontal centre O.
||Preferred horizontal center pattern and its adjustment
of the horizontal centre adjustment parameter by the use of the recommended
pattern is based on the length of BN and MA that are corresponding to
the Hc1 and Hc2, respectively in Fig.
11. Suppose that the points M and N are at the same distance d from
the points A and B, respectively. As shown in Fig. 10 the angular displacement θ° of the TV screen from the exact orthogonal
direction is shown by points A and B. The imaginary points A′p,M′p,N′p and B′p on the exact orthogonal direction
are considered instead of A, M, N and B′, respectively.
The measurement error Eh2 (θ) of the recommended horizontal
centre adjustment parameter due to angular displacement of the screen
equals to the sum of the difference between the length of MA and M′pA′p and the difference between the length of BN and B′pN′p as follow:
By comparing the two error Eq. (4) and (8)
||Optical essence for chosen H-Center pattern
Eh1 (θ) and T (θ) have non-negative values since
0≤θ<90, z<1 and l>d. In addition T (θ)≤Eh1
(θ). Therefore, it is clear that 0≤Eh2 (θ) ≤Eh1
(θ). By decreasing d, T (θ) approaches Eh1 (θ)
and Eh2 (θ) vanishes.
Actually if we desire to be able to measure from the image that is created
by a CCD, we cannot arbitrary decrease d since its projection must at least
occupy several consecutive discrete cells of the CCD plane. Thus if we use present
definition of the horizontal centre adjustment parameter as shown in Fig.
11, the measurement error due to angular displacement cannot be totally
eliminated. However, it can be effectively reduced.
PARAMETERS ADJUSTMENT ALGORITHM
After the measurement phase, the digital controller must generate proper control
signals based on the errors between the set points and the corresponding measured
adjustment parameters in order to adjust the geometric parameters of the TV
screen. The digital controller uses the resultant errors to create proportional
control signals (Sznaier Camps, 1998; Chroust et al.,
2000). The exact loop gain of each adjustment parameter is related to its corresponding
The process of the measurement and the generation of the control signals
must be repeated until the error between the desired values and the measured
parameters is reduced to an acceptable level. Note that each adjustment
parameter has its own specific acceptable error level.
The spatial discrete structure of the plant and the measurement process
cause the closed loop system to be unstable in some situations. Suppose
that the two nearest adjacent values of a measured adjustment parameter
of the desired value are such that the corresponding error value cannot
fall in the acceptable limit which leads to an unstable state. To overcome
this problem, we use an adaptive mechanism for the acceptable error. If
at the adjustment process, a measured adjustment parameter value changes
between two consecutive values for several iterations, then an unstable
state is detected. This can be used for widening the acceptable error.
In every iteration that an unstable state is detected, the acceptable
error level is smoothly increased. This procedure is continued until the
measured error falls in the increased acceptable error level.
As was previously said, some of the geometric parameters of the TV screen
are related to each other. In the designed system, the adjustment parameters
have been simply modeled separately and the existing mutual effects between
the adjustment parameters has not been considered. So the adjustment process
must be applied several consecutive times until the adjustment parameters
converge to the desired values. This convergence is guaranteed since the
value of every mutual effect factor is less than one.
The system described has been utilized to implement the presented algorithms.
The designed system has been installed on the production line of the 14
Pars colour television at Sirjan Electronics Company in Iran. This system
has successfully undertaken the automatic adjustment task of the geometric
parameters and the white balance of the TV screen in the manufacturing
production line. The total adjustment process contains all horizontal
and vertical geometric parameters as well as colour adjustment parameters.
It is carried out in less than 1.5 min which shows a fivefold improvement
over previous manual adjustment methods at an increased precision.
In this study we have presented the application of previously published
novel fuzzy test pattern and adaptive alignment approach presenting a
novel design for an auto-alignment system for the television sets which
we have practically implemented on a real production line. The problems
encountered were explained and the suggested solutions were utilized in
designing the system. Present alignment algorithms take only relative
measurements that require no camera calibration. Therefore, an uncalibrated
camera can be used to obtain the misalignment information in the closed-loop
structure. We considered several points for defining the adjustment parameters to reduce the measurement error and utilized proportional
digital control rules with adaptive error levels to stabilize the process
of adjusting the geometric characteristics.
We implemented our automatic alignment algorithm and applied it to a
production line of the 14" colour television. The experimental results
and in-house practical use of our system have shown that our system works
well in real setting.