In order to obtain high precise and high reliable micromanipulation, the microscope
visual servoing should be employed, which can finish micromanipulation tasks
such as object tracking, position, grabbing, assembly and so on (Mezouar
and Peter Allen, 2002; Ferrira et al., 2004). For the visual servoing
including image processing, control, real-time computation. So, we must consider
vision delay effect from visual servoing process. The vision delay has serious
effect for the system control performance and can bring the instability of system
for dynamic object tracking for manipulator. However, a few papers focus on
vision delay. Iwazaki et al. (1997) alemploys
smith method to offset timing delay bringing by image processing in Position
Based Visual Servoing (PBVS), which improves the system performance and enhances
its stability. Hui et al. (2005) present a visual
servoing based on modified smith predictor for the control of micromanipulation,
using micromanipulator to finish experiments about point to point position,
micro-gear tracking and disturbance rejection. Yanfei et
al. (2004) build a precise timing modelling and obtain a high efficiency
and reliable control performance. At present, there are two main approaches
to offset vision delay of visual servoing: one is smith predictor method and
another is filter prediction method. Smith predictor (Yi
De et al., 2007) is good for much time delay system. Its primary
principle is that predicts the response of control object under disturbance
and then controls the delayed control parameters to feedback in advance, making
that controller acts in advance and then improves the control performance.
In order to improve the micromanipulation performance, a control scheme based on fuzzy adaptive PID with a Modified Smith Predicator for the control of micromanipulation is proposed. For the vision delay, a timing modelling of visual servoing system is built. According to analysis for the position based dynamic look and move control scheme, the control scheme employs the proposed controller to eliminate the vision delay. The simulations and experiments show that the vision control system with the proposed control scheme has better dynamic performance than the vision control system with a single PID controller. The proposed control scheme resolves the problems of vision servoing inherent timing delay and enhances the micromanipulation performance.
Figure 1 shows the principle of smith predictor. The control
system consists of controller Gc (S), predictor Gm(S)(1-e-rs),
control object Go(S)e-rs.
Under condition of predictor model matching object model, namely, Go(S)
= Gm(S), The transfer function of control system can be represented
as Eq. 1:
||The principle diagram of smith predictor
It can be seen from (1) that delay link e-rs is not located in close
loop control link, meaning that time delay can not affect the stability of control
system. Delay link e-rs defers only time r for control system and
the control performance is same as the object model with transfer function Go(S).
Therefore, Smith predictor eliminates the timing delay and improves the performance.
MICROSCOPE VISUAL SERVOING TIMING MODELING
Since, the visual servoing include image processing, control, real-time compute, So, It is high time to consider that vision delay affects system control performance, which much vision delay can affect the control performance and even bring the instability of system for dynamic object tracking for manipulator.
Focusing on the vision delay of robotics vision computation and control,
we should design the vision controller to offset vision delay. To arrive this
aim, we must build firstly the timing modelling of control system. Vincze
(2000) presents four timing modellings of visual servoing: on-the-fly, serial,
parallel, pipeline. The timing modelling of on-the-fly has a highest processing
efficiency. Now, we start to analyse and build the timing modelling of system.
The main factors of time delay in manipulator visual servoing include as follows:
the time of image acquisition tc, the time of image processing tp,
the time of vision control tv, the time of joint servoing control
tj. We presume that vision delay includes image acquisition, image
processing and vision control, then the delay time bringing by vision is tc+tp+tv.
Similarly, we presume that the movement delay includes joint servoing control
ts. Then, the timing modelling can be built as shown in Fig. 2. From
Fig. 2, four visual servo threads from top to down represent vision acquisition,
image processing, visual control, joints servo control, respectively. The horizontal
axis represents time (Fig. 2). A, B and C represent three continuous visual
servoing process, respectively.
||The timing modelling of microscope visual servoing
The time of image processing tpi(i = 1, 2, 3) and the time of manipulator
movement tmi(i = 1, 2, 3) are main delay factor in visual servoing,
which tpi is decided by complexity level of image processing algorithm
and tsi lies on the range of manipulator movement, movement speed
and response time. According to Fig. 2, if it meets tpi = tsi,
the runtime can be assigned precisely for every thread. The image processing
thread tpi(i = 1, 2, 3) and manipulator movement thread tmi(i
= 1, 2, 3) can also run continuously, without timing delay. We choose the max
time tmax from above two threads and give tpi = tsi
= tmax. Then, the servoing cycle of microscope visual servoing is
FUZZY ADAPTIVE PID CONTROL WITH A MODIFIED SMITH PREDICTOR
Generally, The structure of position based visual servoing can be constructed
as Fig. 3. Firstly, Control system obtains object position
r(s) and end-effector position c(s) using image processing. Secondly, employs
the trajectory planning as feedback to predict the next movement for manipulator.
In real task, although robotic current position can be obtained from joint feedback,
we need that estimate relative position of object and end-effector for calibration
error, position estimation error and system error. During robotic visual servoing,
the vision cycle may be more longer than joint servoing cycle, meaning that
the timing delay from image processing, computation and control should be considered.
So, we present a control scheme based on fuzzy adaptive PID with a modified
smith predicator to offset vision delay (Takagi and Sugeno,
The visual servoing structure with modified smith predictor is shown in Fig. 4. We give that the transfer function of micro-manipulator is Gr and the transfer
function of PID controller is G with Kp, Ki, Kd from fuzzy deduction. Gm(S)(1-e-rs)
is the transfer function of smith predictor. D is the transfer function of disturbance.
M is a new added the transfer function of disturbance rejection.
||The structure of position based visual servoing
||The visual servoing structure with modified smith predictor
It can be seen from Fig. 4 that control system is a standard smith predictor
when M is lacked in system structure.
The response of control system can be written as Eq. 2 when
meets Gm(S) = Gr(S):
According to Eq. 3, Characteristic equation has not delay
link, then PID controller can be decided only by system itself. In addition,
the response of disturbance can be also controlled only by the new-added controller
M. It has been shown (Watanabe and Ito, 1981) that the
smith predictor gives a steady state error under disturbances for open loop
unstable process. Clearly, the new-added controller M can improve the system
performance of disturbance rejection. Next, the control process based new structure
will be given.
At the beginning of the visual servo process (Liu, 2003;
Sim et al., 2002; Kaya, 2004),
the manipulator position and expectations position of the target in micro-vision
is ξ(t0) and ξ*(t0), respectively. Then, according
to the timing modelling in Fig. 2, the manipulator position
in last time should be and
expectations position at present should be Δt1
and Δt2 represents delay factor coming from manipulator movement
and object movement, respectively.
Defines that control variable
in visual servoing process t is the error between manipulator movement and
object movement then
be written as Eq. 5:
Since, the microscope visual field is finite and controller is to avoid integral
saturation, we design a two DOF visual controller based fuzzy adaptive PID control
law for micro-manipulation.
The fuzzy adaptive PID controller applies error e and error change ec as its
input. Then, we build a fuzzy rule table based practice experiences, which gives
the counterpart relationship between Kp,Ki,Kd and error e, error change ec.
So, we can revise online control system parameters. Equation
8 shows the revised computation formula:
Then, the vision control output can be represented as Eq. 9:
where, u(t) is the control output of controller in manipulator task space and Ĵv(t) is image jacobian matrix by identifying online using broyden method.
The micromanipulation system (Vikramaditya and Nelson, 1997;
Ferreira et al., 2004; Ralis
et al., 2000) consists of micromanipulation stage, microscopes vision,
micro-gripper. The system structure is shown in Fig. 5.
Firstly, we test the control performance of point to point movement using the
proposed method. The closed-loop responses results with a single PID controller
and with a fuzzy adaptive PID companying with modified Smith predictor are presented.
Figure 6a and b show the performance,
respectively. The results show that the visual servoing control system with
proposed scheme in comparison to a single PID controller provides more robustness
and disturbance rejection.
||The experimental system of micromanipulation
(a) The closed-loop responses results with a single PID controller and
(b) with a fuzzy adaptive PID companying with modified Smith predictor
Next, we finish two tests employed the proposed scheme. One is micromanipulator
movement in XY plane. We make micromanipulator to move from point to point in
plane under optical microscopy with the 2x4x lens magnification.
||Micromanipulator XY plane movement (a) 2 X lens magnification.
(b) 4 X lens magnification
The initial position is given at random and expectations position is (100,
200). Under microscopic micrometer scale calibration results, 2x lens image
resolution is 5.70 and 5.65 μm/pixel and 4x lens is 2.85 and 3.15 μm/pixel.
So micromanipulator movement step in the 2x4x lens magnification is 4 and 3
μm with a speed of 1.20 mm sec-1. Micromanipulator visual servo
positioning results is shown in Fig. 7a and b,
the two final positioning errors of X, Y directions are less than one pixel.
One is the movement trajectory tracking. We presume that target moves in the
XY image plane in invariable speed. The expectation trajectory is y = 120+5(x-100)/3,
then the movement equation can be represented as Eq. 10:
We make that the manipulator tracks target in XY plane with 2X lens and 5 μm
step. The start point is 1.(175,222) and end point is 2.(183,268).
Figure 8 shows the movement trajectory of micromanipulator and target. Figure
9 shows the servo tracking result of X Y coordinate of micromanipulator.
||The movement trajectory of micromanipulator and target
||The servo tracking result of X, Y position coordinate of micromanipulator
The original microscopic image of object and the endeffector. (a) Vertical
view field of microscopic images. (b) Horizontal view field of microscopic
The image of the end-effector automatically locating and gripping object.
(a) Vertical view field of microscopic images. (b) Horizontal view field
of microscopic images
Finally, we employ the proposed scheme to finish micro-size part automatic
position and grabbing. Figure 10 is the original microscopic
image of two view fields, the object (cylindrical parts) and the end-effector
(clip shape objects). Figure 11 is the image of the end-effector
automatically locating and gripping object.
The experiment results above show that the vision control system with the proposed control scheme has better performance than the single feedback vision servo control system. It has shown the proposed control scheme is a simple and pragmatic approach for the time delay of visual servoing systems.
Focusing on the vision delay of robotics vision computation and control, we present a control scheme based on fuzzy adaptive PID with a Modified Smith Predicator for the control of micromanipulation. For the sake of precise control, a timing modelling of visual servoing system is built. According to the proposed controller, the control scheme eliminates the vision delay and improves system disturbance rejection. The simulations and experiments show that the vision control system with the proposed control scheme has better dynamic performance than the vision control system with a single PID controller. The micromanipulator automatic position and grabbing results has shown the proposed control scheme is a simple and pragmatic approach for the time delay of visual servoing systems, which meeting the requirements of micromanipulation.
This study is supported by the National Nature Science Foundation of China under Grant No. 60275023 and China 863 Program No. 2007AA844220.