Subscribe Now Subscribe Today
Research Article
 

Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing



Linfeng Bai, Fugui Chen and Xiangjin Zeng
 
Facebook Twitter Digg Reddit Linkedin StumbleUpon E-mail
ABSTRACT

This study presents a control scheme based on fuzzy adaptive PID with a modified Smith predicator for the control of micromanipulation. For the vision delay, a timing modeling of visual servoing system is built. According to analysis for the position based dynamic look and move control scheme, the control scheme employs fuzzy PID with a similar structure to the Smith predictor called modified Smith predictor to eliminate the vision delay. The simulations and experiments show that the vision control system with the proposed control scheme has better dynamic performance than the vision control system with a single PID controller. The proposed control scheme resolves the problems of vision servoing’s inherent time delay, which meets the requirements of micromanipulation.

Services
Related Articles in ASCI
Search in Google Scholar
View Citation
Report Citation

 
  How to cite this article:

Linfeng Bai, Fugui Chen and Xiangjin Zeng, 2009. Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing. Information Technology Journal, 8: 195-201.

DOI: 10.3923/itj.2009.195.201

URL: https://scialert.net/abstract/?doi=itj.2009.195.201
 

INTRODUCTION

In order to obtain high precise and high reliable micromanipulation, the microscope visual servoing should be employed, which can finish micromanipulation tasks such as object tracking, position, grabbing, assembly and so on (Mezouar and Peter Allen, 2002; Ferrira et al., 2004). For the visual servoing including image processing, control, real-time computation. So, we must consider vision delay effect from visual servoing process. The vision delay has serious effect for the system control performance and can bring the instability of system for dynamic object tracking for manipulator. However, a few papers focus on vision delay. Iwazaki et al. (1997) alemploys smith method to offset timing delay bringing by image processing in Position Based Visual Servoing (PBVS), which improves the system performance and enhances its stability. Hui et al. (2005) present a visual servoing based on modified smith predictor for the control of micromanipulation, using micromanipulator to finish experiments about point to point position, micro-gear tracking and disturbance rejection. Yanfei et al. (2004) build a precise timing modelling and obtain a high efficiency and reliable control performance. At present, there are two main approaches to offset vision delay of visual servoing: one is smith predictor method and another is filter prediction method. Smith predictor (Yi De et al., 2007) is good for much time delay system. It’s primary principle is that predicts the response of control object under disturbance and then controls the delayed control parameters to feedback in advance, making that controller acts in advance and then improves the control performance.

In order to improve the micromanipulation performance, a control scheme based on fuzzy adaptive PID with a Modified Smith Predicator for the control of micromanipulation is proposed. For the vision delay, a timing modelling of visual servoing system is built. According to analysis for the position based dynamic look and move control scheme, the control scheme employs the proposed controller to eliminate the vision delay. The simulations and experiments show that the vision control system with the proposed control scheme has better dynamic performance than the vision control system with a single PID controller. The proposed control scheme resolves the problems of vision servoing inherent timing delay and enhances the micromanipulation performance.

SMITH PREDICTOR

Figure 1 shows the principle of smith predictor. The control system consists of controller Gc (S), predictor Gm(S)(1-e-rs), control object Go(S)e-rs.

Under condition of predictor model matching object model, namely, Go(S) = Gm(S), The transfer function of control system can be represented as Eq. 1:

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Fig. 1: The principle diagram of smith predictor

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
(1)

It can be seen from (1) that delay link e-rs is not located in close loop control link, meaning that time delay can not affect the stability of control system. Delay link e-rs defers only time r for control system and the control performance is same as the object model with transfer function Go(S). Therefore, Smith predictor eliminates the timing delay and improves the performance.

MICROSCOPE VISUAL SERVOING TIMING MODELING

Since, the visual servoing include image processing, control, real-time compute, So, It is high time to consider that vision delay affects system control performance, which much vision delay can affect the control performance and even bring the instability of system for dynamic object tracking for manipulator.

Focusing on the vision delay of robotic’s vision computation and control, we should design the vision controller to offset vision delay. To arrive this aim, we must build firstly the timing modelling of control system. Vincze (2000) presents four timing modellings of visual servoing: on-the-fly, serial, parallel, pipeline. The timing modelling of on-the-fly has a highest processing efficiency. Now, we start to analyse and build the timing modelling of system.

The main factors of time delay in manipulator visual servoing include as follows: the time of image acquisition tc, the time of image processing tp, the time of vision control tv, the time of joint servoing control tj. We presume that vision delay includes image acquisition, image processing and vision control, then the delay time bringing by vision is tc+tp+tv. Similarly, we presume that the movement delay includes joint servoing control ts. Then, the timing modelling can be built as shown in Fig. 2. From Fig. 2, four visual servo threads from top to down represent vision acquisition, image processing, visual control, joints servo control, respectively. The horizontal axis represents time (Fig. 2). A, B and C represent three continuous visual servoing process, respectively.

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Fig. 2: The timing modelling of microscope visual servoing

The time of image processing tpi(i = 1, 2, 3) and the time of manipulator movement tmi(i = 1, 2, 3) are main delay factor in visual servoing, which tpi is decided by complexity level of image processing algorithm and tsi lies on the range of manipulator movement, movement speed and response time. According to Fig. 2, if it meets tpi = tsi, the runtime can be assigned precisely for every thread. The image processing thread tpi(i = 1, 2, 3) and manipulator movement thread tmi(i = 1, 2, 3) can also run continuously, without timing delay. We choose the max time tmax from above two threads and give tpi = tsi = tmax. Then, the servoing cycle of microscope visual servoing is tmax.

FUZZY ADAPTIVE PID CONTROL WITH A MODIFIED SMITH PREDICTOR

Generally, The structure of position based visual servoing can be constructed as Fig. 3. Firstly, Control system obtains object position r(s) and end-effector position c(s) using image processing. Secondly, employs the trajectory planning as feedback to predict the next movement for manipulator. In real task, although robotic current position can be obtained from joint feedback, we need that estimate relative position of object and end-effector for calibration error, position estimation error and system error. During robotic visual servoing, the vision cycle may be more longer than joint servoing cycle, meaning that the timing delay from image processing, computation and control should be considered. So, we present a control scheme based on fuzzy adaptive PID with a modified smith predicator to offset vision delay (Takagi and Sugeno, 1985).

The visual servoing structure with modified smith predictor is shown in Fig. 4. We give that the transfer function of micro-manipulator is Gr and the transfer function of PID controller is G with Kp, Ki, Kd from fuzzy deduction. Gm(S)(1-e-rs) is the transfer function of smith predictor. D is the transfer function of disturbance. M is a new added the transfer function of disturbance rejection.

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Fig. 3: The structure of position based visual servoing

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Fig. 4: The visual servoing structure with modified smith predictor

It can be seen from Fig. 4 that control system is a standard smith predictor when M is lacked in system structure.

The response of control system can be written as Eq. 2 when meets Gm(S) = Gr(S):

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing

(2)

Where:

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
(3)

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
(4)

According to Eq. 3, Characteristic equation has not delay link, then PID controller can be decided only by system itself. In addition, the response of disturbance can be also controlled only by the new-added controller M. It has been shown (Watanabe and Ito, 1981) that the smith predictor gives a steady state error under disturbances for open loop unstable process. Clearly, the new-added controller M can improve the system performance of disturbance rejection. Next, the control process based new structure will be given.

At the beginning of the visual servo process (Liu, 2003; Sim et al., 2002; Kaya, 2004), the manipulator position and expectations position of the target in micro-vision is ξ(t0) and ξ*(t0), respectively. Then, according to the timing modelling in Fig. 2, the manipulator position in last time should be Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoingand expectations position at present should be Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual ServoingΔt1 and Δt2 represents delay factor coming from manipulator movement and object movement, respectively.

Defines that control variable Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing in visual servoing process t is the error between manipulator movement Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoingand object movement Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoingthen Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoingcan be written as Eq. 5:

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
(5)

Where:

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
(6)

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
(7)

Since, the microscope visual field is finite and controller is to avoid integral saturation, we design a two DOF visual controller based fuzzy adaptive PID control law for micro-manipulation.

The fuzzy adaptive PID controller applies error e and error change ec as its input. Then, we build a fuzzy rule table based practice experiences, which gives the counterpart relationship between Kp,Ki,Kd and error e, error change ec. So, we can revise online control system parameters. Equation 8 shows the revised computation formula:

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
(8)

Then, the vision control output can be represented as Eq. 9:

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing

(9)

where, u(t) is the control output of controller in manipulator task space and Ĵv(t) is image jacobian matrix by identifying online using broyden method.

RESULTS

The micromanipulation system (Vikramaditya and Nelson, 1997; Ferreira et al., 2004; Ralis et al., 2000) consists of micromanipulation stage, microscopes vision, micro-gripper. The system structure is shown in Fig. 5.

Firstly, we test the control performance of point to point movement using the proposed method. The closed-loop responses results with a single PID controller and with a fuzzy adaptive PID companying with modified Smith predictor are presented. Figure 6a and b show the performance, respectively. The results show that the visual servoing control system with proposed scheme in comparison to a single PID controller provides more robustness and disturbance rejection.

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Fig. 5: The experimental system of micromanipulation

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Fig. 6:
(a) The closed-loop responses results with a single PID controller and (b) with a fuzzy adaptive PID companying with modified Smith predictor

Next, we finish two tests employed the proposed scheme. One is micromanipulator movement in XY plane. We make micromanipulator to move from point to point in plane under optical microscopy with the 2x4x lens magnification.

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Fig. 7: Micromanipulator XY plane movement (a) 2 X lens magnification. (b) 4 X lens magnification

The initial position is given at random and expectations position is (100, 200). Under microscopic micrometer scale calibration results, 2x lens image resolution is 5.70 and 5.65 μm/pixel and 4x lens is 2.85 and 3.15 μm/pixel. So micromanipulator movement step in the 2x4x lens magnification is 4 and 3 μm with a speed of 1.20 mm sec-1. Micromanipulator visual servo positioning results is shown in Fig. 7a and b, the two final positioning errors of X, Y directions are less than one pixel.

One is the movement trajectory tracking. We presume that target moves in the XY image plane in invariable speed. The expectation trajectory is y = 120+5(x-100)/3, then the movement equation can be represented as Eq. 10:

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing

(10)

We make that the manipulator tracks target in XY plane with 2X lens and 5 μm step. The start point is 1.(175,222) and end point is 2.(183,268). Figure 8 shows the movement trajectory of micromanipulator and target. Figure 9 shows the servo tracking result of X Y coordinate of micromanipulator.

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Fig. 8: The movement trajectory of micromanipulator and target

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Fig. 9: The servo tracking result of X, Y position coordinate of micromanipulator

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Fig. 10:
The original microscopic image of object and the endeffector. (a) Vertical view field of microscopic images. (b) Horizontal view field of microscopic images

Image for - Fuzzy Adaptive Proportional Integral and Differential with Modified Smith Predictor for Micro Assembly Visual Servoing
Fig. 11:
The image of the end-effector automatically locating and gripping object. (a) Vertical view field of microscopic images. (b) Horizontal view field of microscopic images

Finally, we employ the proposed scheme to finish micro-size part automatic position and grabbing. Figure 10 is the original microscopic image of two view fields, the object (cylindrical parts) and the end-effector (clip shape objects). Figure 11 is the image of the end-effector automatically locating and gripping object.

The experiment results above show that the vision control system with the proposed control scheme has better performance than the single feedback vision servo control system. It has shown the proposed control scheme is a simple and pragmatic approach for the time delay of visual servoing systems.

CONCLUSION

Focusing on the vision delay of robotic’s vision computation and control, we present a control scheme based on fuzzy adaptive PID with a Modified Smith Predicator for the control of micromanipulation. For the sake of precise control, a timing modelling of visual servoing system is built. According to the proposed controller, the control scheme eliminates the vision delay and improves system disturbance rejection. The simulations and experiments show that the vision control system with the proposed control scheme has better dynamic performance than the vision control system with a single PID controller. The micromanipulator automatic position and grabbing results has shown the proposed control scheme is a simple and pragmatic approach for the time delay of visual servoing systems, which meeting the requirements of micromanipulation.

ACKNOWLEDGMENT

This study is supported by the National Nature Science Foundation of China under Grant No. 60275023 and China 863 Program No. 2007AA844220.

REFERENCES

1:  Ferreira, A., C. Cassier and S. Hirai, 2004. Automatic microassembly system assisted by vision servoing and virtual reality. IEEE Trans. Mechatronics, 9: 321-323.
CrossRef  |  

2:  Hui, X., S. Lining, R. Weibin and Y. Xiufeng, 2005. Visual servoing with modified smith predictor for micromanipulation tasks. Proceedings of the International Conference on Mechatronics and Automation, July 29-August 1, 2005, IEEE Xplore, London, pp: 71-76
Direct Link  |  

3:  Kaya, I., 2004. IMC based automatic tuning method for PID controllers in a Smith predictor configuration. Comput. Chem. Eng., 28: 281-290.
CrossRef  |  

4:  Watanabe, K. and M. Ito, 1981. A process-model control for linear systems with delay. IEEE Trans. Automat. Control, 26: 1261-1266.

5:  Liu, J., 2003. Advance PID Control and MATLAB Simulation. Electronics Industry Publishing House, Beijing

6:  Vincze, M., 2000. Dynamics and system performance of visual servoing. Proceedings of the International Conference on Robotics and Automation, April 24-28, 2000, San Francisco, CA., USA., pp: 644-649
CrossRef  |  Direct Link  |  

7:  Ralis, S.J., B. Vikramaditya and B.J. Nelson, 2000. Micropositioning of a weakly calibrated microassembly system using coarse to fine visual servoing st rategies. IEEE Trans. Elect. Packing Manufact., 23: 123-131.
CrossRef  |  

8:  Sim, T.P., G.S. Hong and K.B. Lim, 2002. Multirate predictor control scheme for visual servo control. IEE Proc. Control Theory Appli., 149: 117-124.
CrossRef  |  

9:  Takagi, T. and M. Sugeno, 1985. Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst. Man Cybern., 15: 116-132.
CrossRef  |  Direct Link  |  

10:  Vikramaditya, B. and B.J. Nelson, 1997. Visually guided microassembly using optical microscopes and active visiontechniques. Proceedings of the International Conference on Robotics and Automation, April 20-25, 1997, IEEE Xplore, London, pp: 3172-3177
CrossRef  |  Direct Link  |  

11:  Yanfei, L., W. Adam Hoover and D. Ian Walker, 2004. A timing model for vision-based control of industrial robot manipulators. IEEE Trans. Robotics, 20: 891-898.
Direct Link  |  

12:  Yi De, C., T.P.I. Cheng and F.C. Charu, 2007. Modified smith predictor scheme for periodic disturbance reduction in linear delay systems. J. Process Control, 17: 799-804.
CrossRef  |  

13:  Mezouar, Y. and K. Peter Allen, 2002. Visual servoed micropositioning for protein manipulation tasks. Proceedings of the Conference on Intelligent Robots and System, September 30-October 2, 2002, IEEE Xplore, London, pp: 1766-1771
CrossRef  |  Direct Link  |  

14:  Iwazaki, Y., T. Murakami and K. Ohnishi, 1997. An approach of visual servoing control considering compensation of time delay. Proceedings of the International Symposium on Industrial Electronics, July 7-11, 1997, Guimaraes, Portugal, pp: 723-728
CrossRef  |  Direct Link  |  

©  2022 Science Alert. All Rights Reserved