Abstract: Camera calibration is an important part of computer vision. The theory is based on the formation of orthogonal vanishing point under perspective projection and linear calibration for a camera. A novel camera self-calibration method based on orthogonal vanishing point is presented, of which the target is the regular prismoid where a top is the regular polygon that can contain a set of points at infinity in orthogonal directions and sides are congruent isosceles trapezoids that also contain a set of points in infinity in orthogonal directions. The point at infinity is called the vanishing point in the image plane. According to the constraints of the orthogonal vanishing points to absolute conic, the intrinsic parameters of a camera can be solved linearly. The experimental results show that the algorithm can accurately and reliably estimate the intrinsic parameters of the camera and it is very close to the actual situation.
INTRODUCTION
Camera calibration is the basic requirement to obtain three-dimensional (3D) information from two-dimensional images in the field of computer vision and it is an indispensable step to complete many visual works. Accurate calibration of internal and external camera parameters can not only improve quality of a camera screen but establish a good foundation for subsequent matching of multiple images and 3D reconstruction (Hor et al., 2011). At the same time, the accuracy of calibration can better satisfy the need of measurement system, such as industrial machine vision (Vo et al., 2011).
Camera calibration is the process which determinates the intrinsic parameters. At present, existing calibration algorithms use geometric property of calibration object (Junejo and Foroosh, 2010; De Franc et al., 2012) or use the constraint of camera's own motion (De Ma, 1996; Grobe et al., 2012). De Ma (1996) proposed the calibration algorithm using two sets of motions. And the motion in each set is constituted by three sets of mutually orthogonal pure transformation vector. This algorithm uses rotation or translation constraint of camera to calibrate the camera’s parameters. These algorithms require higher constraint for the camera motion but also need to calculate the complex nonlinear equations. In literature (De Franc et al., 2012), the method used geometric information of a calibration block from a single image to calibrate. This method requires camera neither to make any motion, nor to seek world coordinates but need to extract feature points in the calibration process. The calibration process is too complicated and it is difficult to avoid the error which is caused by recovering the feature points from an image. The literature (Suziedelyte-Visockiene, 2012) proposed a calibration method using Matlab. Using this procedure of camera calibration could get much higher accuracy of calibration results but it requires users to be familiar with the software, therefore, which restricts the application and extension. A self-calibration method is proposed in the literature (Maybank and Faugeras, 1992), which does not need calibration object and using the relationship between corresponding points in multiple images can directly calibrate the camera intrinsic parameters but this calibration algorithm has poor robustness and needs an estimation of the large number of parameters. Ricolfe-Viala and Sanchez-Salmeron (2011) proposed that using a set of optimal conditions resolved calibration process accurately. The calibration method uses several images of a 2D pattern. The optimal condition defines that the number of points and the number of images resolve the calibration accurately. Criminisi proposed the method using properties of vanishing point and vanishing line to calculate the camera intrinsic parameters in a single image (Criminisi et al., 2000). However, the calibration method using the vanishing point need to extract ellipse from an image to determine the image point of a circle center, which makes the solution process relatively complex. The improved method based on vanishing point was reported in Junejo and Foroosh (2010) and Pu et al. (2011), which only needed to extract six points in a line. He and Yung (2007) and Zhao and Lv (2012) proposed a new method to overcome ill-conditioning in the calibration method based on vanishing point. Tang et al. (2012) presented a novel family of rigorous and comprehensive mathematical self-calibration additional parameters for the camera intrinsic parameters, of which the theoretical basis is the Laplace’s Equation and the Fourier Theorem. The various methods were compared by Kanhere and Birchfield (2010) and Zhao et al. (2012) and used the best method for traffic motion application.
In this study, we use a top and sides of a regular prismoid in two images to fit vanishing point. Using constraints of the vanishing point in the orthogonal directions to the absolute conic can solve out the equation of the absolute conic and then can obtain all of the intrinsic parameters of a camera.
PINHOLE CAMERA MODEL
The camera model is the classic pinhole imaging model in this paper, also known as the pinhole camera model. The intrinsic matrix of a camera is (Criminisi et al., 2000):
| (1) |
where, fu, fv are the focal lengths, s is the skew factor, (u0, v0) is the principal point in image coordinate system.
In this study, the calibration pattern is a regular prismoid. The regular prismoid is obtained by cutting a regular pyramid. As Fig. 1, assume that there is a regular prismoid in the scene, which is composed of congruent isosceles trapezoids and two regular polygons of which sides are the above and bottom sides of the isosceles trapezoids. A regular prismoid has the following properties:
| • | The sides of a regular prismoid are congruent isosceles trapezoids |
| • | Two bottom sides of a regular prismoid and its cross section which is parallel with the bottom sides are regular polygons |
| Fig. 1(a-b): | Regular prismoid: (a) Regular hexagonal frustum pyramid (b) Regular pentagonal frustum pyramid |
Regular prismoid can be divided into two types according to the number of side n, in which the even regular prismoid is 2n, the odd regular prismoid is 2n+1, with n≥3 natural number.
CONSTRAINT OF REGULAR PRISMOID ON THE CAMERA INTRINSIC PARAMETERS
Proposition 1: According to the property (2) of a regular prismoid, upper and lower bottoms of the regular prismoid are regular polygons. If the number of edges of regular polygon is equal to the number of the lateral edges of the regular prismoid, there exist at least a set of orthogonal points at infinity in each upper bottom of regular prismoid, of which the images are called vanishing point.
Below two proof methods of orthogonal vanishing points are given covering the parity regular prismoid:
| Proof: | When the number of lateral edges is 2n, the number of edges of a regular polygon on the bottom is 2n and the number of lateral edges is 2n, denoted as A1A2L A2n. The centrosymmetric two edges of a regular polygon are parallel, so that is: |
| (2) |
In a regular polygon A1, A2, L, A2n, the points at infinity in the direction of AiAi+1 are denoted as Pi (1≤i≤n). The images of A1, A2, L, A2n, P1, P2, L, P2n are a1, a2, L, a2n, p1, p2, L, p2n, respectively, therefore, the number of vanishing points formed by 2n edges is n.
If n is odd, as Fig. 2, the orthogonal vanishing points formed by the edges of the regular polygon are on the diagonals of the regular polygon.
| Fig. 2(a-b): | (a) Regular hexagonal (n = 3) and (b) Its corresponding image |
In the regular polygon A1, A2, L, A2n, we have:
| (3) |
with 1≤i≤n-1.
If i = n, then:
| (4) |
The points at infinity in the direction AiAi+1 are denoted as Pi and the points at infinity in the direction Ai+1Ai+n are denoted as P*i. The corresponding image points of Pi, P*i are pi, p*I, respectively, with 1≤i≤n, thus having:
| (5) |
According to the relation of corresponding points in the projective transformation, Eq. 6 can be gotten from Eq. 5:
| (6) |
For i = n, Eq. 5 and 6 become:
| (7) |
Due to AiAi+1⊥Ai+1Ai+n (1≤i≤n-1), AnAn+1⊥An+1A2n, pi and p*i (1≤i≤n) are a group of orthogonal vanishing points.
If n is even, as Fig. 3, the orthogonal vanishing points formed by the edges of a regular polygon are in the diagonal of the regular polygon.
| Fig. 3(a-b): | (a) Regular quadrilateral (n = 2) and (b) Its corresponding image |
In a regular polygon, we have:
| (8) |
For:
Equation 8 becomes:
| (9) |
Since the points at infinity in the direction AiAi+1 are denoted by Pi and of which the corresponding image points denoted by Pi with 1≤i≤n, we get:
| (10) |
According to the relation of corresponding points in the projective transformation, Eq. 11 can be gotten by Eq. 10:
| (11) |
Due to:
Are a group of orthogonal vanishing points according to the property of the vanishing point in the orthogonal direction.
When the number of lateral edges is odd 2n+1, the number of edges of a regular polygon is 2n+1, as shown in Fig. 4.
The number of edges of a regular polygon is 2n+1 while the number of lateral edges is 2n+1 in a regular prismoid, denoted by A1A2L A2+1. Under these conditions, a vertical relationship does not exist between edge and diagonal in the regular polygon. In order to achieve the camera calibration, draw the inscribed circle to the regular polygon, in which the center of the circle is O and the interactions between the edges of regular polygon A1A2, A2A3, L, A2n+1A1 and the inscribed circle are B1 B2, B2n+1.
| Fig. 4(a-b): | (a) Odd regular pentagon and (b) Its corresponding image |
The points at infinity in the directions A1A2, A2A3, L, A2n+1A1 are P1 P2, P2n+1, respectively. The points at infinity in the directions B1An+2, B2An+3, BnA2n+1, Bn+1A1, Bn+1A1, Bn+2A2, L ,B2n+1An+1 are P1*, P2*, L, P2n+1*, respectively. The image points are denoted as ai, bi, pi, pi* corresponding to AI, BI, PI, Pi*, respectively, with 1≤i≤2n+1.
In the regular polygon A1A2, L, A2n+1, we have:
| (12) |
The points at infinity are P1P2, L, P2n+1, in the direction A1A2, A2A3, L, A2n+1A1, respectively, corresponding to the image points p1p2, l, p2n+1. The midpoints B1 B2, B2n+1 of A1A2, A2A3, L, A2n+1A1, are known. According to the properties of cross-ratios, we can get:
| (13) |
According to the corresponding relation between a space point and its image point in the projective transformation, Eq. 14 can be gotten by Eq. 13:
| (14) |
The points at infinity in the direction .
B1An+2, B2An+3, L, B4A2n+1, Bn+1A1, Bn+2A2, L, B2n+1An+1, are P1*, P2*, L, P2n+1*, are P1*, P2*, L, P2n+1*, respectively, corresponding to P1*, P2*, L, P2n+1*, according to the property of the cross-ratio, we have:
| (15) |
Where:
| Fig. 5(a-b): | (a) Isosceles trapezoid ABCD and (b) Its corresponding image |
According to the correspondence of the corresponding relation between a space point and its image point in the projective transform, we obtain:
| (16) |
From Eq. 14 and 16, the coordinates of vanishing points pi and pi* can be obtained in the orthogonal directions.
Proposition 2: According to the property (a) of a regular prismoid, if the side of a regular prismoid is an isosceles trapezoid and of which the upper and lower are the two parallel segments which are perpendicular to the line connecting the middle points of the upper and lower, the vanishing points are perpendicular, corresponding to the points at infinity in the two directions.
Proof: One side of a regular prismoid is denoted as ABCD. On isosceles trapezoid ABCD, AB, CD are two parallel edges, the midpoints of AB and CD are denoted as M, N and the midpoint of MN is O.
The points at infinity in the directions AB, MN are P, P*, respectively. The images of A, B, C, D, M, N, O, P, P* are a, b, c, d, m, n, o, p, p*, respectively (Fig. 5a-b). Because:
| (17) |
According to the corresponding relation between a space point and its image point in the projective transformation and the characteristic of cross-ratio invariability, Eq. 17 can becomes:
| (18) |
By the property of the isosceles trapezoid, we have MN⊥AB, so p and p* are orthogonal vanishing points, of which coordinates can be obtained from (18).
The constraint equation of the orthogonal vanishing points obtained from Proposition 1 and Proposition 2 to the absolute conic is:
| (19) |
SOLVING ALGORITHM OF THE INTRINSIC PARAMETERS
| Step 1: | Input two images of a regular prismoid, of which each image at least includes the upper bottom and two adjacent sides and extract the feature points on imaging plane |
| Step 2: | Obtain a group of orthogonal vanishing points on the upper bottom and the orthogonal vanishing points on each side according to Proposition 1 and Proposition 2, respectively. Each image can provide three pairs of orthogonal vanishing points, so two images can provide six pairs of orthogonal vanishing points |
| Step 3: | Based on the constraints of orthogonal vanishing points to the absolute conic pTωp* = 0, p and p* are a group of orthogonal vanishing points. Six pairs of orthogonal vanishing points can provide six groups equations similar to the above. Owing to ω containing five independent parameters, ω can be obtained through solving these equations, denoted as: |
Utilize the method of Zhang (2000) can individually calculate each intrinsic parameter. Furthermore, the camera intrinsic matrix K can be gotten.
SIMULATION
A regular quadrangular frustum pyramid, a regular pentagonal frustum pyramid and a regular hexagonal frustum pyramid are used in the simulation as a calibration pattern, as shown in Fig. 6. A simulation camera is set as follows: fu = 1000, fv = 1000, s = 2, uo = 400, vo = 300.
For each calibration pattern, we perform 100 independent trials and the mean values of the intrinsic parameters are computed over each run, shown as in Table 1. It can be seen that the results of the two methods are both in the allowed error range but the values of fv = 1000, uo = 400, vo = 300. obtained from our method are much closer to true values (1000, 400, 300) than the DLT method.
In order to verify the robustness of the algorithm, impose the Gaussian random noises to the image points with mean 0 and variance 0.0-6.8 (pixels). Hundred independent experiments are made under each noise level. Figure 7 and 8 represent the curve of absolute average error about the intrinsic parameters for this method and Direct Linear Transformation (DLT) method with the noise changes. It can be seen that the experimental results of this algorithm has better robustness, when the noise level is large, the standard deviation of each parameter remains small.
| Fig. 6(a-c): | Simulation calibration pattern (a) Regular quadrangular frustum pyramid (b) Regular pentagonal frustum pyramid and (c) Regular hexagonal frustum pyramid |
| Fig. 7(a-e): | Curves of the mean absolute error of the five camera intrinsic parameters (a) fu (b) fv (c) u0, (d) vo and (e) s under different noises levels and calibration patterns |
| Fig. 8(a-c): | Curve of absolute mean error of intrinsic parameters (a) fu, fv (b) u0, vo and (c) sin DLT method changes with noises |
| Table 1: | The mean values of the intrinsic parameters over each run in different simulation patterns |
| Table 2: | Camera Intrinsic Parameters with Real Data for Different Calibration Blocks |
| Fig. 9(a-c): | Real image of regular prismoid |
REAL DATA
In the real data, respectively make calibration blocks, regular quadrangular, pentagonal and hexagonal frustum pyramids, as shown in Fig. 9. For each calibration pattern take two images from different positions with the camera. The image resolution is set to 640x480.
In order to verify the feasibility and accuracy of the algorithm and use Direct Linear Algorithm (DLT) for calibration and average the results (Hartley and Zisserman, 2004).
From Table 2, we can see that the camera intrinsic parameters uo (312-317) and vo (237-339) obtained from this chapter are much closer to their mean values (314, 238) than the DLT method.
In the method, the process of using linear equations to solve the parameters is simple and the robustness is much stronger.
CONCLUSION
This study proposes the linear calibration method which has the merits of real-time, simple operation and high accuracy. This method has the following characteristics: the calibration pattern is a regular prismoid. Take advantage of the property of vanishing point, camera need neither to do any movements nor to match characteristic point on the calibration pattern. Thus avoid errors caused by the irregular feature points extracted from image. The calibration process only needs two images of the calibration pattern. The solving method is linear. The experimental results show that the algorithm has a higher accuracy. There is great value in the field of vision measurement.
ACKNOWLEDGMENTS
This research was supported by the Natural Foundation of Yunnan Province, China (2011FB017).