论文部分内容阅读
Abstract: Omni-directional imaging system is becoming more and more common in reducing the maintenance fees and the number of cameras used as well as increasing the angle of view in a single camera. Due to omni-directional images are not directly understandable, an approach namely the un-warping process, has been implemented in converting the omni-directional image to a panoramic image, making it understandable. There are different kinds of methods used for the implementation of this approach. This paper evaluates the performance of the 3 universal un-warping methods currently applied actively around the world in transforming omni-directional image to panoramic image, namely the pano-mapping table method, discrete geometry method (DGT) and the log-polar mapping method. The algorithm of these methods will first be proposed, and the code will then be generated and be tested on several different omni-directional images. The images converted will then be compared among each other and be evaluated based on their performance on the resolutions, quality, algorithm used, complexity based on Big-O computations, processing time, and finally their data compression rate available for each of the methods. The most preferable un-warping method will then be concluded, taking into considerations all these factors.
Key words: Omnidirectional image, evaluation of image quality, un-warping, discrete geometry techniques, DGT (discrete geometry method), pano-mapping table method, log-polar mapping.
1. Introduction
With the widespread adoption of imaging system especially in surveillance or observation system, it is desirable to maximize the coverage area with the minimum number of cameras in order to reduce the cost, such as the maintenance fees and the cost of the camera, as well as reducing the complexity and troubles in installing and calibrating cameras. However, due to conventional type of cameras are usually having a very limited field of view, leading to the increased in network cost and the needs for mutual camera calibration as well as additional maintenance effort, omni-directional imaging system is devised in order to overcome this problem. Omni-directional imaging system uses a special type of mirror to reflect light rays from 360o into a perspective camera lens and forms an omni-directional image [1-2]. However, these captured images have different properties and characteristics compared to perspective images in terms of imaging deformation, making these images difficult to be directly understandable. Such distortion leads to the images being difficult to be directly implemented as well. Thus, it is necessary to work out an efficient method to make the images to be understandable, namely, the un-warping method.
Un-warping, generally, is a method used in digital image processing in “opening” up an omni-directional image into a panoramic image, or in other words, converting the image from a circular coordinate system
image into a Cartesean coordinate system image, making the information on the image to be able to be directly implemented and understood. Un-warping method is actively adopted in the application of camera visual surveillance system [3-5] as well as thermal monitoring system [6] in omni-directional imaging system.
There are 3 un-warping methods currently in practice actively around the world being studied, which are the pano-mapping table method [3], discrete geometry techniques (DGT) method [4], and log-polar mapping method [5]. This paper studies the advantages and disadvantages for each method, compared and evaluated their performance among each others in order to select an optimum unwarping method for omni-directional imaging.
The paper is organized in the following way: Section 2 will be briefly commented on the different types of un-warping method currently in practice actively around the world and their corresponding algorithm. Section 3 presents the experimental setup which is the few conditions or parameters that are needed to be determined before processing the conversion of omni-directional image to a panoramic image, while section 4 reports some experimental results obtained with the comparison. Finally in section 5, we draw some conclusion and envision future studies.
2. Currently Available Methods
There are three universal un-warping methods commonly used, which are the pano-mapping table method, discrete geometry techniques (DGT) method, and log-polar mapping method. Each of the methods’algorithm and their basic idea behind the implementation obtained from the materials will be discussed individually in the following section.
2.1 Pano-Mapping
This method uses a table, which is so-called the pano-mapping table, to process the image conversion. Pano-mapping table will be created “once and forever”, consisting of many co-ordinates corresponding to the co-ordinates taken from the omni-directional image which will then be mapped into a new panoramic image respectively. In other words, the table of M number of rows and N number of columns will be containing M × N number of entries, and each of these entries will store a co-ordinate value for each of the pixels that will be taken into mapping process from the omni-directional image, into a panoramic image. It is practically used in omni-directional visual tracking [7], and the unwarping process of omni-images taken by almost any kind of omni-cameras prior to requiring any knowledge about the camera parameters in advance, as proposed by S.W. Jeng, C.J. Wu, W.H. Tsai [3, 8].
In pano-mapping table method, it is required to select 5 landmark points from the omni-directional image first. These points will be taken from the same line, drawing from the center of the omni-image to the circumference of the image which in other words, called the radius of the image. 5 points in between the end of this line will be picked, and the value corresponding to their radius from the center is obtained. It is then used in order to obtain the 5 co-efficient of a0 through a4 in the “radial stretching function”, fr(ρ), described by the following 4th-degree polynomial function of
?????ρ???????ρ????ρ????ρ????ρ? (1)
where r correspond to the radius, ρ is the particular radius for the each of the 5 points taken, and a0~a4 are five co-efficient to be estimated using the values obtained from the landmark points.
Once the 5 co-efficient has been obtained, the pano-mapping table, TMN can then be generated. The size of the table will first be determined manually, by setting it to a table of size M × N. Hence, in order to fill up a table with M × N entries, the landmark point ρ, which correspond to the radius of the omni-directional image will be divided into M separated parts, and the angle, ? will be divided into N parts as follows:
2.2 Discrete Geometry Techniques (DGT)
Discrete Geometry Techniques (DGT) method, by the name itself, means that this technique is used by applying one-by-one, the geometry of the image, discretely, in order to successfully un-warp the omnidirectional image into a panoramic image. This method is practically used in transforming the omni-directional images into panoramic images on a cylindrical surface using PDE based resampling models [4].
In DGT method, it is required to perform the calculation of each and every pixels in the omni-image first, and then check for its corresponding radius from the center of the omni-directional image, and later to determine whether it should be considered or not. The calculations will start from a fixed position and direction, such as from the right, going counter-clockwise for 360o. For a radius of 1, a circle of radius 1 will be visualized in the center of omni-directional image, which in other words means that the circle will be in size of 3 × 3 pixels. All the pixels in this boundary of 3 × 3 pixels will be considered, and their corresponding radius will be calculated. For all pixels which fall into the radius of 1, which is the radius currently in concern, these pixels will be considered in the conversion. However, due to the pixels are generally an area of data information, it is possible that the circle will lie in between the pixels.
Therefore, a tolerance of ±? radius is set to counter this problem. In other words, a circle of radius 1, will consider the pixels lying in radius of 0.5 to 1.4, and a circle of radius 2, will consider the pixels lying in radius of 1.5 to 2.4, and so on. An example is shown in Fig. 1.
As soon as a pixel in the boundary is deemed to be considered or in range of the radius, it will be mapped into a new matrix of panoramic-image. However, since the pixels mapped into the panoramic image must be in order so that the image will not be distorted, the image will be split into 4 sections of 90o each, as show in Fig. 2, where each section will perform the calculations based on the direction move of the circle.
For example, for a circle drawn, starting from right in a counter-clockwise direction, the pixels in the section at the upper right part, will be taken and calculated one by one, from the bottom part of the
section, and from right to left, which will then be increased one by one, till the upper part of the section, from right to left as well. On the other hand, for the lower left part of the section, the calculation will go from the top of the section, going from left to right.
However, due to the pixels being considered for different circle of different radius will be non-uniform, as shown in Fig. 3, a re-sampling process is needed to standardize the pixels in every row of the panoramic image.
Therefore, after finished mapping every pixel in the whole omni-image onto the panoramic image plane, spacing will be inserted between pixels in every row(as shown by the black dots in Fig. 4) in order to standardize the resolution of the panoramic image for each row.
This will generate a standard resolution of panoramic image. However, due to spacing are generally empty pixels with no data information, a row with very little pixels will be hardly understandable. Therefore, the pixels will be duplicated over the spacing, instead of inserting empty pixels into it, and an understandable uniform resolution panoramic image can be generated, as shown in Fig. 5.
2.3 Log-Polar Mapping
Log-polar mapping is a type of spatially-variant image representation whereby pixel separations increase
linearly with distance. It enables the concentration of computational resource on a region of interest as well as maintaining the information from a wider view. This method is implemented by applying log-polar geometry representations. The captured omni-directional image will first be sampled by spatially-variant grid from a Cartesian form into a log-polar form. The spatially-variant grid representing log-polar mapping will then be formed by i number of concentric circles with N number of samples and the omni-directional image will then be un-warped into a panoramic image in another Cartesian form.
This method is practically used in robust image registration [5], or in robotic vision, particularly in visual attention, target tracking, ego motion estimation, and 3D perception [9] as well as in vision-based navigation, environmental representations and imaging geometries by J.S.-Victor and A. Bernardino [10].
In log-polar mapping method, the center pixel for log-polar sampling is calculated by
????,???? ??????????????????
????,??? ?sin???? (9)
where ??,?? is the center point of our original cartesean form co-ordinate, and N is the number of samples in each and every concentric circles taken. Fig. 6 shows the sampling of log-polar co-ordinate original(ρ, ?), into (xi, yi) in Cartesian form.
The center point will then be calculated by using Eqs.(6) and (7) to get the respective ρ and ?, which will cover a region of the original Cartesian pixels of radius:
???????? (10)
and(11)
where r is the sampling circle radius and b is the ratio between 2 apparent sampling circles. Fig. 7 shows the circular sampling structure and the unwarping process done by using the log-polar mapping method [5].
The mean value of pixels within each and every circular sampling will then be calculated and will be assigned to the center point of the circular sampling. The process will then continue by mapping the mean value at log-polar pixel (ρ, ?) into another Cartesian form using Eqs. (8) and (9), and the un-warping is done at the end of mapping. The next section will discuss about the methods in implementing each of the unwarping processes.
3. Algorithm and Implementation
In this section, each of the algorithms of the 3 universal unwarping methods will be described. 3.1 Pano-Mapping Table
In implementing the first method, which is the pano-mapping table method method, there are several important steps to follow based on the proposed materials in the previous page. The following steps are as follow:
Step 1: Estimation of landmark point pairs by landmark learning procedure. In this step, a few sets of landmark point pairs, whereby their co-ordinates can be easily identified in the world space points, such as the center of the image, or the edge of the image, is selected first, and the number of sets selected must be equal to, or more than 5 sets (n ≥ 5). This is because of the “radial stretching function” which is a 4-degree polynomial consisting of 5 unknown variables, which are a0, a1, a2, a3 and a4 that will be shown later in Step 2.
Hence, the 5 easiest to identify points are
? The center of the omni-image, with by the co-ordinate of (Xc, Yc);
? The left-most point from the center of the omni-image, given by the co-ordinate of (Xc-r, Yc);
? The right-most point from the center of the omni-image, given by the co-ordinate of (Xc+r, Yc);
? The top-most point from the center of the omni-image, given by the co-ordinate of (Xc, Yc-r);
? The bottom-most point from the center of the omni-image, given by the co-ordinate of (Xc, Yc+r).
where r denotes the radius of the omni-image, and the co-ordinate (Xc, Yc) is the co-ordinate of the center of the omni-image.
Step 2: Generation of the pano-mapping table. Since the 5 landmark point pairs had been selected, the next step is to generate the pano-mapping table by using all the information provided by the landmark point pairs. This table will basically be arranged in the sense that the row denotes each of the elevation angles, ρ, taken,
and the column specifying each of the azimuth angles,θ, taken into consideration.
Hence, in order to generate a pano-mapping table, Tpm, with M × N number of entries, the azimuth angles,θ, of 2π, or 360o equally will be divided with M units, and that the elevation angles of one point to another point, for example, from ρ1 to ρ2, equally divided by N units (Table 1).
And (uij, vij), where i = 1, 2, 3 ... M and j = 1, 2, 3 ... N, denotes the pixels at the coordinate (u, v) at ith position of azimuth angle, and jth position of elevation angles taken.
Each entries, of (θ, ρ) in this table, will be mapped into each of the pixels, say p, with co-ordinates of (u, v) in the input omni-image. By referring to the table above, the mappings of the table can actually be decomposed into 2 separate components, which will be the mapping in the azimuth direction, also known as azimuth-directional mapping, and the mapping in the radial direction, or the radial-directional mapping.
Due to the rotation-invariant property of the omni-camera, the azimuth angle, θ, of the world space point, is actually the same as the angle ? of the respective pixel p, with respect to the u-axis in the input omni-image. Hence, it is said that the azimuth-directional mapping is actually an identity function of fa in a way that
???θ????θ (12)
However, it is a different case for the radial-directional mapping due to the non-linear property of the mirror surface shape, hence a non-linear function, denoted by fr, also known as the “radial stretching function”, shall be used, where the radial distance r from each image pixel of p of co-ordinates of (u, v) in the omni-image, with respect to the image center Oc at (u0, v0) can be denoted by
?????ρ? (13)
and a 4th-degree polynomial function is given to denote the non-linear function of fr(ρ) such as:
?????ρ??a??a?ρ??a?ρ??a?ρ??a?ρ? (14)
where a0, a1, a2, a3, and a4 are the coefficients to be determined using the selected landmark point pairs earlier.
By using the two mapping function which are the azimuth-directional mapping and radial-directional mapping, by using the equality of
??,???????ρ?,θ? (15)
A polar-coordinate system is formed with the following image co-ordinates (u, v), using the equations below:
???cos??f??ρ?cosθ (16)???sin??f??ρ?sinθ (17)
Since there are 5 unknown variables in the radial stretching function, this is where the 5 or more landmark point pairs will be used in order to estimate the coefficients of the radial stretching function. In order to get the coefficients of a0-a4 in the radial stretching function, the co-ordinates of the respective landmark point pairs with respect to the image center need to be obtained first. This can be done by subtracting the selected landmark point pair’s co-ordinate (xk, yk) with the image center at (x0, y0).
These landmark point co-ordinates will then be used to calculate the elevation angle ρk and the radial distance rk in the image by the following equations:
??ρ? (22)
where i = 0, 1, 2, ... , M-1 and j = 0, 1, 2, ... , N-1.
With the M sets of azimuth angles, and N sets of elevation angles, the value of ρj for each j, is then substituted into the radial stretching functions in order to get the respective rj. This rj will then be used in order to obtain the values of each u and v to be filled into the pano-mapping table, by using the equation as follows:
???????cosθ? (23)???????sinθ? (24)
Step 3: Image un-warping procedure. Since the pano-mapping table of M × N entries had been filled, the generic panoramic image can then be re-constructed from the omni-image, say Q, with the help of the pano-mapping table. First of all, the co-ordinate of (uij, vij) is extracted out from each of the entries in the table corresponding to the specific azimuth angle, θi and elevation angle ρj. Once obtaining the specific co-ordinate for specific azimuth and elevation angles, the color values of the pixel pij contained in the omni-image at that particular co-ordinate (uij, vij) will be copied to the pixels of a new image, say P, at co-ordinate (i, j). After finished copying all the color values in every co-ordinates specified by every entries in the table into the new image P, the end result will be P, being the converted panoramic-image from the omni-directional image Q, with the use of pano-maping table method. 3.2 Discrete Geometry Techniques (DGT)
In implementing the second method, which is the DGT method, there are several important steps to follow based on the proposed materials in the previous page. The following steps are as follow:
Step 1: Finding the center points. First of all, the(27)
Step 3: Estimate the length of the panoramic circle. The maximum length of the generated panoramic image will obviously be constituted by the perimeter of the largest circle, or in other words, the circle of pixels furthest from the center points, of that particular omni-directional image. The perimeter of a circle, or the length, can be calculated using the equation below:
?????? ? 2?? (28)
where R is the radius of the circle, denoted by ro. This will gives us an approximation of the dimensions of the generated panoramic images, which is of length x radius resolution, and since the pixels are in matrices form, the calculated result is rounded up into an integer as the index or the pixel co-ordinate only accepts integer values.
Step 4: Splitting the circle into 4 for easier calculations. The omni-directional image is then divided into 4 parts, each of 90 degrees as shown in Fig. 8.
Step 5: Checking for all pixels that matched into the radius. A circle will then be taken, surrounding the center points for radius of 1, increasing until the length of radius = ro, which is the largest circle available in the omni-directional image, or in other words, the radius of the omni-directional image used. This is also the boundary of the DGT calculations. For each radius taken, all the pixels will be checked whether it matches with the radius taken or not. Let us start checking the points from the bottom part, which is 6 o’clock of the
circle. This point will be on the coordinate of
x_coordinate ? x_center; and (29) y_coordinate ? y_center ?radius (30)
It can then be performed to check whether each and every pixels is included in the circle or not.
Since these are of pixels and coordinates, there might be cases where the circle of such radius, lies between 2 pixels as illustrated in Fig. 9.
In this case, we will need to determine whether to take the outer pixel, or the inner pixel from the circle of that specific radius. Hence, a tolerance is set so that it takes a pixel that satisfies the length of the radius ± 0.5 from the center point. For example, when considering pixels that is matched into the circle of radius 5, the pixels of radius ranging from 4.6 to 5.5 units from the center points, will be taken into consideration.
Step 6: Extracting each pixels taken into a new matrix for a specific radius. For the lower left part of the quarter of the circle, and since the circle will be going in a clockwise direction, the movement of the circle will be from right to left, and it will be wise to check the pixels from right to left. Hence, we can first set a fixed value of y, that’s to say:
y?y_center ? radius; and (31) x ? x_center (32)
The x value can then be decreased one by one, until it reaches the boundary, which will be
x_boundary ? x_center ? radius (33)
Once the co-ordinate of x had reached its boundary value, the y coordinate can then be decreased by 1, and to check again for each and every x coordinate starting from x_center to x_boundary for that specific y. While each and every point are taken into considerations, we can check for its length from the radius. If it satisfied the condition of radius ± 0.5, then it will be taken into consideration, and the pixel value of that particular coordinate, will be copied into another matrix for future usage.
Whereas for the upper left part of the quarter of the circle, since the circle will going from left to right, it will then be wise to check for each and every x coordinate starting from x_boundary to x_center, with the decrement of y_coordinate for every completion of x_boundary to x_center checking.
The same methods can then be applied to the upper right, and lower right of the quarter of the circle, and each and every pixels’ values corresponding to that particular radius (together with the tolerance), can be copied into another new matrix.
Step 7: Extracting pixels from radius of 1 till ro. After finishing for the particular radius value, the radius should then be increase one-by-one, and to check until it reaches its boundary, which is the radius of the omni-directional image.
Step 8: Standardizing the length of panoramic images for every y-coordinate. Since the maximum number of pixels available in x- and y-coordinate of the panoramic images is not known, or in other words, the exact amount of pixels which will be considered in the largest circle that satisfied the radius with its tolerance of ± 0.5 is unknown, a variable is used to increment one by one when a point is considered for the largest circle, which will gives us the exact amount of pixels available, or the amount of pixels that had been considered, in the largest circle. This will then be the maximum length of our panoramic image in
x-coordinate, whereas for the height of the panoramic image, it is simply the radius of the omni-directional image.
However, since this is an omni-directional images converting into a panoramic images, the circle nearest to the center point will have least pixels extracted, and the circle furthest to the center point will have the most pixels considered. And this will make the panoramic image produced, or the matrix where the pixels information are copied to, in a triangular shape. This happens because the x-coordinate is set to increase 1 by 1 in the coordinate of the panoramic images, for a specific y-coordinate obtained from the current radius. Fig. 10 shows the allocation of pixels in the panoramic images.
So, in order to make it standardized, where each and every y-coordinate (or radius) will be having the same total length, the ratio of the number of pixels for that particular y-coordinate (or radius), corresponding to the maximum pixels contained in the image can be calculated, such as:(34)
This will give a brief understanding on how big, or how long should the spacing between pixels be assigned in that particular y-coordinate, so that the total length in that y-coordinate (or radius) can be equal to that of the longest length in the panoramic images, which is the number of the total pixels in the largest circle. This is done, in order to get a square matrix or a square images instead of a triangle shaped images, and the resultant image can have a look in Fig. 11, where the black color area denotes the spacing between pixels.
Step 9: Generating a uniform resolution panoramic image. Due to the increasing in spacing between pixels for smaller radius, the image might not be viewable to the eye of human, especially for the bottom part of the panoramic image. Therefore, in order to fill up these gaps or spacing between pixels, these pixels are actually duplicated over the spacing to make the image
information viewable. In other words, to solve this problem, we can try to generate a uniform resolution panoramic image, by continuously copying the value of the pixels into the spacing created in the image, before the next bit of pixel. Fig. 12 shows the end result of the above procedure.
Hence the image will then be able to be produced in a square form, instead of a triangle form.
Following up is the log-polar mapping method. 3.3 Log-Polar Mapping
In implementing the last method, which is the log-polar mapping method, there are several important steps to follow based on the proposed materials in the previous page. The following steps are as follow:
Step 1: Determining the periphery part of the omni-directional image. In this step, we first divide the omni-directional image into N number of samples, for each radius in M number of radius. In other words, the omni-directional image will be separated into M number of radius, with each radius having N number of samples separated by a radius of
?? (35)
where R is the radius of the points taken in the particular radius from the center point, N is the number of samples, and i is the ith radius currently taken into consideration. For the first radius to be taken, i = 1, and for the second radius to be taken, i = 2, and so on. Once the R is calculated, the radius of the omni-directional image taken will then be incremented by 1 to calculate for the radius at this sample circle. The radius taken will be increased by M times while calculating the radius, R, of the point in each and every particular radius. These values will then be used to obtain the xand y-coordinate in the omni-directional image, given by
??????????????cos? (36)??????????????sin? (37)
where θ is equals to? (38)
and j is the jth sample taken into consideration at that time. In other words, for a particular radius, theta will be increased from 1 up to 2π for N number of times. Therefore, we will obtain approximately N number of xper and yper values. These will be the x and y coordinates used in the omni-directional image.
The distance between these points will then be calculated by taking:
?????????? (39)?????????? (40)
where k is the kth sample taken in the particular radius, x1 is the distance between points in x-coordinate, and y1 is the distance between points in y-coordinate. Once the distance between the points had been obtained, these points can then be allocated a rectangular grid sampling with the width equals to x1 and the height equals to y1. Calculations will then be proceeded to obtain the mean intensity in each and every rectangular sampling by summing up all the color values in the rectangular sample pixels divided by the total amount of pixels available in the sample. The entire rectangular sample will then be replaced by this mean pixel values? (43)
N is the number of samples taken, same as in Step 1, and l is the lth sample taken. After obtaining the points for (xfov, yfov), the pixels at that point will be mapped into the matrix O. For panoramic image using fovea, the image and the matrix V is then divided into C number of radius and D number of samples in a particular radius for the inner part of the omni-directional image.
Step 3: Combining the periphery and fovea part. In this step, we will then combine the two matrices obtained from Step 1 and Step 2 into a new matrix. Since Step 1 is processing on the periphery part which is the outer part, and step 2 is processing on the fovea part which is the inner part, the combination of these two matrices will produce a complete version of panoramic image. Therefore the conversion is completed.
4. Results and Discussions.
In this section, the panoramic images will first be compared using the 3 universal un-warping methods, which are the pano-mapping table method, Discrete Geometry Techniques (DGT) method, and log-polar mapping method. There are several aspects taken into considerations that are used for comparison, which are
? Resolutions;
? Quality of image;
? Algorithm used;
? Complexity;
? Data compression;? Processing time.
It will then be discussed to determine which methods overwhelm the others. 4.1 Resolutions
The resolutions of each generated panoramic images after the conversion of log-polar method, discrete geometry techniques and pano-mapping method is such that, the log-polar method provides a smaller resolution of length equals to the radius and the width equals to 1/4 of the radius of the omni-directional image, whereas for the discrete geometry methods and pano-mapping method, the resolution of the panoramic-image produced can be as large as the length of the perimeter of the omni-directional image, with the width equals to the radius of the omni-directional image. Fig. 13 shows the original un-warped omni-directional image.
Fig. 14a shows the panoramic image generated by using log-polar mapping method, whereas Fig. 14b shows the panoramic image generated by using Discrete Geometry Techniques (DGT), and Fig. 14c by using pano-mapping table method.
4.2 Quality of Image
Based on the figure above, it is clear that pano-mapping method produces the highest quality of image, as compared to the other 2 methods, followed by the “Log-Polar Mapping” method which is slightly lower, and lastly, the DGT method.
4.3 Algorithm Used
The algorithm used in performing log-polar mapping method is that, the omni-directional image is considered in the form of a number of sectors which consists of a group of pixels, that will then be extracted sector by sector to be arranged into a rectangular form of image, whereas for the discrete geometry techniques, we are considering pixels by pixels to be extracted, to be arranged into a rectangular form image. These pixels
will then be re-produced, or duplicated, in order to standardize the number of pixels available in each row of the panoramic image. For the pano-mapping method, it is using an algorithm where a table will be created at initialization, to indicate the co-ordinates of the pixels to be extracted from the omni-directional image. Once the table is created, it will then be used over and over again to map each of the pixel at that particular co-ordinate, one by one, from the omni-directional image into a panoramic image, hence the name “once and forever”. 4.4 Complexity
The complexity of using log-polar mapping method, discrete geometry techniques, and pano-mapping method, as in the code used in generating or converting the omni-directional image to a panoramic image, are different as well. Table 2 shows the Big-O complexity
DGT, log-polar mapping and pano-mapping method, where X denotes the length of the panoramic image, which is also the perimeter of the omni-directional image taken into considerations, and Y denotes the height of the panoramic image, which is also the radius of the omni-directional image taken into considerations as shown in Fig. 15. 4.5 Data Compression
Out of all the 3 methods in concern, the generated panoramic image produces by log-polar method has the resolution, which is 473 × 114, DGT having a resolution of 1472 × 235, and 1146 × 243 for pano-mapping method, where the original omni-directional image is of resolution 473 × 473. From the resolutions, it is clear that log-polar mapping has the highest compression, which compresses the image to approximately 1/4 of its original size, compared to DGT and pano-mapping, which expanded the image.
4.6 Processing Time
The processing time for all the 3 methods to successfully convert an omni-directional image into a panoramic image is performed using matlab function“cputime”. The codes are processed 5 times, and the average processing time is computed. It is found that log-polar having the fastest computation time, which is 2.80176 seconds, followed by pano-mapping method having almost the same computation time, which is 2.84232 seconds, and lastly, 4.87344 seconds for the DGT method, which is the slowest out of the 3 methods. 4.7 Discussion between the Three Methods
In terms of resolution, although the image generated by discrete geometry techniques and pano-mapping method is larger as compared to the image generated by log-polar mapping method, these 2 methods seem to elongate the actual size of the image. In other words, this method tends to makes the objects in the image extended, and “fatter” than the original image. (Due to the image above being scaled to a smaller image, the difference is not really obvious). Due to this elongation, it will be harder to examine the picture and the objects, as the sense of the size had been eliminated. Whereas for the log-polar method, the extension is not as much and as obvious as DGT and pano-mapping methods.
In terms of quality of image, it is generally the most important factor that constitute to the preference of a method, and it is obvious that pano-mapping method produces the highest quality out of the 3 methods, followed by a slightly lower quality which is still within an acceptable range for the log-polar method, and the blurred DGT method.
In terms of algorithm used, pano-mapping method uses the simplest and easy algorithm, followed by a simple but slightly complex algorithm which is the log-polar method, and lastly, a complicated and complex algorithm from the DGT method.
In terms of complexity, by using the computations of the Big-O complexity, it is found that pano-mapping having the least complexity, followed by DGT, and lastly log-polar mapping methods.
In terms of data compression, log-polar mapping method is shown to have the best data compression rate out of the other 3 methods. This is very good in preserving memory in CPU as our memory is very limited.
In terms of processing time, on average, log polar mapping has the fastest processing time to convert 1 omni-directional image into a panoramic image, followed by pano-mapping method which is slightly slower, and DGT method, which is almost 2 times slower than log-polar and pano-mapping method.
In the final section, we will be going to a conclusion on which un-warping methods are the most efficient, taking into considerations of all the aspects compared previously.
5. Conclusions
A good quality of image is usually the most important factor, taking into consideration of other minor factors such as the processing speed and the complexity, in order to optimize the performance. After several considerations and research, log-polar method achieves the highest preference compared to the other 2 methods. This is because, while pano-mapping method produces a highest quality image out of the 3 methods, it also generates a large size of the panoramic image, which is very bulky and waste of memory space as the number of images generated increase from time to time. Besides that, even though log-polar method produces a slightly lower quality compared to pano-mapping method, it is still within an acceptable range and the image is still clear enough to be recognized, and that the data compression is most magnificent compared to the DGT and pano-mapping method.
By taking the highest complexity of Big-O in each of the methods, although log-polar has a highest complexity, which is X2Y2 compared to DGT of XY2, and pano-mapping with complexity of Y2, the time needed to process or to convert an omni-directional image into a panoramic image is sufficiently small, which is 2.80176 seconds on average, compared to the other 2 methods which are 4.87344 seconds for DGT and 2.84232 for pano-mapping, and is clearly the fastest out of the 3 methods. Hence, the complexity is not of much concern here.
As for the resolution of log-polar method which is smaller than DGT and pano-mapping converted panoramic image, it can basically be fixed by taking a larger resolution image to be processed, and the disadvantages of log-polar having small resolution can be solved. Furthermore, the algorithm used for log-polar methods is sufficiently simple and easy to understand as well.
Therefore, log-polar methods is most preferred to be used in omni-directional to panoramic image conversion, given that it has a good quality of image with the highest data compression rate, and a fastest processing speed out of the 3 methods.
References
[1] T. Kawanishi, K. Yamazawa, H. Iwasa, H. Takemura, N. Yokoya, Generation of high-resolution stereo panoramic images by omnidirectional imaging sensor using hexagonal pyramidal mirrors, in: Proc. 14th Int. Conf. in Pattern Recognition, Aug. 1998, Vol. 1, pp. 485-489.
[2] T. Padjla, H. Roth, Panoramic imaging with SVAVISCA camera: simulations and reality, Research Reports of CMP, Czech Technical University in Prague, No. 16, 2000.
[3] S.W. Jeng, W.H. Tsai, Using pano-mapping tables for unwarping of omni-images into panoramic and perspective-view images, IET Image Processing 1 (2)(2007) 149-155.
[4] A. Torii, A. Imiya, Panoramic image transform of omnidirectional images using discrete geometry techniques, in: Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04), 2004, pp. 608-615.
[5] G. Wolberg, S. Zokai, Robust image registration using log-polar transform, in: Proc. of IEEE Intl. Conf. on Image Processing, Sep. 2000, pp. 493-496.
[6] W.K. Wong, P.N. Tan, C.K. Loo, W.S. Lim, Omnidirectional surveillance system using thermal
camera, Journal of Computer Science and Engineering 3(2) (2010) 42-51.
[7] D.S. Huang, D.C. Wunsch, D.S. Levine, K-H Jo, Advanced intelligent computing theories and applications: with aspects of theoretical and methodological issues, in: 4th International Conference on Intelligent Computing(ICIC 2008), Shanghai, China, September 2008.
[8] C.J. Wu, W.H. Tsai, Unwarping of images taken by misaligned omnicameras without camera calibration by curved quadrilateral morphing using quadratic pattern classifiers, Opt. Eng. 48 (2009) 087003(1)-087003(11).
[9] V.J. Traver, A. Bernardino, A review of log-polar imaging for visual perception in robotics, Robotics and Autonomous Systems 58 (2010) 378-398.
[10] J.S.-Victor, A. Bernardino, Vision-based navigation, environmental representations and imaging geometries, in Proc. 10th Int. Symp. of Robotics Research, 2001, pp. 347-360.
Key words: Omnidirectional image, evaluation of image quality, un-warping, discrete geometry techniques, DGT (discrete geometry method), pano-mapping table method, log-polar mapping.
1. Introduction
With the widespread adoption of imaging system especially in surveillance or observation system, it is desirable to maximize the coverage area with the minimum number of cameras in order to reduce the cost, such as the maintenance fees and the cost of the camera, as well as reducing the complexity and troubles in installing and calibrating cameras. However, due to conventional type of cameras are usually having a very limited field of view, leading to the increased in network cost and the needs for mutual camera calibration as well as additional maintenance effort, omni-directional imaging system is devised in order to overcome this problem. Omni-directional imaging system uses a special type of mirror to reflect light rays from 360o into a perspective camera lens and forms an omni-directional image [1-2]. However, these captured images have different properties and characteristics compared to perspective images in terms of imaging deformation, making these images difficult to be directly understandable. Such distortion leads to the images being difficult to be directly implemented as well. Thus, it is necessary to work out an efficient method to make the images to be understandable, namely, the un-warping method.
Un-warping, generally, is a method used in digital image processing in “opening” up an omni-directional image into a panoramic image, or in other words, converting the image from a circular coordinate system
image into a Cartesean coordinate system image, making the information on the image to be able to be directly implemented and understood. Un-warping method is actively adopted in the application of camera visual surveillance system [3-5] as well as thermal monitoring system [6] in omni-directional imaging system.
There are 3 un-warping methods currently in practice actively around the world being studied, which are the pano-mapping table method [3], discrete geometry techniques (DGT) method [4], and log-polar mapping method [5]. This paper studies the advantages and disadvantages for each method, compared and evaluated their performance among each others in order to select an optimum unwarping method for omni-directional imaging.
The paper is organized in the following way: Section 2 will be briefly commented on the different types of un-warping method currently in practice actively around the world and their corresponding algorithm. Section 3 presents the experimental setup which is the few conditions or parameters that are needed to be determined before processing the conversion of omni-directional image to a panoramic image, while section 4 reports some experimental results obtained with the comparison. Finally in section 5, we draw some conclusion and envision future studies.
2. Currently Available Methods
There are three universal un-warping methods commonly used, which are the pano-mapping table method, discrete geometry techniques (DGT) method, and log-polar mapping method. Each of the methods’algorithm and their basic idea behind the implementation obtained from the materials will be discussed individually in the following section.
2.1 Pano-Mapping
This method uses a table, which is so-called the pano-mapping table, to process the image conversion. Pano-mapping table will be created “once and forever”, consisting of many co-ordinates corresponding to the co-ordinates taken from the omni-directional image which will then be mapped into a new panoramic image respectively. In other words, the table of M number of rows and N number of columns will be containing M × N number of entries, and each of these entries will store a co-ordinate value for each of the pixels that will be taken into mapping process from the omni-directional image, into a panoramic image. It is practically used in omni-directional visual tracking [7], and the unwarping process of omni-images taken by almost any kind of omni-cameras prior to requiring any knowledge about the camera parameters in advance, as proposed by S.W. Jeng, C.J. Wu, W.H. Tsai [3, 8].
In pano-mapping table method, it is required to select 5 landmark points from the omni-directional image first. These points will be taken from the same line, drawing from the center of the omni-image to the circumference of the image which in other words, called the radius of the image. 5 points in between the end of this line will be picked, and the value corresponding to their radius from the center is obtained. It is then used in order to obtain the 5 co-efficient of a0 through a4 in the “radial stretching function”, fr(ρ), described by the following 4th-degree polynomial function of
?????ρ???????ρ????ρ????ρ????ρ? (1)
where r correspond to the radius, ρ is the particular radius for the each of the 5 points taken, and a0~a4 are five co-efficient to be estimated using the values obtained from the landmark points.
Once the 5 co-efficient has been obtained, the pano-mapping table, TMN can then be generated. The size of the table will first be determined manually, by setting it to a table of size M × N. Hence, in order to fill up a table with M × N entries, the landmark point ρ, which correspond to the radius of the omni-directional image will be divided into M separated parts, and the angle, ? will be divided into N parts as follows:
2.2 Discrete Geometry Techniques (DGT)
Discrete Geometry Techniques (DGT) method, by the name itself, means that this technique is used by applying one-by-one, the geometry of the image, discretely, in order to successfully un-warp the omnidirectional image into a panoramic image. This method is practically used in transforming the omni-directional images into panoramic images on a cylindrical surface using PDE based resampling models [4].
In DGT method, it is required to perform the calculation of each and every pixels in the omni-image first, and then check for its corresponding radius from the center of the omni-directional image, and later to determine whether it should be considered or not. The calculations will start from a fixed position and direction, such as from the right, going counter-clockwise for 360o. For a radius of 1, a circle of radius 1 will be visualized in the center of omni-directional image, which in other words means that the circle will be in size of 3 × 3 pixels. All the pixels in this boundary of 3 × 3 pixels will be considered, and their corresponding radius will be calculated. For all pixels which fall into the radius of 1, which is the radius currently in concern, these pixels will be considered in the conversion. However, due to the pixels are generally an area of data information, it is possible that the circle will lie in between the pixels.
Therefore, a tolerance of ±? radius is set to counter this problem. In other words, a circle of radius 1, will consider the pixels lying in radius of 0.5 to 1.4, and a circle of radius 2, will consider the pixels lying in radius of 1.5 to 2.4, and so on. An example is shown in Fig. 1.
As soon as a pixel in the boundary is deemed to be considered or in range of the radius, it will be mapped into a new matrix of panoramic-image. However, since the pixels mapped into the panoramic image must be in order so that the image will not be distorted, the image will be split into 4 sections of 90o each, as show in Fig. 2, where each section will perform the calculations based on the direction move of the circle.
For example, for a circle drawn, starting from right in a counter-clockwise direction, the pixels in the section at the upper right part, will be taken and calculated one by one, from the bottom part of the
section, and from right to left, which will then be increased one by one, till the upper part of the section, from right to left as well. On the other hand, for the lower left part of the section, the calculation will go from the top of the section, going from left to right.
However, due to the pixels being considered for different circle of different radius will be non-uniform, as shown in Fig. 3, a re-sampling process is needed to standardize the pixels in every row of the panoramic image.
Therefore, after finished mapping every pixel in the whole omni-image onto the panoramic image plane, spacing will be inserted between pixels in every row(as shown by the black dots in Fig. 4) in order to standardize the resolution of the panoramic image for each row.
This will generate a standard resolution of panoramic image. However, due to spacing are generally empty pixels with no data information, a row with very little pixels will be hardly understandable. Therefore, the pixels will be duplicated over the spacing, instead of inserting empty pixels into it, and an understandable uniform resolution panoramic image can be generated, as shown in Fig. 5.
2.3 Log-Polar Mapping
Log-polar mapping is a type of spatially-variant image representation whereby pixel separations increase
linearly with distance. It enables the concentration of computational resource on a region of interest as well as maintaining the information from a wider view. This method is implemented by applying log-polar geometry representations. The captured omni-directional image will first be sampled by spatially-variant grid from a Cartesian form into a log-polar form. The spatially-variant grid representing log-polar mapping will then be formed by i number of concentric circles with N number of samples and the omni-directional image will then be un-warped into a panoramic image in another Cartesian form.
This method is practically used in robust image registration [5], or in robotic vision, particularly in visual attention, target tracking, ego motion estimation, and 3D perception [9] as well as in vision-based navigation, environmental representations and imaging geometries by J.S.-Victor and A. Bernardino [10].
In log-polar mapping method, the center pixel for log-polar sampling is calculated by
????,???? ??????????????????
????,??? ?sin???? (9)
where ??,?? is the center point of our original cartesean form co-ordinate, and N is the number of samples in each and every concentric circles taken. Fig. 6 shows the sampling of log-polar co-ordinate original(ρ, ?), into (xi, yi) in Cartesian form.
The center point will then be calculated by using Eqs.(6) and (7) to get the respective ρ and ?, which will cover a region of the original Cartesian pixels of radius:
???????? (10)
and(11)
where r is the sampling circle radius and b is the ratio between 2 apparent sampling circles. Fig. 7 shows the circular sampling structure and the unwarping process done by using the log-polar mapping method [5].
The mean value of pixels within each and every circular sampling will then be calculated and will be assigned to the center point of the circular sampling. The process will then continue by mapping the mean value at log-polar pixel (ρ, ?) into another Cartesian form using Eqs. (8) and (9), and the un-warping is done at the end of mapping. The next section will discuss about the methods in implementing each of the unwarping processes.
3. Algorithm and Implementation
In this section, each of the algorithms of the 3 universal unwarping methods will be described. 3.1 Pano-Mapping Table
In implementing the first method, which is the pano-mapping table method method, there are several important steps to follow based on the proposed materials in the previous page. The following steps are as follow:
Step 1: Estimation of landmark point pairs by landmark learning procedure. In this step, a few sets of landmark point pairs, whereby their co-ordinates can be easily identified in the world space points, such as the center of the image, or the edge of the image, is selected first, and the number of sets selected must be equal to, or more than 5 sets (n ≥ 5). This is because of the “radial stretching function” which is a 4-degree polynomial consisting of 5 unknown variables, which are a0, a1, a2, a3 and a4 that will be shown later in Step 2.
Hence, the 5 easiest to identify points are
? The center of the omni-image, with by the co-ordinate of (Xc, Yc);
? The left-most point from the center of the omni-image, given by the co-ordinate of (Xc-r, Yc);
? The right-most point from the center of the omni-image, given by the co-ordinate of (Xc+r, Yc);
? The top-most point from the center of the omni-image, given by the co-ordinate of (Xc, Yc-r);
? The bottom-most point from the center of the omni-image, given by the co-ordinate of (Xc, Yc+r).
where r denotes the radius of the omni-image, and the co-ordinate (Xc, Yc) is the co-ordinate of the center of the omni-image.
Step 2: Generation of the pano-mapping table. Since the 5 landmark point pairs had been selected, the next step is to generate the pano-mapping table by using all the information provided by the landmark point pairs. This table will basically be arranged in the sense that the row denotes each of the elevation angles, ρ, taken,
and the column specifying each of the azimuth angles,θ, taken into consideration.
Hence, in order to generate a pano-mapping table, Tpm, with M × N number of entries, the azimuth angles,θ, of 2π, or 360o equally will be divided with M units, and that the elevation angles of one point to another point, for example, from ρ1 to ρ2, equally divided by N units (Table 1).
And (uij, vij), where i = 1, 2, 3 ... M and j = 1, 2, 3 ... N, denotes the pixels at the coordinate (u, v) at ith position of azimuth angle, and jth position of elevation angles taken.
Each entries, of (θ, ρ) in this table, will be mapped into each of the pixels, say p, with co-ordinates of (u, v) in the input omni-image. By referring to the table above, the mappings of the table can actually be decomposed into 2 separate components, which will be the mapping in the azimuth direction, also known as azimuth-directional mapping, and the mapping in the radial direction, or the radial-directional mapping.
Due to the rotation-invariant property of the omni-camera, the azimuth angle, θ, of the world space point, is actually the same as the angle ? of the respective pixel p, with respect to the u-axis in the input omni-image. Hence, it is said that the azimuth-directional mapping is actually an identity function of fa in a way that
???θ????θ (12)
However, it is a different case for the radial-directional mapping due to the non-linear property of the mirror surface shape, hence a non-linear function, denoted by fr, also known as the “radial stretching function”, shall be used, where the radial distance r from each image pixel of p of co-ordinates of (u, v) in the omni-image, with respect to the image center Oc at (u0, v0) can be denoted by
?????ρ? (13)
and a 4th-degree polynomial function is given to denote the non-linear function of fr(ρ) such as:
?????ρ??a??a?ρ??a?ρ??a?ρ??a?ρ? (14)
where a0, a1, a2, a3, and a4 are the coefficients to be determined using the selected landmark point pairs earlier.
By using the two mapping function which are the azimuth-directional mapping and radial-directional mapping, by using the equality of
??,???????ρ?,θ? (15)
A polar-coordinate system is formed with the following image co-ordinates (u, v), using the equations below:
???cos??f??ρ?cosθ (16)???sin??f??ρ?sinθ (17)
Since there are 5 unknown variables in the radial stretching function, this is where the 5 or more landmark point pairs will be used in order to estimate the coefficients of the radial stretching function. In order to get the coefficients of a0-a4 in the radial stretching function, the co-ordinates of the respective landmark point pairs with respect to the image center need to be obtained first. This can be done by subtracting the selected landmark point pair’s co-ordinate (xk, yk) with the image center at (x0, y0).
These landmark point co-ordinates will then be used to calculate the elevation angle ρk and the radial distance rk in the image by the following equations:
??ρ? (22)
where i = 0, 1, 2, ... , M-1 and j = 0, 1, 2, ... , N-1.
With the M sets of azimuth angles, and N sets of elevation angles, the value of ρj for each j, is then substituted into the radial stretching functions in order to get the respective rj. This rj will then be used in order to obtain the values of each u and v to be filled into the pano-mapping table, by using the equation as follows:
???????cosθ? (23)???????sinθ? (24)
Step 3: Image un-warping procedure. Since the pano-mapping table of M × N entries had been filled, the generic panoramic image can then be re-constructed from the omni-image, say Q, with the help of the pano-mapping table. First of all, the co-ordinate of (uij, vij) is extracted out from each of the entries in the table corresponding to the specific azimuth angle, θi and elevation angle ρj. Once obtaining the specific co-ordinate for specific azimuth and elevation angles, the color values of the pixel pij contained in the omni-image at that particular co-ordinate (uij, vij) will be copied to the pixels of a new image, say P, at co-ordinate (i, j). After finished copying all the color values in every co-ordinates specified by every entries in the table into the new image P, the end result will be P, being the converted panoramic-image from the omni-directional image Q, with the use of pano-maping table method. 3.2 Discrete Geometry Techniques (DGT)
In implementing the second method, which is the DGT method, there are several important steps to follow based on the proposed materials in the previous page. The following steps are as follow:
Step 1: Finding the center points. First of all, the(27)
Step 3: Estimate the length of the panoramic circle. The maximum length of the generated panoramic image will obviously be constituted by the perimeter of the largest circle, or in other words, the circle of pixels furthest from the center points, of that particular omni-directional image. The perimeter of a circle, or the length, can be calculated using the equation below:
?????? ? 2?? (28)
where R is the radius of the circle, denoted by ro. This will gives us an approximation of the dimensions of the generated panoramic images, which is of length x radius resolution, and since the pixels are in matrices form, the calculated result is rounded up into an integer as the index or the pixel co-ordinate only accepts integer values.
Step 4: Splitting the circle into 4 for easier calculations. The omni-directional image is then divided into 4 parts, each of 90 degrees as shown in Fig. 8.
Step 5: Checking for all pixels that matched into the radius. A circle will then be taken, surrounding the center points for radius of 1, increasing until the length of radius = ro, which is the largest circle available in the omni-directional image, or in other words, the radius of the omni-directional image used. This is also the boundary of the DGT calculations. For each radius taken, all the pixels will be checked whether it matches with the radius taken or not. Let us start checking the points from the bottom part, which is 6 o’clock of the
circle. This point will be on the coordinate of
x_coordinate ? x_center; and (29) y_coordinate ? y_center ?radius (30)
It can then be performed to check whether each and every pixels is included in the circle or not.
Since these are of pixels and coordinates, there might be cases where the circle of such radius, lies between 2 pixels as illustrated in Fig. 9.
In this case, we will need to determine whether to take the outer pixel, or the inner pixel from the circle of that specific radius. Hence, a tolerance is set so that it takes a pixel that satisfies the length of the radius ± 0.5 from the center point. For example, when considering pixels that is matched into the circle of radius 5, the pixels of radius ranging from 4.6 to 5.5 units from the center points, will be taken into consideration.
Step 6: Extracting each pixels taken into a new matrix for a specific radius. For the lower left part of the quarter of the circle, and since the circle will be going in a clockwise direction, the movement of the circle will be from right to left, and it will be wise to check the pixels from right to left. Hence, we can first set a fixed value of y, that’s to say:
y?y_center ? radius; and (31) x ? x_center (32)
The x value can then be decreased one by one, until it reaches the boundary, which will be
x_boundary ? x_center ? radius (33)
Once the co-ordinate of x had reached its boundary value, the y coordinate can then be decreased by 1, and to check again for each and every x coordinate starting from x_center to x_boundary for that specific y. While each and every point are taken into considerations, we can check for its length from the radius. If it satisfied the condition of radius ± 0.5, then it will be taken into consideration, and the pixel value of that particular coordinate, will be copied into another matrix for future usage.
Whereas for the upper left part of the quarter of the circle, since the circle will going from left to right, it will then be wise to check for each and every x coordinate starting from x_boundary to x_center, with the decrement of y_coordinate for every completion of x_boundary to x_center checking.
The same methods can then be applied to the upper right, and lower right of the quarter of the circle, and each and every pixels’ values corresponding to that particular radius (together with the tolerance), can be copied into another new matrix.
Step 7: Extracting pixels from radius of 1 till ro. After finishing for the particular radius value, the radius should then be increase one-by-one, and to check until it reaches its boundary, which is the radius of the omni-directional image.
Step 8: Standardizing the length of panoramic images for every y-coordinate. Since the maximum number of pixels available in x- and y-coordinate of the panoramic images is not known, or in other words, the exact amount of pixels which will be considered in the largest circle that satisfied the radius with its tolerance of ± 0.5 is unknown, a variable is used to increment one by one when a point is considered for the largest circle, which will gives us the exact amount of pixels available, or the amount of pixels that had been considered, in the largest circle. This will then be the maximum length of our panoramic image in
x-coordinate, whereas for the height of the panoramic image, it is simply the radius of the omni-directional image.
However, since this is an omni-directional images converting into a panoramic images, the circle nearest to the center point will have least pixels extracted, and the circle furthest to the center point will have the most pixels considered. And this will make the panoramic image produced, or the matrix where the pixels information are copied to, in a triangular shape. This happens because the x-coordinate is set to increase 1 by 1 in the coordinate of the panoramic images, for a specific y-coordinate obtained from the current radius. Fig. 10 shows the allocation of pixels in the panoramic images.
So, in order to make it standardized, where each and every y-coordinate (or radius) will be having the same total length, the ratio of the number of pixels for that particular y-coordinate (or radius), corresponding to the maximum pixels contained in the image can be calculated, such as:(34)
This will give a brief understanding on how big, or how long should the spacing between pixels be assigned in that particular y-coordinate, so that the total length in that y-coordinate (or radius) can be equal to that of the longest length in the panoramic images, which is the number of the total pixels in the largest circle. This is done, in order to get a square matrix or a square images instead of a triangle shaped images, and the resultant image can have a look in Fig. 11, where the black color area denotes the spacing between pixels.
Step 9: Generating a uniform resolution panoramic image. Due to the increasing in spacing between pixels for smaller radius, the image might not be viewable to the eye of human, especially for the bottom part of the panoramic image. Therefore, in order to fill up these gaps or spacing between pixels, these pixels are actually duplicated over the spacing to make the image
information viewable. In other words, to solve this problem, we can try to generate a uniform resolution panoramic image, by continuously copying the value of the pixels into the spacing created in the image, before the next bit of pixel. Fig. 12 shows the end result of the above procedure.
Hence the image will then be able to be produced in a square form, instead of a triangle form.
Following up is the log-polar mapping method. 3.3 Log-Polar Mapping
In implementing the last method, which is the log-polar mapping method, there are several important steps to follow based on the proposed materials in the previous page. The following steps are as follow:
Step 1: Determining the periphery part of the omni-directional image. In this step, we first divide the omni-directional image into N number of samples, for each radius in M number of radius. In other words, the omni-directional image will be separated into M number of radius, with each radius having N number of samples separated by a radius of
?? (35)
where R is the radius of the points taken in the particular radius from the center point, N is the number of samples, and i is the ith radius currently taken into consideration. For the first radius to be taken, i = 1, and for the second radius to be taken, i = 2, and so on. Once the R is calculated, the radius of the omni-directional image taken will then be incremented by 1 to calculate for the radius at this sample circle. The radius taken will be increased by M times while calculating the radius, R, of the point in each and every particular radius. These values will then be used to obtain the xand y-coordinate in the omni-directional image, given by
??????????????cos? (36)??????????????sin? (37)
where θ is equals to? (38)
and j is the jth sample taken into consideration at that time. In other words, for a particular radius, theta will be increased from 1 up to 2π for N number of times. Therefore, we will obtain approximately N number of xper and yper values. These will be the x and y coordinates used in the omni-directional image.
The distance between these points will then be calculated by taking:
?????????? (39)?????????? (40)
where k is the kth sample taken in the particular radius, x1 is the distance between points in x-coordinate, and y1 is the distance between points in y-coordinate. Once the distance between the points had been obtained, these points can then be allocated a rectangular grid sampling with the width equals to x1 and the height equals to y1. Calculations will then be proceeded to obtain the mean intensity in each and every rectangular sampling by summing up all the color values in the rectangular sample pixels divided by the total amount of pixels available in the sample. The entire rectangular sample will then be replaced by this mean pixel values? (43)
N is the number of samples taken, same as in Step 1, and l is the lth sample taken. After obtaining the points for (xfov, yfov), the pixels at that point will be mapped into the matrix O. For panoramic image using fovea, the image and the matrix V is then divided into C number of radius and D number of samples in a particular radius for the inner part of the omni-directional image.
Step 3: Combining the periphery and fovea part. In this step, we will then combine the two matrices obtained from Step 1 and Step 2 into a new matrix. Since Step 1 is processing on the periphery part which is the outer part, and step 2 is processing on the fovea part which is the inner part, the combination of these two matrices will produce a complete version of panoramic image. Therefore the conversion is completed.
4. Results and Discussions.
In this section, the panoramic images will first be compared using the 3 universal un-warping methods, which are the pano-mapping table method, Discrete Geometry Techniques (DGT) method, and log-polar mapping method. There are several aspects taken into considerations that are used for comparison, which are
? Resolutions;
? Quality of image;
? Algorithm used;
? Complexity;
? Data compression;? Processing time.
It will then be discussed to determine which methods overwhelm the others. 4.1 Resolutions
The resolutions of each generated panoramic images after the conversion of log-polar method, discrete geometry techniques and pano-mapping method is such that, the log-polar method provides a smaller resolution of length equals to the radius and the width equals to 1/4 of the radius of the omni-directional image, whereas for the discrete geometry methods and pano-mapping method, the resolution of the panoramic-image produced can be as large as the length of the perimeter of the omni-directional image, with the width equals to the radius of the omni-directional image. Fig. 13 shows the original un-warped omni-directional image.
Fig. 14a shows the panoramic image generated by using log-polar mapping method, whereas Fig. 14b shows the panoramic image generated by using Discrete Geometry Techniques (DGT), and Fig. 14c by using pano-mapping table method.
4.2 Quality of Image
Based on the figure above, it is clear that pano-mapping method produces the highest quality of image, as compared to the other 2 methods, followed by the “Log-Polar Mapping” method which is slightly lower, and lastly, the DGT method.
4.3 Algorithm Used
The algorithm used in performing log-polar mapping method is that, the omni-directional image is considered in the form of a number of sectors which consists of a group of pixels, that will then be extracted sector by sector to be arranged into a rectangular form of image, whereas for the discrete geometry techniques, we are considering pixels by pixels to be extracted, to be arranged into a rectangular form image. These pixels
will then be re-produced, or duplicated, in order to standardize the number of pixels available in each row of the panoramic image. For the pano-mapping method, it is using an algorithm where a table will be created at initialization, to indicate the co-ordinates of the pixels to be extracted from the omni-directional image. Once the table is created, it will then be used over and over again to map each of the pixel at that particular co-ordinate, one by one, from the omni-directional image into a panoramic image, hence the name “once and forever”. 4.4 Complexity
The complexity of using log-polar mapping method, discrete geometry techniques, and pano-mapping method, as in the code used in generating or converting the omni-directional image to a panoramic image, are different as well. Table 2 shows the Big-O complexity
DGT, log-polar mapping and pano-mapping method, where X denotes the length of the panoramic image, which is also the perimeter of the omni-directional image taken into considerations, and Y denotes the height of the panoramic image, which is also the radius of the omni-directional image taken into considerations as shown in Fig. 15. 4.5 Data Compression
Out of all the 3 methods in concern, the generated panoramic image produces by log-polar method has the resolution, which is 473 × 114, DGT having a resolution of 1472 × 235, and 1146 × 243 for pano-mapping method, where the original omni-directional image is of resolution 473 × 473. From the resolutions, it is clear that log-polar mapping has the highest compression, which compresses the image to approximately 1/4 of its original size, compared to DGT and pano-mapping, which expanded the image.
4.6 Processing Time
The processing time for all the 3 methods to successfully convert an omni-directional image into a panoramic image is performed using matlab function“cputime”. The codes are processed 5 times, and the average processing time is computed. It is found that log-polar having the fastest computation time, which is 2.80176 seconds, followed by pano-mapping method having almost the same computation time, which is 2.84232 seconds, and lastly, 4.87344 seconds for the DGT method, which is the slowest out of the 3 methods. 4.7 Discussion between the Three Methods
In terms of resolution, although the image generated by discrete geometry techniques and pano-mapping method is larger as compared to the image generated by log-polar mapping method, these 2 methods seem to elongate the actual size of the image. In other words, this method tends to makes the objects in the image extended, and “fatter” than the original image. (Due to the image above being scaled to a smaller image, the difference is not really obvious). Due to this elongation, it will be harder to examine the picture and the objects, as the sense of the size had been eliminated. Whereas for the log-polar method, the extension is not as much and as obvious as DGT and pano-mapping methods.
In terms of quality of image, it is generally the most important factor that constitute to the preference of a method, and it is obvious that pano-mapping method produces the highest quality out of the 3 methods, followed by a slightly lower quality which is still within an acceptable range for the log-polar method, and the blurred DGT method.
In terms of algorithm used, pano-mapping method uses the simplest and easy algorithm, followed by a simple but slightly complex algorithm which is the log-polar method, and lastly, a complicated and complex algorithm from the DGT method.
In terms of complexity, by using the computations of the Big-O complexity, it is found that pano-mapping having the least complexity, followed by DGT, and lastly log-polar mapping methods.
In terms of data compression, log-polar mapping method is shown to have the best data compression rate out of the other 3 methods. This is very good in preserving memory in CPU as our memory is very limited.
In terms of processing time, on average, log polar mapping has the fastest processing time to convert 1 omni-directional image into a panoramic image, followed by pano-mapping method which is slightly slower, and DGT method, which is almost 2 times slower than log-polar and pano-mapping method.
In the final section, we will be going to a conclusion on which un-warping methods are the most efficient, taking into considerations of all the aspects compared previously.
5. Conclusions
A good quality of image is usually the most important factor, taking into consideration of other minor factors such as the processing speed and the complexity, in order to optimize the performance. After several considerations and research, log-polar method achieves the highest preference compared to the other 2 methods. This is because, while pano-mapping method produces a highest quality image out of the 3 methods, it also generates a large size of the panoramic image, which is very bulky and waste of memory space as the number of images generated increase from time to time. Besides that, even though log-polar method produces a slightly lower quality compared to pano-mapping method, it is still within an acceptable range and the image is still clear enough to be recognized, and that the data compression is most magnificent compared to the DGT and pano-mapping method.
By taking the highest complexity of Big-O in each of the methods, although log-polar has a highest complexity, which is X2Y2 compared to DGT of XY2, and pano-mapping with complexity of Y2, the time needed to process or to convert an omni-directional image into a panoramic image is sufficiently small, which is 2.80176 seconds on average, compared to the other 2 methods which are 4.87344 seconds for DGT and 2.84232 for pano-mapping, and is clearly the fastest out of the 3 methods. Hence, the complexity is not of much concern here.
As for the resolution of log-polar method which is smaller than DGT and pano-mapping converted panoramic image, it can basically be fixed by taking a larger resolution image to be processed, and the disadvantages of log-polar having small resolution can be solved. Furthermore, the algorithm used for log-polar methods is sufficiently simple and easy to understand as well.
Therefore, log-polar methods is most preferred to be used in omni-directional to panoramic image conversion, given that it has a good quality of image with the highest data compression rate, and a fastest processing speed out of the 3 methods.
References
[1] T. Kawanishi, K. Yamazawa, H. Iwasa, H. Takemura, N. Yokoya, Generation of high-resolution stereo panoramic images by omnidirectional imaging sensor using hexagonal pyramidal mirrors, in: Proc. 14th Int. Conf. in Pattern Recognition, Aug. 1998, Vol. 1, pp. 485-489.
[2] T. Padjla, H. Roth, Panoramic imaging with SVAVISCA camera: simulations and reality, Research Reports of CMP, Czech Technical University in Prague, No. 16, 2000.
[3] S.W. Jeng, W.H. Tsai, Using pano-mapping tables for unwarping of omni-images into panoramic and perspective-view images, IET Image Processing 1 (2)(2007) 149-155.
[4] A. Torii, A. Imiya, Panoramic image transform of omnidirectional images using discrete geometry techniques, in: Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT’04), 2004, pp. 608-615.
[5] G. Wolberg, S. Zokai, Robust image registration using log-polar transform, in: Proc. of IEEE Intl. Conf. on Image Processing, Sep. 2000, pp. 493-496.
[6] W.K. Wong, P.N. Tan, C.K. Loo, W.S. Lim, Omnidirectional surveillance system using thermal
camera, Journal of Computer Science and Engineering 3(2) (2010) 42-51.
[7] D.S. Huang, D.C. Wunsch, D.S. Levine, K-H Jo, Advanced intelligent computing theories and applications: with aspects of theoretical and methodological issues, in: 4th International Conference on Intelligent Computing(ICIC 2008), Shanghai, China, September 2008.
[8] C.J. Wu, W.H. Tsai, Unwarping of images taken by misaligned omnicameras without camera calibration by curved quadrilateral morphing using quadratic pattern classifiers, Opt. Eng. 48 (2009) 087003(1)-087003(11).
[9] V.J. Traver, A. Bernardino, A review of log-polar imaging for visual perception in robotics, Robotics and Autonomous Systems 58 (2010) 378-398.
[10] J.S.-Victor, A. Bernardino, Vision-based navigation, environmental representations and imaging geometries, in Proc. 10th Int. Symp. of Robotics Research, 2001, pp. 347-360.