Method for spherical camera image stitching

Abstract

The present invention provides a method for Spherical Camera Image Stitching. By using two fisheye lens to catch two fisheye images and then being developed into three pairs of flat figures based on Segmented Sphere Projection (SSP) method. Thereafter each corresponding pair is stitched based on a similar-edge method, and then three pairs are combined to form a panoramic image. At the end, the combined panoramic image is projected to a 3-D ball sphere space.

Claims

1. A method for spherical camera image stitching, comprising steps as below: (a) using a fisheye camera with two fisheye lens back to back for catching two hemispherical fisheye images with 190 respectively; (b) said two hemispherical fisheye images are each divided into three parts: an arctic part, an equator part and an antartic part, with boundary lines at north latitude +45 and south latitude 45; (c) said arctic part and said antartic part of said two hemispherical fisheye images are transformed into semicircular shapes respectively; (d) said two semicircular shapes of said arctic part and said antartic part are stitched into a circular shape respectively by a similar-edge method (e) a feather blending method is used for overlapping areas at stitching, and then said two circular shapes are transformed into spherical shapes to form a spherical arctic part and a spherical antartic part respectively; (f) said equator parts of said two hemispherical fisheye images are stitched by ERP method based on said similar-edge method said feather blending method, and then combine the arctic part, the equator part and the antartic part to form a panoramic image; (g) at the end, the combined panoramic image is projected to a 3-D ball sphere space to form a spherical panoramic image.

2. The method for spherical camera image stitching according to claim 1, wherein said similar-edge method is to let pixels of said fisheye images from RGB images to form grayscale images, and then let pixels which has significant brightness change to be set as edge 1, others are set as non-edge 0; two regions for being stitched will each take a range L (master match) and a range W (being matched) subject to the center thereof respectively; move the range L on the range W to cover ranges L of same width as L, and calculate a similarity between the range L and the ranges L, find the most similar range L, and then stitch the range L with the most similar range L; the equation below shows the calculation for similarity, similarity closer to 1 is more similar, similarity equal to 1 is exactly the same, a and b are individual pixel edge values (1 or 0) in the range L and the ranges L respectively, h is the height of L and L, w is the width of L and L, i,j represent vertical coordinate value and horizontal coordinate value of the pixel respectively, $\mathrm{Similarity}=1-\frac{\underset{i=1}{\overset{h}{.Math.}}\underset{j=1}{\overset{w}{.Math.}}.Math.a_{i,j}-b_{i,j}.Math.}{hw}100\%.$

3. The method for spherical camera image stitching according to claim 2, wherein said feather blending method is to establish a weight diagram for the range L and the most similar range L, the range L is from left to overlap with the range L from right, the weight is 1 for the left side of range L and the weight is 0 for the right side of range L, and the weight is 0 for the right side of range L and the weight is 1 for the left side of range L, the weight is 0.5 at the center; in this way, the overlapping of the range L and the the most similar range L will be added up to 1, so the overlapping of the range L and the the most similar range L is the most like the original images.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 shows schematically a diagram for describing the prior art of Equirectangular Projection (ERP) method.

(2) FIGS. 2, 3 and 4 show schematically a Segmented Sphere Projection (SSP) method according to the present invention.

(3) FIG. 5 shows schematically a similar-edge method according to the present invention.

(4) FIG. 6 shows schematically a feather blending method according to the present invention.

DETAILED DESCRIPTIONS OF THE PREFERRED EMBODIMENTS

(5) FIGS. 2, 3 and 4 show schematically a Segmented Sphere Projection (SSP) method according to the present invention. The steps in FIG. 2 are the same as the steps 13 in FIG. 1. The steps 4, 51 and 52 in FIG. 3 and the step 6 in FIG. 4 describe the SSP method according to the present invention.

(6) Referring to FIG. 2, A fisheye camera 1 uses two fisheye lens 11, 12 back to back for catching images (step 1). Two fisheye pictures A and B are catched (step 2). The two fisheye pictures A and B represent two hemispherical images respectively for having about 190, so each has a loop 13, 14 of about 5 at below respectively (step 3).

(7) The steps 4, 51 and 52 in FIG. 3 describe the Segmented Sphere Projection (SSP) method according to the present invention. The two hemispherical fisheye images of step 3 in FIG. 2 are each divided into three parts: arctic part, equator part and antartic part, with the boundary lines at north latitude +45 and south latitude 45.

(8) Referring to steps 4 in FIG. 3, the equator parts of the two hemispherical fisheye images A and B are spreaded into rectangular shape by ERP method to form the two flat figures as shown in the center of FIG. 3

(9) The arctic parts and the antartic parts of the two fisheye images A and B are transformed into semicircular shape respectively, as shown by step 51.

(10) Then the two arctic parts are stitched based on a similar-edge method and a feather blending method, i.e. the belt 31 and the belt 32 are stitched to form a circle (step 52).

(11) The two antartic parts are stitched based on a similar-edge method and a feather blending method, i.e. the belt 33 and the belt 34 are stitched to form a circle (step 52).

(12) Referring to FIG. 4, the equator parts of A-left, A-right and B-left, B-right in FIG. 3 are stitched as shown in FIG. 4 as A-right, B-left and B-right, A-left by similar-edge method and feather blending method, and then the three parts (arctic part, antartic part and equator part) are combined to form a panoramic image (step 6).

(13) At the end, the combined panoramic image is projected to a 3-D ball sphere space.

(14) The similar-edge method used by arctic part, antartic part and equator part for stitching is described as follows: Two regions for being stitched is performed with an edge detecting method. The edge detecting method will change the pixels of the images of the two regions into 1 or 0, 1 means edge, 0 means non-edge. A pixel is composed of red light brightness, green light brightness and blue light brightness (i.e, RGB images), with values between 0 to 255 to form various colors. While a grayscale image is simple from darkest black to multilevel gray to brightest white. RGB images can be average weighted to form grayscale images, and then to perform the edge detecting. Edge means where the brightness has significant change. The pixel which has significant brightness change is set as edge 1, others are set as non-edge 0. In this way, the data volume will be decreased significantly and still maintain the structural attribute of the images.

(15) Referring to FIG. 5, two regions for being stitched (for example 31 and 32, or 33 and 34, or 132 and 141, or 131 and 142) are shown at left side of FIG. 5, each of the two regions will each take a range L (master match) and a range W (being matched) subject to the center respectively as shown. Move the range L on the range W to cover ranges L of same width as L (shown at center of FIG. 5), and calculate the similarity between the range L and the ranges L, find the most similar range L, and then stitch the range L with the most similar range L. The equation below shows the calculation for similarity, similarity closer to 1 is more similar, similarity equal to 1 is exactly the same, a and b are individual pixel edge values (1 or 0) in the range L and the ranges L respectively (shown at right side of FIG. 5), h is the height of L and L, w is the width of L and L, i,j represent vertical coordinate value and horizontal coordinate value of the pixel respectively.

(16) $\mathrm{Similarity}=1-\frac{\underset{i=1}{\overset{h}{.Math.}}\underset{j=1}{\overset{w}{.Math.}}.Math.a_{i,j}-b_{i,j}.Math.}{hw}100\%$

(17) Referring to FIG. 6, the overlapping of the range L and the the most similar range L is described. A weight diagram is established as shown. The range L is from left to overlap with the range L from right, so the weight is 1 for the left side of range L and the weight is 0 for the right side of range L, and the weight is 0 for the right side of range L and the weight is 1 for the left side of range L, the weight is 0.5 at the center. In this way, the overlapping of the range L and the the most similar range L will be added up to 1, so the overlapping of the range L and the the most similar range L is the most like the original images. This is so-called the feather blending method.

(18) The scope of the present invention depends upon the following claims, and is not limited by the above embodiments.