STEREOSCOPIC IMAGE RECORDING METHOD AND STEREOSCOPIC IMAGE RECORDING APPARATUS

Abstract

In a stereoscopic image recording method (34), it is provided to transform depth information (13) with respect to an image pair (40) of a stereoscopic image (24) based on an alignment angle (23) in order to generate an aligned stereoscopic image (29), the transformed depth information (26) being used in order to generate a complementary image (27) of the stereoscopic image (40) in a computer-aided manner.

Claims

1. A stereoscopic image recording method (34), comprising: recording an image pair (40); calculating depth information (13 from the image pair (40); determining an alignment angle (23) which describes a position of the image pair (40) in relation to a preferred direction (30); transforming the depth information (13) based on the alignment angle (23); using the transformed depth information (26) for calculating at least one transformed image pair (28); and outputting the transformed image pair (28) as the stereoscopic image (29).

2. The stereoscopic image recording method (34) as claimed in claim 1, wherein, for calculating the transformed image pair (28), the method further comprises applying the transformed depth information (26) to a selected image (25) of the recorded image pair (40) and generating a complementary image (27), such that the selected image (25) and the complementary image (27) define the stereoscopic image (29).

3. The stereoscopic image recording method (34) as claimed claim 2, wherein, for generating the complementary image (27), the method further comprises displacing local image contents (31) of the selected image (25) by a distance (32) corresponding to the depth information (13).

4. The stereoscopic image recording method (34) as claimed in claim 1, further comprising selecting a search direction for calculating the depth information (13) along an image edge of at least one image (9, 10) of the image pair (40).

5. The stereoscopic image recording method (34) as claimed in claim 1, further comprising measuring the alignment angle (23) by an angle measuring device (22).

6. The stereoscopic image recording method (34) as claimed in claim 4, wherein the depth information (13) is calculated as a depth map, with ascertainment of content-related correspondences in the image pair (40) along a search direction.

7. The stereoscopic image recording method (34) as claimed in claim 6, wherein the search direction is predefined by an epipolar direction (11).

8. The stereoscopic image recording method (34) as claimed in claim 1, wherein the image pair (40) is recorded recurrently over time.

9. The stereoscopic image recording method (34) as claimed in claim 1, further comprising rotating a selected image of the recorded image pair (40) by the alignment angle (23).

10. The stereoscopic image recording method (34) as claimed in claim 1, further comprising rotating the depth information (13) by the alignment angle (23) in order to generate the transformed depth information (26).

11. The stereoscopic image recording method (34) as claimed in claim 1, wherein the image pair (40) is recorded by a stereoscopic endoscope (14).

12. A stereoscopic image processing apparatus (1), configured for carrying out the method as claimed in claim 1.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The invention will now be described in greater detail on the basis of an exemplary embodiment, but is not restricted to the exemplary embodiment. Further exemplary embodiments arise through combination of the features of individual or a plurality of claims among one another and/or with individual or a plurality of features of the exemplary embodiment.

[0024] In the figures:

[0025] FIG. 1: shows a stereoscopic image recording apparatus in a highly schematic illustration,

[0026] FIG. 2: shows a left image and a right image, recorded in the situation in accordance with FIG. 1,

[0027] FIG. 3: shows the construction of a stereoscopic image from a left image and a right image in a basic illustration,

[0028] FIG. 4: shows the recording of a stereoscopic image by means of a stereoscopic image processing apparatus in a first rotation position,

[0029] FIG. 5: shows a second stereoscopic image of the scene from FIG. 4 in the case of a rotated stereoscopic image processing apparatus,

[0030] FIG. 6: shows the stereoscopic image in accordance with FIG. 5 after an image rotation for the purpose of horizon placement with erroneously aligned disparity,

[0031] FIG. 7: shows the obtaining of depth information from an image pair of the stereoscopic image in accordance with FIG. 6,

[0032] FIG. 8: shows the rotation of the depth information from FIG. 7 by the alignment angle,

[0033] FIG. 9: shows a selected image of the image pair with respect to the stereoscopic image in accordance with FIG. 6,

[0034] FIG. 10: shows the calculation of a stereoscopic image from the selected image in accordance with FIG. 9,

[0035] FIG. 11: shows a flowchart of a stereoscopic image recording method, and

[0036] FIG. 12: shows a stereoscopic endoscope as stereoscopic image processing apparatus.

DETAILED DESCRIPTION

[0037] A stereoscopic image processing apparatus, designated in its entirety by 1, has a left image recorder 2 and a right image recorder 3 in a manner known per se, the viewing directions 4 of which image recorders are arranged parallel to one another and offset by a distance 5 relative to one another.

[0038] The stereoscopic image processing apparatus 1 is directed at a scene 6 in a manner known per se, said scene having two objects 7, 8 in the example.

[0039] In this case, the left image recorder 2 records a left image 9 and the right image recorder 3 records a right image 10 as an image pair 40.

[0040] In this case, the image positions of the objects 7, 8 in the left image 9 and the right image 10 are offset relative to one another along an epipolar direction or epipolar line 11.

[0041] If the imaging distance with respect to the objects 7, 8 is changed, then the apparent position of these objects 7, 8 in the left image 9 and in the right image 10 changes. If the imaging distance is changed such that the apparent position of one of the objects 7, 8 in one image remains constant, then the object in the other image apparently moves on the epipolar line 11. An offset, the disparity 12, thus arises as a function of the imaging distance.

[0042] FIG. 3 shows how depth information 13 can be obtained from the disparity 12 of the local image contents 31: the greater the disparity 12 of corresponding image contents, the further away the latter are from the image recorders 2, 3. This depth information 13 is typically present as a depth map.

[0043] More precisely, in this case, corresponding image contents 31 in the left image 9 and in the right image 10 are sought, for example along a search direction given by a horizontal image edge or in some other way, for example by an epipolar line 11. The disparity 12 of these image contents 31 leads, in a manner known per se, to local depth information which can be ascertained pixel by pixel and combined to form a depth map.

[0044] FIG. 4 shows an abstracted scene 6 that was recorded by a stereoscopic image processing apparatus 1. The direction of the disparity 12 is predefined by the relative arrangement of the image recorders 2, 3 with respect to one another.

[0045] FIG. 12 shows one example of an image processing apparatus 1. A stereoscopic endoscope 14 is illustrated.

[0046] The endoscope 14 has a shaft 15 in a manner known per se. A camera head 17 is embodied at a proximal end 16. A side view unit 19 is embodied at a distal end 18.

[0047] The side view unit 19 ensures that a viewing direction 20 of the image processing apparatus 1 is angled relative to the longitudinal axis 21.

[0048] In further exemplary embodiments, the image recorders 2, 3 are arranged directly at the distal end 18.

[0049] If the image processing apparatus 1 from FIG. 12 is then rotated about the longitudinal axis 21, then the viewing direction 20 is spatially pivoted.

[0050] This leads to an apparent rotation and translation of the scene 6 in the left image 9 and the right image 10 of the stereoscopic image 24, as illustrated in FIG. 5. In this case, the dash-dotted line indicates the alignment of the preceding images 9, 10.

[0051] The apparent rotation of the scene 6 is unpleasant for a user. Therefore, for monoendoscopes, it has become customary to reposition the horizon.

[0052] An angle measuring device 22 (cf. FIG. 12) is designed for this purpose, and detects an alignment angle 23 by which the image processing apparatus 1 was rotated. In this case, the alignment angle 23 describes a position or orientation of the image recording apparatus 1 in relation to a preferred direction 30.

[0053] In further exemplary embodiments, the alignment angle 23 is calculated by a comparison of contents of two successive (left or right) images 9, 10.

[0054] For horizon placement, the images 9, 10 of the image pair 40 would then have to be individually rotated by the alignment angle 23. In this case, the alignment angle 23 can be related to a preferred direction 30, for example to a horizontal direction (lower image edge before the rotation, cf. FIG. 4).

[0055] This results in the situation in FIG. 6. The dash-dotted line indicates an apparent position of the images 9, 10 from this processing step.

[0056] FIGS. 4 and 5 need not necessarily succeed one another directly. It is also possible that, between FIGS. 4 and 5, further image pairs were recorded at different alignment angles 23, which were transformed in the manner described.

[0057] It is evident that the disparity 12 is no longer pointing horizontally as a result of the rotation of the images 9, 10. It is thus no longer possible to perceive the stereoscopic image 24 as a spatial image.

[0058] Therefore, depth information 13 is calculated from the images 9, 10 in the manner described. This is shown in FIG. 7.

[0059] One of the images 9, 10, for example the left image 9, is then processed further as a selected image 25 and is rotated by the alignment angle 23.

[0060] In further exemplary embodiments, the right image 10 or a middle image is used as the selected image 25.

[0061] Furthermore, the depth information 13 from FIG. 7 is rotated by the alignment angle 23, FIG. 8. The dash-dotted line shows the boundary of the depth information 13 before the rotation. Transformed depth information 26 arises.

[0062] The transformed depth information 26 is then applied to the rotated selected image 25 (FIG. 9) in order to generate a complementary image 27. In this case, the individual local image contents 31, for example image pixels 33, are horizontally displaced by a distance 32 corresponding to the respective transformed (local) depth information 26.

[0063] The complementary image 27—as right image in the example—together with the rotated selected image 25 forms a transformed image pair 28, FIG. 10. The transformed image pair 28 defines a stereoscopic image 29, the depth information of which corresponds to the transformed depth information 26. This stereoscopic image 29 has a horizontally aligned disparity 12 again and can thus be perceived spatially again.

[0064] A stereoscopic image recording method 34 depicted in FIG. 11 thus proceeds recurrently in the image processing apparatus 1.

[0065] The recording of an image pair 40 in accordance with FIG. 5 as a starting point or input variable of the image recording method 34 is not illustrated in further detail.

[0066] In an angle determining step 35, firstly the alignment angle 23 is determined.

[0067] In a depth information calculating step 36, the depth information 13 is then calculated, for example as a disparity map.

[0068] In a rotation step 37, the selected image 25 and the depth information 13 are rotated by the alignment angle 23. Transformed depth information 26 is thus present.

[0069] In an image calculating step 38, a complementary image 27 is calculated with respect to the rotated selected image 25, for example on the basis of the formula R(x,y)=L(x+dx,y+dy), where R denotes the complementary image 27, L denotes the rotated selected image 25, and (dx,dy) denotes the local disparity 12 in accordance with the transformed depth information 26.

[0070] In an output step 39, the rotated selected image 25 and the complementary image 27 are output as a transformed image pair 28 in the form of a stereoscopic image 29.

[0071] These method steps are carried out again with close repetition over time.

[0072] In a stereoscopic image recording method 34, it is thus proposed to transform depth information 13 with respect to an image pair 40 of a stereoscopic image 24 on the basis of an alignment angle 23 in order to generate an aligned stereoscopic image 29, the transformed depth information 26 being used in order to generate a complementary image 27 of the stereoscopic image 29 in a computer-aided manner.

LIST OF REFERENCE SIGNS

[0073] 1 Stereoscopic image processing apparatus [0074] 2 Left image recorder [0075] 3 Right image recorder [0076] 4 Viewing direction [0077] 5 Distance [0078] 6 Scene [0079] 7 Object [0080] 8 Object [0081] 9 Left image [0082] 10 Right image [0083] 11 Epipolar direction [0084] 12 Disparity [0085] 13 Depth information [0086] 14 Endoscope [0087] 15 Shaft [0088] 16 Proximal end [0089] 17 Camera head [0090] 18 Distal end [0091] 19 Side view unit [0092] 20 Viewing direction [0093] 21 Longitudinal axis [0094] 22 Angle measuring device [0095] 23 Alignment angle [0096] 24 Stereoscopic image [0097] 25 Selected image [0098] 26 Transformed depth information [0099] 27 Complementary image [0100] 28 (Transformed) image pair [0101] 29 Stereoscopic image [0102] 30 Preferred direction [0103] 31 Local image content [0104] 32 Corresponding distance [0105] 33 Image pixels [0106] 34 (Stereoscopic) image recording method [0107] 35 Angle determining step [0108] 36 Depth information calculating step [0109] 37 Rotation step [0110] 38 Image calculating step [0111] 39 Output step [0112] 40 Image pair