METHOD AND SYSTEM FOR AUTOMATICALLY OPTIMIZING 3D STEREOSCOPIC PERCEPTION, AND MEDIUM

Abstract

Provided by the present invention are a method and system for automatically optimizing 3D stereoscopic perception, and a medium. The method comprises the following steps that are executed successively: step 1: given current left and right images, calculating a stereo disparity to generate a disparity map; step 2: calculating a depth value corresponding to each individual pixel by using the calculated disparity; step 3: calculating a depth distance of a target to be observed; step 4: acquiring corresponding left and right image displacement values by using the depth distance calculated in step 3; and step 5: applying the acquired image displacement values into a 3D display. The beneficial effect of the present invention is that: the method for automatically optimizing 3D stereoscopic perception according to the present invention solves fatigue and dizziness that are easily generated during the use of a 3D endoscope.

Claims

1. A method for automatically optimizing 3D stereoscopic perception for a 3D electronic endoscope system, comprising the following steps that are executed successively: step 1: calculating a stereo disparity of given current left and right images to generate a disparity map; step 2: calculating a depth value corresponding to each individual pixel by using the calculated stereo disparity; step 3: calculating a depth distance of a target to be observed; step 4: acquiring corresponding left and right image displacement values by using the depth distance calculated in step 3; and step 5: moving the left and right images according to the acquired image displacement values and performing them in a 3D display; wherein the steps 1-5 are steps that perform an optimization process once, the optimization process being triggered actively by a user or being triggered automatically and continuously according to a predetermined interval.

2. The method according to claim 1, wherein in the step 1, the disparity map is acquired by a plurality of algorithms including SGBM algorithm and BM algorithm.

3. The method according to claim 1, wherein in the step 1, the calculation of the stereo disparity comprises calculating the stereo disparity for an entire image or only for a selected region of interest (ROI).

4. The method according to claim 1, wherein in the step 2, a calculation formula of the depth value is: $Z\left(x,y\right)=\frac{f\times T_x}{d\left(x,y\right)}$ where f is the focal length of a camera, T.sub.x is the center distance between left and right cameras, and (x,y) is a current pixel position.

5. The method according to claim 1, wherein in the step 3, the average value, median value or maximum value of the depth values corresponding to all pixels in the entire image or a region of interest (ROI) is calculated as the depth distance of the target to be observed.

6. The method according to claim 1, further comprising: a step of generating a lookup table in advance: placing a target to be observed at different positions between 10 mm and 100 mm in front of a camera at an interval of 10 mm respectively, adjusting the left and right image displacement at each position until obtaining a 3D reconstruction effect desired by an observer, and recording information about each position and corresponding image displacement values to generate a lookup table; and the step 4 comprises: when a current depth distance is between 10 mm and 100 mm, obtaining an image displacement value corresponding to the current depth distance by using linear interpolation according to the lookup table; when the current depth distance is less than 10 mm, adopting an image displacement value corresponding to the depth distance of 10 mm; and when the current depth distance is greater than 100 mm, adopting an image displacement value corresponding to the depth distance of 100 mm.

7. The method according to claim 6, wherein when performing the step of generating a lookup table in advance, an environment that is identical or similar to an actual application scenario is used, including: adopting a display of the same size, and the same distance from the observer to a screen.

8. (canceled)

9. A system for automatically optimizing 3D stereoscopic perception, comprising: a disparity map acquisition unit configured to calculate a stereo disparity of given current left and right images to generate a disparity map; a depth value calculation unit configured to calculate a depth value corresponding to each individual pixel by using the calculated stereo disparity; a depth distance calculation unit configured to calculate a depth distance of a target to be observed; a left and right image displacement values acquisition unit configured to acquire corresponding left and right image displacement values by using the calculated depth distance; and a display unit configured to move the left and right images according to the acquired image displacement values and perform them in a 3D display.

10. The system according to claim 9, wherein in the depth distance calculation unit, the average value, median value or maximum value of the depth values corresponding to all pixels in the entire image or a region of interest (ROI) is calculated as the depth distance of the target to be observed; and the left and right image displacement values acquisition unit comprises: a pre-generated lookup table unit configured to place a target to be observed at different positions between 10 mm and 100 mm in front of a camera at an interval of 10 mm respectively, adjust the left and right image displacement at each position until obtaining a 3D reconstruction effect desired by an observer, and record information about each position and corresponding image displacement values to generate a lookup table; and an image displacement value acquisition unit configured to obtain an image displacement value corresponding to the current depth distance by using linear interpolation according to the lookup table when a current depth distance is between 10 mm and 100 mm; adopt an image displacement value corresponding to the depth distance of 10 mm when the current depth distance is less than 10 mm; and adopt an image displacement value corresponding to the depth distance of 100 mm when the current depth distance is greater than 100 mm.

11. A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program configured to implement the steps of the method according to claim 1 when called by a processor.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0038] FIG. 1 is a typical 3D electronic endoscope system in the prior art;

[0039] FIG. 2 is the comfort zone at different focal distances found in the prior art;

[0040] FIG. 3 is a flowchart of a method for automatically optimizing 3D stereoscopic perception according to the present disclosure; and

[0041] FIG. 4 is a schematic diagram of the principle of disparity map according to the present disclosure.

DETAILED DESCRIPTION

[0042] As shown in FIG. 3, a method for automatically optimizing 3D stereoscopic perception disclosed according to the present disclosure may comprises the following steps that are executed successively:

[0043] step 1: calculating a stereo disparity of given current left and right images to generate a disparity map;

[0044] step 2: calculating a depth value corresponding to each individual pixel by using the calculated stereo disparity;

[0045] step 3: calculating a depth distance of a target to be observed;

[0046] step 4: acquiring corresponding left and right image displacement values by using the depth distance calculated in step 3; and

[0047] step 5: moving the left and right images according to the acquired image displacement values and performing them in a 3D display.

[0048] In the step 1, the disparity map may be acquired by a plurality of algorithms which may include SGBM algorithm and BM algorithm, see [1] for details.

[0049] SGBM is the abbreviation of Semiglobal Block Matching; and

[0050] BM is the abbreviation of Block Matching.

[0051] Introduction to the principle of disparity map:

[0052] As shown in FIG. 4, given that there is a point X in a three-dimensional space, the coordinate thereof on the left view is x, and x′ on the right view, and a stereo disparity is x-x′.

[0053] In the step 1, the calculation of the stereo disparity may comprise calculating the stereo disparity for an entire image or only for a selected region of interest (ROI).

[0054] In the step 2, a calculation formula of the depth value may be:

[00003]$Z\left(x,y\right)=\frac{f\times T_x}{d\left(x,y\right)}$

[0055] where f is the focal length of a camera, T.sub.x is the center distance between left and right cameras, and (x,y) is a current pixel position.

[0056] In the step 3, the average value, median value or maximum value of the depth values corresponding to all pixels in the entire image or a region of interest (ROI) is calculated as the depth distance of the target to be observed.

[0057] The step 4 may be implemented by:

[0058] a step of generating a lookup table in advance: placing a target to be observed at different positions between 10 mm and 100 mm in front of a camera at an interval of 10 mm respectively, adjusting the left and right image displacement at each position until obtaining a 3D reconstruction effect desired by an observer, and recording information about each position and corresponding image displacement values to generate a lookup table; and

[0059] a step of obtaining an image displacement value: when a current depth distance is between 10 mm and 100 mm, obtaining an image displacement value corresponding to the current depth distance by using linear interpolation according to the lookup table; when the current depth distance is less than 10 mm, adopting an image displacement value corresponding to the depth distance of 10 mm; and when the current depth distance is greater than 100 mm, adopting an image displacement value corresponding to the depth distance of 100 mm.

[0060] When performing the step 4, an environment that is identical or similar to an actual application scenario may be used, including: adopting a display of the same size, and the same distance from the observer to a screen.

[0061] The steps 1-5 may be steps that perform an optimization process once and may be performed by using a user-triggered mode or an automatically and continuously triggered mode. The user-triggered mode refers to a user using a handle button, a touch screen button, a foot pedal, a voice control, or other ways to trigger the optimization process to be run once. The automatically and continuously triggered mode refers to optimizing automatic triggering at certain intervals without user intervention.

[0062] [1] Heiko Hirschmuller. Stereo processing by semiglobal matching and mutual information. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 30(2):328-341, 2008.

[0063] A system for automatically optimizing 3D stereoscopic perception further disclosed according to the present disclosure may comprise:

[0064] a disparity map acquisition unit configured to calculate a stereo disparity of given current left and right images to generate a disparity map;

[0065] a depth value calculation unit configured to calculate a depth value corresponding to each individual pixel by using the calculated stereo disparity;

[0066] a depth distance calculation unit configured to calculate a depth distance of a target to be observed;

[0067] a left and right image displacement values acquisition unit configured to acquire corresponding left and right image displacement values by using the calculated depth distance; and

[0068] a display unit configured to move the left and right images according to the acquired image displacement values and perform them in a 3D display.

[0069] In the depth distance calculation unit, the average value, median value or maximum value of the depth values corresponding to all pixels in the entire image or a region of interest (ROI) may be calculated as the depth distance of the target to be observed.

[0070] The left and right image displacement values acquisition unit may comprise:

[0071] a pre-generated lookup table unit configured to place a target to be observed at different positions between 10 mm and 100 mm in front of a camera at an interval of 10 mm respectively, adjust the left and right image displacement at each position until obtaining a 3D reconstruction effect desired by an observer, and record information about each position and corresponding image displacement values to generate a lookup table; and

[0072] an image displacement value acquisition unit configured to obtain an image displacement value corresponding to the current depth distance by using linear interpolation according to the lookup table when a current depth distance is between 10 mm and 100 mm; adopt an image displacement value corresponding to the depth distance of 10 mm when the current depth distance is less than 10 mm; and adopt an image displacement value corresponding to the depth distance of 100 mm when the current depth distance is greater than 100 mm.

[0073] A computer-readable storage medium may further be disclosed according to the present disclosure. The computer-readable storage medium may store a computer program configured to implement the steps of the method mentioned above when being called by a processor.

[0074] The beneficial effect of the present invention is that: the method for automatically optimizing 3D stereoscopic perception according to the present invention solves fatigue and dizziness that are easily generated during the use of a 3D endoscope. By means of automatic optimization, symptoms can be alleviated, guaranteeing user comfort for a long time.

[0075] The above is a further detailed description of the present disclosure in combination with specific preferred embodiments, and it cannot be considered that the specific implementation of the present disclosure is limited to these descriptions. For those of ordinary skill in the technical field of the present disclosure, without departing from the concept of the present disclosure, several simple deductions or substitutions can be made, which should be deemed to belong to the protection scope of the present disclosure.