Reference plate and method for calibrating and/or checking a deflectometry sensor system

Abstract

The disclosure relates to a reference plate for calibrating and/or checking a deflectometry sensor system, said deflectometry sensor system including an image generation device and a capturing device having at least one capturing element, wherein the reference plate includes a reflective surface, and wherein, for the purpose of checking at least one system parameter of said deflectometry sensor system, the reflective surface is provided with a predefined pattern including markings. A corresponding method for calibrating and/or checking a deflectometry sensor system is moreover indicated.

Claims

1. A reference plate configured to provide a calibration and/or check of a deflectometry sensor system, the deflectometry sensor system comprising an image generation device and a capturing device including a plurality of cameras to provide an enlarged field of view, the reference plate corresponding to a size of the field of view and comprising: a reflective surface having a predefined pattern that includes markings in order to check at least one system parameter of the deflectometry sensor system, wherein the predefined pattern that includes markings is configured to provide information that calibrates and/or checks the deflectometry sensor system and includes a plurality of marking groups, such that for each camera of the capturing device, a marking group is provided for recording an image of the respective marking group.

2. The reference plate according to claim 1, wherein the markings of the predefined pattern are arranged, at least in a top view, in a form of a regular grid on the reflective surface.

3. The reference plate according to claim 1, wherein the markings are in a non-planar arrangement on the reflective surface.

4. The reference plate according to claim 1, wherein the markings include predefined contour edges and/or a predefined test chart.

5. The reference plate according to claim 1, wherein the markings include coded marks.

6. The reference plate according to claim 1, wherein the markings include Siemens stars.

7. The reference plate according to claim 6, wherein the Siemens stars are arranged in the predefined pattern or in the marking groups such that in a central area of an image recorded by a camera of the capturing device, a Siemens star is present.

8. The reference plate according to claim 6, wherein the Siemens stars are arranged in the predefined pattern or in the marking groups such that in each corner area of an image recorded by a camera of the capturing device, a Siemens star is present.

9. A method of calibrating and/or checking a deflectometry sensor system, the deflectometry sensor system including an image generation device and a capturing device including a plurality of cameras for producing an enlarged field of view, the method comprising the steps of: providing a reference plate having a reflective surface including a predefined pattern that includes markings, the reference plate corresponding to a size of the field of view, wherein the predefined pattern includes a plurality of marking groups such that, for each camera of the capturing device, a marking group is provided for recording an image of the marking group; recording, by the cameras, images of the reference plate; and determining at least one system parameter of the deflectometry sensor system based on the recorded images.

10. The method according to claim 9, wherein the predefined pattern and/or a predefined pattern sequence is generated on the image generation device, overall brightness.

11. The method according to claim 9, wherein the reference plate is configured such that image points included by the image generation device are reproduced via the reference plate on the capturing device.

12. The method according to claim 9, wherein the reference plate is positioned at a predefined distance from the capturing device.

13. The method according to claim 9, wherein the image generation device is configured to be operated with homogeneous brightness to generate a white image.

14. The method according to claim 9, wherein the image or images recorded with one or more cameras includes a marking group having markings, and the at least one system parameter of the deflectometry sensor system is determined, based on the marking group in the recorded one or more images.

15. The method according to claim 9, wherein the at least one system parameter of the deflectometry sensor system is one of the group comprising: a sharpness parameter; a viewing range parameter; a brightness parameter; a position parameter; and a robot correction parameter.

16. The method according to claim 15, wherein determining the sharpness parameter of a camera of the capturing device further comprises: determining a sharpness value of Siemens stars within a marking group of the predefined pattern.

17. The method according to claim 15, wherein determination of the viewing range parameter of a camera of the capturing device further comprises: determining positions of Siemens stars within a marking group of the predetermined pattern.

18. The method according to claim 15, wherein determination of the brightness parameter of a camera of the capturing device further comprises: performing a grayscale value determination on a settable, central image area of the image captured by the camera, wherein the image area is selected such that no markings of the predefined pattern are present in the image area.

19. The method according to claim 15, wherein determining the brightness parameter of the deflectometry sensor system further comprises: determining a new brightness setting of the image generation device for reaching a target grayscale value of the capturing device based on a grayscale value of the capturing device recorded with a predefined brightness setting of the image generation device.

20. The method according to claim 15, wherein determination of the position parameter of a camera further comprises: detecting positions of coded marks within a marking group of the predefined pattern.

21. The method according to claim 20, wherein determination of the robot correction parameter, including determination of a 3D transformation as the robot correction parameter further comprises: computing, using a minimization algorithm, distances between target position and a current position of the coded marks.

Description

(1) The figures in the drawing show

(2) FIG. 1 in a diagrammatic view, an example of a configuration of a deflectometry sensor system for the use of an embodiment example of the reference plate according to the disclosure or the method according to an embodiment of the disclosure,

(3) FIG. 2 in a diagrammatic top view, an embodiment example of a reference plate according to an embodiment of the disclosure,

(4) FIG. 3 in a diagrammatic top view, the embodiment example according to FIG. 2, wherein multiple marking groups are represented on the exemplary reference plate,

(5) FIG. 4 in a diagrammatic view, four images of the reference plate according to FIG. 2 and according to FIG. 3, each image having been recorded with a camera of a deflectometry sensor system including four cameras,

(6) FIG. 5 an exemplary Siemens star for an embodiment example of a reference plate according to the disclosure and for the sharpness check according to an embodiment example of the method according to an embodiment of the disclosure,

(7) FIG. 6 in a diagrammatic view, an illustration of a viewing range check according to an embodiment example of the method according an embodiment of to the disclosure,

(8) FIG. 7 in a diagrammatic view, an illustration of a camera position check according to an embodiment example of the method according an embodiment of to the disclosure,

(9) FIG. 8 a diagram for illustrating the aging of a screen with regard to the relative light emission,

(10) FIG. 9 a diagram for illustrating the relationship between camera grayscale value and screen brightness of a deflectometry sensor,

(11) FIG. 10 a diagram for illustrating the relationship between screen brightness and backlight setting of the screen of a deflectometry sensor,

(12) FIG. 11 a diagram for illustrating the relationship between camera grayscale value and backlight setting of the screen of a deflectometry sensor, and

(13) FIG. 12 a diagram for illustrating the relationship between screen brightness and backlight setting in the interval [80; 100] of a deflectometry sensor.

(14) FIG. 1 shows, in a diagrammatic view, an exemplary configuration of a deflectometry sensor system for the use of an embodiment example of the reference plate according to the disclosure or of the method according to the disclosure. The deflectometry sensor system according to FIG. 1 includes a deflectometry sensor 1. The deflectometry sensor 1 includes a screen 2 as image generation device for generating an image pattern. Moreover, the deflectometry sensor 1 includes a camera 3 as capturing element of a capturing device. The camera 3 records the mirror image of the image pattern reflected by the measurement object 4. The measurement object 4 has a shiny or reflective surface.

(15) FIG. 2 shows, in a diagrammatic top view, an embodiment example of a reference plate according to the disclosure. The reference plate 5 represented in FIG. 2 has a predefined pattern including markings—on its surface. The predefined pattern consists of coded marks 6 arranged in the form of a regular grid. Instead of multiple coded marks, Siemens stars 7 are arranged as markings. The reference plate 5 shown in a top view is designed for the calibration of deflectometry sensor with four cameras.

(16) FIG. 3 shows, in a diagrammatic top view, the embodiment example according to FIG. 2, wherein multiple marking groups are represented on the exemplary reference plate 5. FIG. 3 shows the grouping of the Siemens stars 7 in a marking group for each camera. In each case, four Siemens stars 7 are arranged so that they are located in the field of view of a camera. The first marking group 8 is for the first camera, the second marking group 9 for the second camera, the third marking group 10 for the third camera, and the fourth marking group 11 for the fourth camera. The Siemens stars 7 are here arranged so that a perspective distortion due to the image recording is taken into consideration.

(17) FIG. 4 shows, in a diagrammatic view, four images of the reference plate 5 according to FIG. 2 and according to FIG. 3, which were recorded each with a camera of a deflectometry sensor system including four cameras. Concretely, FIG. 4 shows the images 8′, 9′, 10′ and 11′ of the respective camera. The image 8′ reproduces the marking group 8 according to FIG. 3. The image 9′ reproduces the marking group 9 according to FIG. 3. The image 10′ represents the marking group 10 according to FIG. 3. The image 11′ represents the marking group 11 according to FIG. 3. By the perspective arrangement of the Siemens stars 7 on the reference plate, the stars appear in each case at the corners in the camera image, so that the entire camera image of the cameras can be used for the calibration.

(18) According to an embodiment example of the disclosure, a check of the functionality of a deflectometry sensor system is carried out by means of a reference plate as reference target, especially developed for this purpose. Here, the deflectometry sensor—including a screen as image generation device and a capturing device with one or more cameras as capturing elements—is positioned in front of the reference plate at a defined distance, and, for each camera, an image is recorded, which is then evaluated and rated. The arrangement corresponds to the deflectometry condition, i.e., the reference plate is positioned so that the pixels/image points of the screen are represented in a mirroring arrangement—angle of incidence equal to angle of reflection—on the camera.

(19) The reference plate includes a surface mirror which is provided with a predefined pattern having markings at different positions. The predefined pattern is used for checking different system parameter values. The predefined pattern on the reference plate can consist, for example, of a combination of coded marks and Siemens stars.

(20) According to an embodiment example, the coded marks are used for determining the 3D position of the cameras (3×rotation, 3×translation). The Siemens stars are used for determining the sharpness, the contrast and the viewing ranges, wherein the Siemens stars are arranged so that for each camera a star is present in the center and in the four corners of the image recorded by a camera.

(21) The brightness can be determined via the unprinted area of the mirror, in that the image generated by the screen is viewed in the reflective surface of the reference plate.

(22) For the calibration of the cameras of the capturing device of the deflectometry sensor, first the screen is operated with homogeneous brightness, for example, white. The predefined pattern on the reference plate is therefore represented on a white background and can be simply evaluated for calibrating the camera.

(23) In the context of an embodiment example of the disclosure,—in addition to the use of markings on the reference plate—,the use of markings (patterns) or pattern sequences on the image generation device can be used for monitoring purposes and/or for calibrating the position of the image generation device is implemented.

(24) As an example, based on deflectometric measurements with different bandwidths (beat frequency recordings), a correspondence between camera pixels and image pixels can be established. I.e., for each image point of the camera, the associated position on the screen is known. These pixel positions, converted into a metric unit, can be used in addition to markings on the reference plate for the camera calibration, and thus intrinsic and/or extrinsic camera parameters with regard to the virtual screen position (rectilinear beam path) can be determined. With the help of the markings on the reference plate, the real position of the reference plate can be determined, and thus in the end also the real position of the screen.

(25) If, in addition, the geometric properties of the reference plate are known, the deflectometric measurement result of the reference plate can even be used for the purpose of a reference measurement for monitoring and calibrating the deflectometric unit. If the reference plate is, for example, a (nearly) perfect planar mirror, then, under the secondary condition of planarity, it is possible to monitor and/or calibrate the relative positions of screen and camera based on the known positions of the markings.

(26) For the evaluation of the different test criteria, target values and tolerances within which the measurement values must lie are verified in order to be rated as “in order” or acceptable. If determined measurement values of the deflectometry sensor system fall outside of the predetermined tolerances, the deflectometry sensor system is qualified as “out of order” and can thus not be used further.

(27) In a second step, on the screen, a defined image pattern can be generated, which is reflected via the reference plate and recorded by the capturing device or its cameras as capturing elements. Via the reflection of the image pattern represented by the screen, the relative position of screen and cameras with respect to one another can then be calibrated.

(28) The advantage of this method lies in that with one image recording per camera all the system parameters to be checked can be determined, whereby a rapid and simple checking of the deflectometry sensor system, for example, in the production line is possible.

(29) If one uses non-planar designs of the reference plate, i.e., an embodiment example of the reference plate which has, For example, markings—such as, for example, coded marks and/or Siemens stars—at different levels on the surface of the reference plate, or if one carries out the recordings described for the function check from multiple relative positions between deflectometry sensor and reference plate, then, in addition to the determination of external camera parameters, for example, 3D position and 3D orientation of the cameras (3×rotation, 3×translation), a determination of the internal camera parameters is also possible. Consequently, a complete calibration of the optical system and—in addition—of the whole deflectometry sensor system can be implemented.

(30) As an example, a procedure sequence of the function check according to an embodiment example of the method according to the disclosure can be carried out as follows:

(31) 1. The deflectometry sensor (including a screen as image generation device and multiple cameras as capturing elements of the capturing device) and the reference plate are positioned at a predefined distance from one another and at a predefined inclination.

(32) 2. Representation of a white image on the screen of the deflectometry sensor.

(33) 3. Recordings of the reference plate are produced with all the cameras of the deflectometry sensor.

(34) 4. Check of the sharpness. If the sharpness value of the capturing device of the deflectometry sensor is outside of a settable operating range, the deflectometry sensor cannot be operated further.

(35) 5. Check of the viewing ranges. If the deviations of the viewing ranges of the capturing device of the deflectometry sensor are outside of a settable operating range, the deflectometry sensor cannot be operated further.

(36) 6. Check of the brightness. If the brightness of the screen of the deflectometry sensor is outside of a settable operating range, then the screen is recalibrated.

(37) 7. Check of the position of the cameras of the capturing device. If a camera position is outside of a settable operating range, the deflectometry sensor cannot be operated further.

(38) 8. Check of the robot correction. If the calculated correction leads to a better position of the sensor in front of the reference plate and if it is within a settable operating range, the robot on which the deflectometry sensor is mounted is newly corrected. If the correction is outside of a settable operating range, the deflectometry sensor cannot be operated further.

(39) 9. Output of warning signals if an exchange of the deflectometry sensor is imminent, or output of error signals if the deflectometry sensor cannot be operated further.

(40) This procedure can be carried out in an example at regular time intervals such as, for example, once per day.

(41) Below, the individual check steps according to an embodiment example of the method according to the disclosure and in reference to FIG. 5 to FIG. 12 are described:

(42) Sharpness check:

(43) For checking the sharpness of the deflectometry sensor, the sharpness values of all the Siemens stars of each camera are evaluated. In an example, this occurs automatically.

(44) The Siemens stars—arranged as markings on the surface of the reference plate—are configured in such a manner that each Siemens star has a fixed number of segments. The sharper the image recorded by a camera is, the farther inward the individual segments can be seen. Thus, the ratio of the radius at which all the segments of the Siemens star can still be clearly recognized (from outside to inside) to the radius of the Siemens star can be a measure of the sharpness 1 minus. The value range of the sharpness can thus be from 0% (completely outside) to 100% (completely inside).

(45) By converting the sharpness value in percent into a value which is independent of the size of the Siemens star, Siemens stars of different size can be used for evaluating the sharpness. This results in the following corresponding calculation:

(46) $Lp / mm (Line pairs / millimeter) = \frac{Number of segments}{Pi * sharpness value [%] * diameter}$

(47) As an example, FIG. 5 shows a Siemens star 12 for an embodiment example of a reference plate according to the disclosure and for the sharpness check according to an embodiment example of the method according to the disclosure, wherein the no longer resolved, blurry area d is 18 percent of the total range D there.

(48) For the evaluation of the total sharpness of a camera, i.e., a capturing element of the capturing device, the minimum of the sharpness values of all the “visible” Siemens stars of the camera can be used, for example.

(49) “Visible” here is understood to mean Siemens stars which can be detected by means of the recorded image of the camera.

(50) Viewing Range Check:

(51) The checking of the viewing ranges can occur by monitoring the Siemens star positions. The target positions are predetermined, for example, by a CAD model.

(52) FIG. 6 shows, in a diagrammatic view, an illustration of a viewing range check according to an embodiment example of the method according to the disclosure.

(53) Camera Position Check:

(54) For determination of the position of a camera, the coded marks arranged on the reference plate are used. In an example, they are detected automatically.

(55) FIG. 7 shows, in a diagrammatic view, an illustration of a camera position check according to an embodiment example of the method according to the disclosure.

(56) Since the real or actual coordinates of the coded marks—based on the unique numbering by the code—on the reference plate are known, it is possible to determine the current 3D position of the reference plate or of the camera with the help of the precalibrated camera. This can be used for monitoring or checking the camera position, since the reference plate and also the robot position are held stationary.

(57) Robot Correction Check:

(58) For the determination of a robot correction, in order to compensate for mounting and production tolerances, the detected coded marks of all the cameras of the capturing device of the deflectometry sensor can be used. For this purpose, all the coded marks are transferred into a common coordinate system with the help of the information from the camera calibration and a currently determined camera position.

(59) The solution of a minimization problem which is to be computed, then yields a transformation which optimally reproduces all the coded marks in space in the ideal CAD position. This transformation is stored as correction in the robot, in order to optimally align the deflectometry sensor.

(60) Brightness Check:

(61) In order to obtain reproducible results in a deflectometric measurement, the brightness of the screen should be nearly constant. Due to the aging of a screen, the brightness decreases over time. This is represented in FIG. 8, for example. FIG. 8 shows a diagram for illustrating the aging of a screen with regard to the relative light emission, wherein the relative light emission in percent is plotted against time.

(62) As the brightness oscillations increase over time, the parametrization effort and the occurrence of measurement errors also increase.

(63) Therefore, the backlight of the screen of a deflectometry sensor is set so that the brightness in the center of the screen assumes a passable target value which does not correspond to the maximum brightness. At regular intervals, the brightness will be controlled and the brightness loss caused by aging will be compensated by an increase in the backlight brightness. For this purpose, in practice, an external measurement device as well as numerous manual steps are necessary.

(64) For example, in the case of sensors in paint defect control, this is entails considerable effort during use in the production line. Thus, for example, the body inlet must be blocked, the measurement cell must be accessed, the deflectometry sensor on the robot must be moved into a maintenance position, a prolonged waiting time must be observed in order to compensate for storage-caused brightness oscillations, a brightness measurement device must be attached to the screen, a connection between screen and a laptop must be established, the brightness must be set by software taking into consideration the brightness measurement device, by means of a laptop to be connected or on menu keys on the screen, the measurement cell must be left again, the robot must be moved back into a home position, and the body inlet must be unblocked again.

(65) For this sequence, at least 30 minutes per sensor are necessary. Since this is hardly feasible during ongoing production, weekend interventions are sometimes necessary for this purpose. Recalibrating the brightness is therefore possible only at longer time intervals.

(66) This results in the requirement that the image brightness must be resettable in an automated manner at short intervals, without having to interrupt the production for this purpose.

(67) In order to check the function capability of the deflectometry sensor system, for example, on daily basis, the above-described procedure sequence is carried out as “sensor check.” In the context of the function check, for the brightness check, for each camera of the deflectometry sensor, in a rectangle in the center of the respective image, the average grayscale value in the unprinted areas of the mirror, i.e., of the reflective surface of the reference plate, is determined. This average grayscale value is referred to as “camera grayscale value” below.

(68) The apertures of the camera lens are set during the sensor production in such a manner that, at the target brightness of the screen, a defined camera grayscale value is reached.

(69) If the brightness of the screen decreases due to aging of the screen, then the images recorded by the cameras—in the context of the function check—become darker and the camera grayscale value decreases. If the backlight is set to be brighter, the screen brightness and the images recorded by the cameras become brighter, that is to say the camera grayscale value increases. Due to this relationship between camera grayscale value, backlight setting and screen brightness, it is possible to determine, based on the images recorded in the context of the function check, whether and to what extent the backlight must be readjusted in order to compensate for the aging-caused brightness decrease of the screen.

(70) The following experimentally determined results refer, as an example, to a screen with screen controller known from practice, wherein the camera lenses are set so that a screen brightness of 600 cd/m.sup.2 leads to a camera grayscale value of 230.

(71) By means of the screen controller, the backlight of a screen can be regulated in 1% steps from 0% to 100%. The screen has a maximum brightness of 800 cd/m.sup.2.

(72) Between the camera grayscale value GW and the screen brightness DH, a linear relationship exists. This is represented in the diagram according to FIG. 9 which is used for illustrating the relationship between camera grayscale value and screen brightness of a deflectometry sensor. In FIG. 9, the camera grayscale value is plotted against the screen brightness in cd/m.sup.2. A completely overexposed camera image has a camera grayscale value of 255. Due to the overexposure, the linear relationship is valid only for camera grayscale values in the interval [0; 250].

(73) The linear relationship is determined by the target grayscale value GW.sub.target and the target brightness DH.sub.target:

(74) $GW = \frac{{GW}_{target}}{{DH}_{target}} * DH$

(75) where GW.sub.target=230 and DH.sub.target=600 cd/m.sup.2 in the configuration used.

(76) FIG. 10 shows a diagram for illustrating the relationship between image brightness and backlight setting of the screen of a deflectometry sensor, wherein the camera grayscale value is plotted against the backlight setting as a percentage. According to the diagram from FIG. 10, the backlight setting behaves linearly in the interval [0%; 95%] with respect to the screen brightness. In the case of a backlight setting of 0, the screen has a residual brightness of approximately 180 cd/m.sup.2 (compare FIG. 10), resulting in a camera grayscale value GW.sub.0 of approximately 70, compare FIG. 11. FIG. 11 shows a diagram for illustrating the relationship between camera grayscale value and backlight setting of the screen of a deflectometry sensor, wherein the camera grayscale value is plotted against the backlight setting as a percentage.

(77) In the case of a backlight setting from 95% to 100%, the screen brightness is nearly constant. This can be obtained from the diagram according to FIG. 12, which plots the relationship between screen brightness and backlight setting in the interval [80; 100] of a deflectometry sensor. In FIG. 12, the screen brightness in cd/m.sup.2 is plotted against the backlight.

(78) Thus, the backlight setting BLnew can be calculated for a target grayscale value GW.sub.target of the camera as a function of the current camera grayscale value GWcurrent, the zero grayscale value GW.sub.0 and as a function of the current backlight setting BL.sub.current:

(79) ${BL}_{new} = \frac{{GW}_{target} - {GW}_{0}}{{GW}_{current} - {GW}_{0}} * {BL}_{current}$

(80) with GW.sub.target=230 in the configuration used.

(81) The screen controller has an RS232 interface by means of which the screen settings such as, for example, the backlight can be requested and set. The sensor can be connected to the sensor PC on the robot by an optical fiber connection.

(82) Thus, by means of the application software, it is then possible to set the backlight of the screen and thus correct the brightness decrease of the screen during ongoing operation in an automated and regular manner. Since this can occur in the context of the daily function check or sensor check, no additional expenditure of time is necessary. In addition, in an automated manner, a request for replacement of the old screen can be issued as soon as the screen has aged enough so that the maximum backlight setting is no longer sufficient to compensate for the brightness decrease. Thus, a constant measurement value can be achieved in an automated manner during production.

(83) With regard to additional advantageous designs of the reference plate according to the disclosure and of the method according to the disclosure, reference is made to the general part of the description and to the appended claims in order to avoid repetitions.

(84) Finally, it is explicitly pointed out that the above-described embodiment examples of the reference plate according to the disclosure and of the method according to the disclosure are used only for the purpose of explaining the claimed teaching, without, however, limiting it to the embodiment examples.

LIST OF REFERENCE NUMERALS

(85) 1 Deflectometry sensor

(86) 2 Screen

(87) 3 Camera

(88) 4 Measurement object

(89) 5 Reference plate

(90) 6 Coded mark

(91) 7 Siemens star

(92) 8 Marking group

(93) 9 Marking group

(94) 10 Marking group

(95) 11 Marking group

(96) 12 Siemens star

(97) 8′ Camera image

(98) 9′ Camera image

(99) 10′ Camera image

(100) 11′ Camera image

Reference plate and method for calibrating and/or checking a deflectometry sensor system

Assignee

Inventors

Cpc classification

Classification Explorer

G01N2021/9518

PHYSICS

Classification Explorer

G01N21/9515

PHYSICS

Classification Explorer

G01M11/0264

PHYSICS

Classification Explorer

G01N21/93

PHYSICS

Classification Explorer

G01N21/8806

PHYSICS

Classification Explorer

G01B11/25

PHYSICS

Classification Explorer

G01N2021/8829

PHYSICS

Classification Explorer

G01B11/2504

PHYSICS

International classification

Classification Explorer

G06K9/46

PHYSICS

Classification Explorer

G01N21/93

PHYSICS

Classification Explorer

G01N21/88

PHYSICS

Classification Explorer

G01M11/02

PHYSICS

Classification Explorer

G01B11/25

PHYSICS

Classification Explorer

G01N21/95

PHYSICS

Abstract

Claims

Description