Optoelectronic sensor and method for a safe evaluation of measurement data

Abstract

An optoelectronic sensor for detecting objects in a monitored zone is provided having at least one light receiver for generating measurement data from received light from the monitored zone and having a safe evaluation unit that has at least two processing channels for a redundant processing of the measurement data and having a comparison unit for comparing processing results of the processing channels to uncover errors in a processing channel 30a-b. The processing channels are here each configured for a determination of a signature with respect to their processing results; and the comparator unit is configured for a comparison of the signatures.

Claims

1. An optoelectronic sensor for detecting objects in a monitored zone, the optoelectronic sensor having at least one light receiver for generating measurement data from received light from the monitored zone; a safe evaluation unit that has at least two processing channels for a redundant processing of the measurement data; and a comparison unit for comparing processing results of the processing channels to uncover errors in a processing channel, wherein the processing channels are each configured for a determination of a signature with respect to their processing results; and wherein the comparison unit is configured for a comparison of the signatures, the signature obtained by aggregating at least a subset of safety-relevant data in small data blocks, wherein a change in the safety-relevant data renders a change of the signature, and wherein the processing channels each have a plurality of processing stages and are configured to determine a signature in a plurality of processing stages for the processing results of said processing stage.

2. The sensor in accordance with claim 1, wherein the sensor is a camera having at least one image sensor as the light receiver that generates image data as measurement data.

3. The sensor in accordance with claim 2, in which camera the processing channels have at least some of the following processing stages: reading of raw images, preprocessing of raw images, generation of a depth map by a stereoscopic algorithm, generation of a detection map having relevant detected objects, determining shortest distances of the relevant objects from hazard sites, and deriving a safety related response from the shortest distances.

4. The sensor in accordance with claim 2, wherein the camera is a stereo camera.

5. The sensor in accordance with claim 1, wherein the signature has a hash value calculated from the processing results.

6. The sensor in accordance with claim 5, wherein a cyclic redundancy test is used as the hash function.

7. The sensor in accordance with claim 1, wherein the processing channels each have a plurality of processing stages and are configured to determine a signature in all the processing stages for the processing results of said processing stage.

8. The sensor in accordance with claim 1, wherein processing channels are configured to collect signatures from different processing stages.

9. The sensor in accordance with claim 8, wherein signatures are each forwarded to the next processing stage.

10. The sensor in accordance claim 1, wherein the processing channels have a signature store in which the processing stages store their respective signatures.

11. The sensor in accordance with claim 1, wherein the processing channels are configured to determine the signature in at least one processing stage via the processing results of the processing stage and to determine a signature from a preceding processing stage.

12. The sensor in accordance claim 1, wherein the processing channels have processing stages on different elements.

13. The sensor in accordance with claim 1, wherein the processing channels are configured to forward processing results and/or signatures between the processing stages over an unsafe communication link.

14. The sensor in accordance with claim 1, wherein the comparison unit is configured only to compare signatures at the end of the processing channels.

15. The sensor in accordance with claim 1, wherein the safe evaluation unit is configured to forward signatures to the comparison unit via an unsafe communication link.

16. The sensor in accordance with claim 1, wherein the processing channels are configured to channel test data into the measurement data or processing results.

17. The sensor in accordance with claim 16, wherein the processing channels are also configured to determine the signature via processing results of the test data.

18. The sensor in accordance with claim 1, wherein the processing channels are configured to provide data and/or signatures with an error correction option for the transmission.

19. The sensor in accordance with claim 1, wherein the processing channels are configured to delete no longer required portions of the processing results.

20. The sensor in accordance with claim 1, wherein the safe evaluation unit has more than two processing channels and the comparison unit is configured for a voting process for evaluating signatures.

21. A method for a safe evaluation of measurement data of an optoelectronic sensor, wherein the measurement data are redundantly processed in at least two processing channels and the processing results of the processing channels are compared to uncover errors, wherein, in the processing channels, a respective signature of their processing results is determined and only the signatures are compared, the signature obtained by aggregating at least a subset of safety-relevant data in small data blocks, wherein a change in the safety-relevant data renders a change of the signature, and wherein the processing channels each have a plurality of processing stages and are configured to determine a signature in a plurality of processing stages for the processing results of said processing stage.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) The invention will be explained in more detail in the following also with respect to further features and advantages by way of example with reference to embodiments and to the enclosed drawing. The Figures of the drawing show in:

(2) FIG. 1 a schematic three-dimensional representation of a 3D camera and its the monitored zone;

(3) FIG. 2 an exemplary monitoring situation with a plurality of hazard sites and objects;

(4) FIG. 3 a schematic representation of a two channel evaluation unit;

(5) FIG. 4 a schematic representation of a multistage processing channel with a generation of cascaded signatures;

(6) FIG. 5 a schematic representation of a multistage processing channel similar to FIG. 4 in which, however, the signatures of the processing stages are centrally collected;

(7) FIG. 6 a schematic representation of a two channel evaluation unit with multistage processing channels and a one-time comparison at the end of the processing; and

(8) FIGS. 7a-b schematic representations of the attachment of test data by extending image lines or attaching image lines.

DETAILED DESCRIPTION

(9) FIG. 1 shows the general design of a stereo camera 10 for recording a depth map in a schematic three-dimensional representation. The stereo camera 10 serves only as an example for a sensor in accordance with the invention that as a rule generates particularly large data volumes and therefore illustrates the advantages of the invention to a particular degree. Other optoelectronic sensors, inter alia the other 3D cameras named in the introduction, would equally be conceivable with a determination of the time of flight or an evaluation of the interference of passive two-dimensional patterns or with a correlation of image and projected illumination patterns and laser scanners.

(10) To detect a spatial zone 12, two camera modules 14a, 14b are mounted at a known fixed distance from one another and each take images of the spatial zone 12. An image sensor 16a, 16b, usually a matrix-type imaging chip, is provided in each camera and records a rectangular pixel image, for example a CCD or a CMOS sensor. One objective 18a, 18b having an optics which in practice can be realized as any known imaging lens is associated with each of the image sensors 16a, 16b. The maximum angle of view of these optics is shown in FIG. 1 by dashed lines which each form a pyramid of view 20a, 20b.

(11) An illumination unit 22 is provided between the two image sensors 16a, 16b to illuminate the spatial zone 12 with a structured pattern. The stereo camera shown is accordingly configured for active stereoscopy in which the pattern also imparts evaluable contrasts everywhere to scenery that is structure-less per se. Alternatively, no illumination or a homogeneous illumination is provided to evaluate the natural object structures in the spatial one 12, which as a rule, however, results in additional aberrations.

(12) An evaluation and control unit 24 is connected to the two image sensors 16a, 16b and to the lighting unit 22. The control and evaluation unit 24 can be implemented in the most varied hardware, for example digital modules such as microprocessors, ASICS (application specific integrated circuits), FPGAs (field programmable gate arrays), GPUs (graphics processing units) or mixed forms thereof that can be distributed as desired over internal and external components, with external components also being able to be integrated via a network or cloud provided that latencies can be managed or tolerated. Since the generation of the depth map and its evaluation are very computing intensive, an at least partly parallel architecture is preferably formed.

(13) The control and evaluation unit 24 generates the structured illumination pattern with the aid of the illumination unit 22 and receives image data of the image sensors 16a, 16b. It calculates the 3D image data or the depth map of the spatial zone 12 from these image data with the aid of a stereoscopic disparity estimate. The total detectable spatial zone 12 or also the working region can be restricted via a configuration, for example to mask interfering or unnecessary regions.

(14) An important safety engineering application of the stereo camera 10 is the monitoring of a machine 26 that is symbolized by a robot in FIG. 1. The machine 26 can also be substantially more complex than shown, can consist of a number of parts, or can actually be an arrangement of a plurality of machines, for instance of a plurality of robots or robot arms. The control and evaluation unit 24 checks where an object 28, shown as a person, is located with respect to the machine 26. A smallest distance of an object 28 from the machine 26 is output via a safe interface 29, either directly to the machine 26 or to an intermediate station such as a safe control. The stereo camera 10 is preferably in total failsafe in the sense of safety standards such as those named in the introduction. The evaluation in the control and evaluation unit 24 is safe due to a special multichannel structure that will be explained in more detail below with reference to FIGS. 3 to 7.

(15) A control connected to the safe interface 29, either a higher ranking control or that of the machine 26, evaluates the shortest distance. In the hazard case, a safety related response is initiated in order, for example, to stop or brake the machine 26 or to cause it to evade. Whether this is necessary can, in addition to the shortest distance, depend on further conditions such as the speeds or the nature of the object 28 and the machine zone 26 of the impending collision. The safety evaluation can alternatively take place in the control and evaluation unit 24 and can also be based on different criteria than a shortest distance.

(16) A distance monitoring will be described in somewhat more detail, but only as representative, for a human-robot collaboration while taking account of DIN EN ISO 10218 and/or ISO/TS 15066. The starting point is formed by the positions of the machine parts of the machine 26, at least to the extent that they are safety relevant, or by hazard sites defined on this basis and optionally expanded with reference to response and stopping times or other criteria and by the objects 28 detected by the stereo camera 10. The latter is, for example, present in the form of a 2D detection map, its pixels at positions in which an object 28 of a minimum size was detected, the distance value measured for this purpose is entered and otherwise remains empty. The respective distance, and in particular the shortest distance, from the machine 26, that forms a hazard site that is preferably also dynamic is calculated with the aid of these object detections that can naturally also be differently represented. Depending on the distance, a securing then takes place, optionally by a control connected to the safe interface 29, that can, as mentioned multiple times, also comprise an evasion or a slowing down.

(17) FIG. 2 shows an exemplary monitoring situation in the monitored zone 12. The securing task on the basis of the stereo camera 10 then comprises recognizing the presence of persons, here simply defined as objects 28 of a specific minimum size, and initiating a defined response in a safety related manner in dependence on their position and optionally on further parameters and the current machine status so that the safety of the humans is ensured at all times.

(18) In this example, two hazard sites 26a-b have to be monitored, that is machine regions or machines, and four objects 28 are currently recognized in their environment by the stereo camera 10. The stereo camera 10 delivers distance data so that a connected control protects the persons from injury by a reduced speed, an evasive replanning of the routines, or where necessary a stop of the machines in the hazard areas 26a-b in good time.

(19) A hazard site 26a-b is a preferred modeling of the hazardous machine 26. The hazard site 26a-b is a spatial zone in which the machine 26 carries out work movements in a respective time period. The hazard site 26a-b can surround the machine 26 or partial regions of the machine with a little spacing to leave sufficient clearance for the work movements. In addition, it is advantageous for the calculations to define geometrically simple hazard areas 26a-b such as parallelepipeds or spheres, for which purpose certain empty spaces can then be accepted. A plurality of hazard sites 26a-b surround a plurality of machines 26 and/or a plurality of moving part sections of a machine 26. Hazard sites 26a-b can be rigid and can comprise all conceivable work movements. Alternatively, respective hazard sites 26a-b are defined for part sections of the work movement that are utilized in a sequence corresponding to the process and that are smaller and are better adapted.

(20) The control and evaluation unit 24 continuously calculates the shortest distance of the object 28 closes to a respective hazard site 26a-b. Arrows are drawn in FIG. 2 that in the current situation of FIG. 2 represent the two shortest distances with respect to the two hazard sites 26a-b. The shortest distance connects the closest point of a hazard site 26a-b to the nearest point of the next object 28. It is assumed in this representation that the small object 28 at the bottom right exceeds the minimum size. It would otherwise be ignored and instead the distance from the two merged persons who form the second-closest object 28 would be output.

(21) The respective shortest distance last determined with respect to a hazard site 26a-b is provided cyclically or acyclically at the safe interface 29. Typical output rates are multiple times a second; however, a more infrequent updating is also conceivable depending on the required and possible response time of the stereo camera 10. A higher ranking control connected to the safe interface 29, in particular that of the machine 26, plans the next workstep again, where necessary in dependence on the shortest distance, so that the required safety distance between human and machine is always maintained.

(22) The control and evaluation unit 24 preferably also determines a speed of the object 28 from which the shortest distance was measured and outputs it with the shortest distance to the safe interface 29. The hazard can thus be differentiated even better. The closest object 28 is admittedly the most dangerous as a rule—or in more precise terms the one most at risk. The safety distance that the machine 26 maintains on its movement planning can additionally be adapted to a maximum speed of a human movement. The safety related response of the machine is nevertheless best adapted to its environment if more information is present on the closest object 28 and possibly also on further objects 28. A dependence on the machine's own status and on the planned movement of the machine 26, in particular the position and speed of machine parts or even of dangerous tool regions, is also conceivable, with such information preferably being provided by the machine control.

(23) There are a number of further measurement parameters or of parameters derived therefrom that the control and evaluation unit 24 can output, in addition to the shortest distance, to the safe interface 29 so that they can enter into the safety observation of the control connected there. The speed of the closest object 28 from which the shortest distance is measured has already been discussed. Additional shortest distances from further objects 28 or from separate object sections of the closest object 28, for example of a different arm, are preferably output. A possible criterion here would be that there are even further local distance minima in the same object since the direct adjacent points from the shortest distance are of no interest. For example, the stereo camera 10 guarantees the monitoring of the five closest distances per active hazard site 26a-b. A sixth object and further objects or object sections are no longer considered, with an additional piece of information being conceivable, however, that there are more than five objects of the minimum size in the monitored zone 12. The connected control can thus also pre-plan for further future danger situations with other objects 28 than the closest object 28. A plastic example is a still somewhat more remote object 28 that approaches a hazard site 26a-b at high speed.

(24) Further conceivable additional pieces of information are, non-exclusively, the size of the next object 28, its position in the form of a focus or of the closest point, a direction of movement, an object envelope, an enveloping body surrounding the object 28, or a representation of the object 28 in total as an object cloud, 3D point cloud, or 3D voxel representation.

(25) FIG. 3 shows a schematic representation of the control and evaluation unit 24 in an embodiment having two processing channels 30a-b. The actual measurement data of the stereo camera 10 and thus the starting size are the raw image of the two camera modules 14a-b. Each processing channel 30a-b has a plurality of processing stages. However, FIG. 3 does not show the actual processing stages that are only introduced in FIG. 4, but rather the hardware structure with a respective two modules connected after one another, for instance a respective FPGA 32a-b and a respective multicore processor 34a-b. Differently, there can be only one such module or more modules and one or more processing stages can be implemented on each module. Communication to the outside takes place via a communication processor 36 that is not redundantly provided here and whose securing is not further discussed because it is no longer part of the actual evaluation.

(26) The image sensors 16a-b of the two camera modules 14a-b deliver two images at a frame repetition rate of, for example, fifty frames per second with a resolution that is typically in the order of magnitude of megapixels. These images are offset to form a depth map from which then detection maps and other characteristic value maps can be generated. The anyway high data volume thus increases even further in the first processing steps. A complete comparison of the processing results between the processing channels 30a-b on the individual processing stages would therefore be extremely complex and/or expensive. However, an error-free or error-recognizing processing must be guaranteed and if necessary a safety related response has to be triggered. Only signatures are therefore compared in accordance with the invention that are directly introduced with reference to FIG. 4.

(27) However, a further advantageous aspect of the invention should still be explained from FIG. 3. Two kinds of communication are necessary in the two channel evaluation unit 24, namely within the processing channels 30a-b for forwarding data between the processing stages also beyond elements 32a-b, 34a-b, and also between the processing channels 30a-b for the comparison of processing results or signatures. This communication preferably takes place over unsafe communication paths, in particular black channels, and the communication paths are drawn by arrows correspondingly marked by BC in FIG. 3. Black channels were introduced in the introduction; the communication is accordingly implemented cost-efficiently and hardware-efficiently without safety technology and the securing of the data takes place via signatures and functional monitoring.

(28) FIG. 4 shows by way of example only one of the processing channels 30a to now illustrate the multistage structure on the plane of processing stages 38a.sub.1-3. The processing stages 38a.sub.1-3 can be implemented on a plurality of elements as was described with reference to FIG. 3. The processing stages 38a.sub.1-3, whose shown number of three is naturally purely by way of example, can in principle carry out any desired evaluations of the measurement data.

(29) As a specific example, the stereo camera 10 with distance monitoring in accordance with FIGS. 1 and 2 should again be made use of as a representative and a conceivable processing pipeline for it should be described. A stereo camera 10 could also be evaluated differently and the multichannel structure in accordance with the invention is also suitable for different detection principles and sensors.

(30) The camera modules 14a-b of the stereo camera 10 initially generate raw images. They are preprocessed in a first processing stage to compensate smaller interference points, brightness differences, and similar or to carry out a geometry correction. A depth map is then generated from the raw images in a further processing stage by means of a stereo algorithm. Relevant objects are then detected in the depth map. This can per se require a plurality of processing stages, for instance to mask hazard sites 26a-b that are not themselves monitored to exclude background objects or to ignore small interference objects and defects in which no depth values can be detected. Conditions can also be made on the detected objects such as a minimum size or a coincidence with a body model. In a further processing stage, the shortest distance from the next object 28 is then determined for every hazard site 26a-b. This can be the sought output size for the safe interface 29 and thus the last processing stage. Alternatively, the evaluation of the shortest distances represents a further processing stage that results in a safety related response or an unimpeded continuance of the worksteps of the machine 26.

(31) After this specific example, the processing stages 38a.sub.1-3 will from now on be looked at in the abstract. The respective processing result of a processing stage 38a.sub.1-3 is, on the one hand, passed on to the next processing stage. In addition, a signature 40a.sub.1-3 is determined from the processing results that serves to uncover errors in the processing channels 30a-b.

(32) This signature 40a.sub.1-3 for the representation of the information of the processing results can be a hash value of a previously fixed hash function of the data to be compared. In addition, aggregated intermediate values of the data processing can serve as part of the signature 40a.sub.1-3, for instance a hash value for a detected object 28 having features such as its size and position. Finally, results of internal tests are also conceivable that are later compared with an expectation, with either the test results themselves or a signature thereof being stored as a part of the signatures 40a.sub.1-3.

(33) The signatures 40a.sub.1-3 aggregate relevant information in very small data blocks and thus enable an efficient comparison of the data between the two redundant processing channels 30a-b. The signature 40a.sub.1-3 is preferably generated over all the respective generated data of the processing results of the processing stage 38a.sub.1-3 so that a change somewhere in the data is reflected in a change of the signature 40a.sub.1-3. An advantageous hash function for determining signatures is a CRC (cyclic redundancy check) process. This can be efficiently calculated and has all the required properties.

(34) The signatures 40a.sub.1-3 can, as shown in FIG. 4, be passed on in the data flow to the next processing stage 38a.sub.1-3 and can increase in so doing. Each processing stage 38a.sub.1-3 here adds its signature 40a.sub.1-3.

(35) FIG. 5 shows an alternative in which the signatures 40a.sub.1-3 are centrally collected in a signature store 42a. They are then transferred together at the end of the last processing stage 38a.sub.1-3. Mixed forms are conceivable in which signatures 40a.sub.1-3 are passed on in part as in FIG. 4 and are stored centrally in a signature store 42a in at least one processing stage 38a.sub.1-3 as in FIG. 5.

(36) In all these cases, the procedure facilitates the error identification since the deviations of the signatures in the processing channels start at a specific defective processing stage 38a.sub.1-3. Differing from the representations of FIGS. 4 and 5, it is also conceivable not to keep any individual signatures 40a.sub.1-3, i.e. neither in growing form nor centrally stored. Instead, each processing stage 38a.sub.1-3 generates a respective signature into which the signature of the preceding processing stage 38a.sub.1-3 also enters in addition to the processing results of its own processing stage 38a.sub.1-3. Such a signature therefore also represents the previous signatures as a kind of nested meta-signature. At the end, only a single signature thus has to be compared; however, at the price that an error is only uncovered, but cannot be associated with a specific processing stage 38a.sub.1-3.

(37) FIG. 6 again schematically shows an embodiment of a safe two channel control and evaluation unit 24, with the two processing channels 30a-b being on a functional plane with processing stages 38a-b.sub.1-3 as shown in FIG. 4, and not as a hardware structure as in FIG. 3. The number of processing stages 38a-b.sub.1-3 is furthermore purely exemplary and the structure in accordance with FIG. 5 could alternatively also be selected for the individual processing channels 30a-b.

(38) As can be recognized in FIG. 6 and also already in FIG. 2, only the last aggregated signatures 40a-b.sub.1-3 of the last processing stages 38a-b.sub.3 are preferably compared. An intermediate comparison in other processing stages 38a-b.sub.1-2 would be generally conceivable, but means an additional effort that is not at all necessary because the comparison at the end can also localize the error.

(39) The signatures 40a-b.sub.1-3 carried along with the data flow or alternatively the centrally collected signatures 40a-b.sub.1-3 are compared crosswise at the end of the processing chain in a respective comparator unit 44a-b of the processing channels 30a-b. The cross-communication and the comparison effort are thereby even doubly substantially reduced, namely because only signatures 40a-b.sub.1-3 are compared, on the one hand, and this is done only once at a central point, on the other hand. This approach also has the advantage that data no longer required within the processing chain can be deleted since the information required for the comparison is stored in the signatures 40a-b.sub.1-3.

(40) FIG. 6 is simplified in the respect that the forwarding of the processing results of the last processing stage 38a-b.sub.1-3 is not shown. An arrow from the last processing stage 38a-b.sub.1-3 to the communication processor 36 would therefore have to be added notionally. For this purpose, data from any desired one of the two processing channels 30a-b can be used of which it has been separately demonstrated that they coincide.

(41) In addition to the determination of signatures 40a-b.sub.1-3 for the respective processing results of the processing stages 38a-b.sub.1-3, functional tests are also conceivable with specific stimulations to directly check specific aspects of the processing. Such test data can extend the data lines as in FIG. 7a or additional test lines are attached to the data lines as in FIG. 7b. Alternatively, whole test data sets can be inserted into gaps between the functional data sets. They then take up their own test time windows, but this has the advantage that the test coverage can be increased. For example, high and narrow or low and wide images can be applied that together in total efficiently and effectively test the processing over the total image extent.

(42) The processing results of test data can preferably likewise enter into the signature 40a-b.sub.1-3 of the processing stage 38a-b.sub.1-3 or can alternatively be compared with the expectation within the processing stage 38a-b.sub.1-3. A central comparison at the end of the processing channels 30a-b is preferred since this requires the least additional effort. No local comparator units thus have to be provided and the comparator unit 44a-b at the end of the processing channels 30a-b is anyway adapted for signature comparisons and where required is safe in itself by function tests.

(43) As explained for different embodiments, an aspect of the invention is the utilization of signatures 40a-b.sub.1-3 for a resource-saving error discovery. It is an advantageous additional aspect to carry out the comparison only once in a central comparator unit 44a-b at the end of the processing channels 30a-b. As a further advantageous aspect, the communication takes place in an non-safe manner, in particular by means of black channels, both within the processing channels 30a-b and between processing channels 30a-b. The safety protocol achieves a security against transmission errors such as repetition, loss, insertion, incorrect order, delay, or defective data transmission. The data integrity is safe, for example, via continuous numbers of the data packets, a time monitoring, and data CRCs. In the event of an error, the just-named transmission errors can be recognized. In addition defective data packets can be corrected by the additional use of an error-correcting encoding of the transmission secured by a black channel in the standard channel, whereby system availability is increased.

(44) The invention has been described for embodiments having two processing channels 30a-b. A multichannel structure is also conceivable having at least one additional processing channel that then has the same design per se as one of the two processing channels 30a-b. A majority decision (voting) then preferably takes the place of a simple comparison with the assumption of an error in the case of non-coincidence, with the required majority being a means to weigh the safety level and the availability with respect to one another. If, for example, unanimous voting is required, the additional processing channel only increases the safety, while the likelihood that an error occurs in a processing channel increases at the costs of availability. A 2:1 majority that tolerates a deviation in a processing channel at higher availability may still be considered very safe because it is extremely unlikely that the same error occurs in two processing channels at the same time.

(45) Alternatively to a comparison of signatures 40a-b.sub.1-3, a direct comparison of the data or of specific higher value features such as detected objects or test decisions would also be possible. A safe evaluation then also becomes possible, but the advantages of simple, resource-saving comparisons are lost. It is further conceivable not to compare all the data, but rather to reach the data volume by omitting some of the data. However, this brings about a smaller error discovery and is thus disadvantageous for the functional safety of the system.