SYSTEM AND PROGRAM
20250361698 ยท 2025-11-27
Assignee
Inventors
Cpc classification
G05D2105/05
PHYSICS
International classification
Abstract
A system detects a traveling obstacle region that is a region that obstructs traveling of a work machine. The system includes a region angle detector, a region height detector, an angle obstacle region detector, a height obstacle region detector, and a traveling obstacle region detector. The region angle detector detects an angle of a traveling surface based on a distance image around the work machine. The region height detector detects height of the traveling surface based on the distance image. The angle obstacle region detector detects an angle obstacle region that is an obstacle region based on the detected angle. The height obstacle region detector detects a height obstacle region that is an obstacle region based on the detected height. The traveling obstacle region detector detects a traveling obstacle region based on the detected angle obstacle region and the detected height obstacle region.
Claims
1. A system that detects a traveling obstacle region that is a region that obstructs traveling of a work machine, the system comprising: a region angle detection unit that detects an angle of a traveling surface based on a distance image around the work machine; a region height detection unit that detects a height of the traveling surface based on the distance image; an angle obstacle region detection unit that detects an angle obstacle region that is an obstacle region based on the detected angle; a height obstacle region detection unit that detects a height obstacle region that is an obstacle region based on the detected height; and a traveling obstacle region detection unit that detects the traveling obstacle region based on the detected angle obstacle region and the detected height obstacle region.
2. The system according to claim 1, wherein the angle obstacle region detection unit detects the angle obstacle region based on a threshold of the angle.
3. The system according to claim 2, wherein the angle obstacle region detection unit detects the angle obstacle region based on the threshold adjusted according to information concerning the work machine.
4. The system according to claim 2, wherein the angle obstacle region detection unit detects the angle obstacle region based on the threshold adjusted according to information concerning a surrounding environment.
5. The system according to claim 1, wherein the height obstacle region detection unit detects the height obstacle region based on a threshold of the height.
6. The system according to claim 1, wherein the traveling obstacle region detection unit detects the traveling obstacle region by excluding a region having a predetermined height in the detected angle obstacle region.
7. The system according to claim 1, wherein the traveling obstacle region detection unit detects the traveling obstacle region by excluding a region having a predetermined width in the detected height obstacle region.
8. The system according to claim 1, further comprising an object detection unit that detects an object placed on the traveling surface, wherein the traveling obstacle region detection unit further detects a region of the detected object as the traveling obstacle region.
9. The system according to claim 1, further comprising an occlusion region detection unit that detects an occlusion region that is a region where the angle and the height on the traveling surface cannot be detected, wherein the traveling obstacle region detection unit further detects the detected occlusion region as the traveling obstacle region.
10. A program for detecting a traveling obstacle region that is a region that obstructs traveling of a work machine, the program comprising: a region angle detection procedure of detecting an angle of a traveling surface based on a distance image around the work machine; a region height detection procedure of detecting height of the traveling surface based on the distance image; an angle obstacle region detection procedure of detecting an angle obstacle region that is an obstacle region based on the detected angle; a height obstacle region detection procedure of detecting a height obstacle region that is an obstacle region based on the detected height; and a traveling obstacle region detection procedure of detecting the traveling obstacle region based on the detected angle obstacle region and the detected height obstacle region.
Description
BRIEF DESCRIPTION OF DRAWINGS
[0008]
[0009]
[0010]
[0011]
[0012]
[0013]
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021]
[0022]
[0023]
[0024]
DESCRIPTION OF EMBODIMENTS
[0025] Embodiments of the present disclosure are explained in detail below with reference to the drawings. Note that, in the embodiment explained below, redundant explanation is omitted by denoting the same parts with the same reference numerals and signs.
Traveling Path of a Work Device
[0026]
[0027] By detecting such a traveling obstacle region and presenting the traveling obstacle region to a user, it is possible to reduce occurrence of accidents such as a collision and a fall. The traveling obstacle region detection system of the present disclosure detects such a traveling obstacle region. A sensor 12 is disposed in the work machine 10 illustrated in the figure. An image around the work machine 10 is acquired by the sensor 12 and input to the traveling obstacle region detection system. As the sensor 12, for example, a distance measuring sensor that generates a distance image can be used. Here, the distance image is also referred to as depth map and is an image in which distance information is reflected for each pixel.
[0028]
Traveling Obstacle Region Detection System
[0029]
[0030] The data conversion unit 110 converts a distance image, which is input data, into point cloud data. The point cloud data is output to the region angle detection unit 120 and the region height detection unit 150.
[0031] The region angle detection unit 120 detects an angle of a traveling surface based on the point cloud data. The detected angle is output to the angle obstacle region detection unit 140.
[0032] The angle obstacle region detection unit 140 detects an angle obstacle region that is an obstacle region based on the angle of the traveling surface. The detected angle obstacle region is output to the traveling obstacle region detection unit 180.
[0033] The threshold generation unit 130 generates a threshold used for detection of the angle obstacle region in the angle obstacle region detection unit 140.
[0034] The region height detection unit 150 detects the height of the traveling surface based on the point cloud data. The detected height is output to the height obstacle region detection unit 170.
[0035] The height obstacle region detection unit 170 detects a height obstacle region that is an obstacle region based on the height of the traveling surface. The detected height obstacle region is output to the traveling obstacle region detection unit 180.
[0036] The threshold generation unit 160 generates a threshold used for detecting the height obstacle region in the height obstacle region detection unit 170.
[0037] The traveling obstacle region detection unit 180 detects a traveling obstacle region based on the angle obstacle region and the height obstacle region. The detected traveling obstacle region is output to an external device, for example, a display device.
[0038] In the following explanation, details of processing of the units are explained.
Data Conversion Processing
[0039]
[0040] Subsequently, down-sampling is performed on the point cloud data (Step S112). This down-sampling is processing for thinning out row and column data for each of the arrays of the coordinates of the x, y, and z axes. By this down-sampling, it is possible to reduce a processing amount and reduce noise.
[0041] Subsequently, coordinate correction is performed on the point cloud data after the down-sampling (Step S113). This coordinate correction is processing for converting the point cloud data into global coordinates.
[0042] The sensor 12 illustrated in
Angle Detection Processing
[0043]
[0044] In Step S125, the region angle detection unit 120 selects a point of attention of the point cloud data (Step S125). Subsequently, the region angle detection unit 120 calculates an angle with respect to the selected point of attention (Step S126). This can be performed by calculating an inner product of a normal line and a unit vector in the y-axis direction. Subsequently, the region angle detection unit 120 adjusts an angle sign (Step S127). This is to invert a sign of an angle when a slope angle is a depression angle. This can be performed based on a value of the z-axis array of a point. Specifically, when the value of the z-axis array is a positive value, the sign of the angle is inverted. Subsequently, the region angle detection unit 120 determines whether angles have been calculated for all points (Step S128). The region angle detection unit 120 ends the processing (Step S128, Yes) when the angles have been calculated for all the points and selects another point of attention (Step S128, No) when there is a point for which an angle has not been calculated (Step S125).
Adjacent Vector Generation Processing
[0045]
Normal Vector Generation Processing
[0046]
Angle Calculation Processing
[0047]
Detection Angle
[0048]
Angle Obstacle Region Detection Processing
[0049]
[0050] Subsequently, the angle obstacle region detection unit 140 detects an angle obstacle region (Step S132). The angle obstacle region detection unit 140 detects the angle obstacle region based on a threshold generated by the threshold generation unit 130. The threshold generation unit 130 can output a predetermined value, for example, 30, as the threshold. The angle obstacle region detection unit 140 can detect a region having an angle higher than the threshold as the angle obstacle region.
[0051] The threshold generation unit 130 can adjust the threshold according to information concerning the work machine 10. For example, the weight, the size, the center of gravity, an angled, a traveling direction (handle information), a speed, and the like of the work machine 10 correspond to the information concerning the work machine 10. In addition, the threshold generation unit 130 can adjust the threshold according to information concerning a surrounding environment. For example, weather, temperature, humidity, the geology of a traveling surface, and the like correspond to the information concerning the surrounding environment. Note that the information concerning the work machine 10 and the information concerning the surrounding environment can be acquired from an external device, for example, a cloud server.
Angle Obstacle Region
[0052]
Height Obstacle Region Detection Processing
[0053]
[0054] First, the region height detection unit 150 generates a height map (Step S141). This height map is an array representing the height of each region. The array of the coordinates of the y axis explained with reference to
[0055] Subsequently, the height obstacle region detection unit 170 detects a high position obstacle region (Step S142). Specifically, the height obstacle region detection unit 170 detects a region at a position higher than a high position threshold generated by the threshold generation unit 160 as the high position obstacle region. The height obstacle region detection unit 170 detects a low position obstacle region (Step S143). Specifically, the height obstacle region detection unit 170 detects a region at a position lower than a low position threshold generated by the threshold generation unit 160 as the low position obstacle region. The threshold generation unit 130 can output predetermined values, for example, a high position threshold 0.5 m and a low position threshold 0.5 m as the thresholds. As explained above, by separately detecting the high position obstacle region and the low position obstacle region, different kinds of processing can be respectively performed for the high position obstacle region and the low position obstacle region.
[0056] Note that the threshold generation unit 160 can adjust the thresholds according to information concerning the work machine 10. For example, the size of the work machine 10 corresponds to the information concerning the work machine 10.
Height Obstacle Region
[0057]
Traveling Obstacle Region Detection Unit
[0058]
[0059] The exception region detection unit 181 detects an exception region of the angle obstacle region and the height obstacle region. The angle obstacle region and the height obstacle region other than the exception region detected by the exception region detection unit 181 are traveling obstacle regions.
[0060] The object detection unit 182 detects an object placed on a traveling surface. The object detection unit 182 detects an object that obstructs traveling such as a person or another work machine 10.
[0061] The occlusion region detection unit 183 detects an occlusion region. This occlusion region is a region where an angle and height on the traveling surface cannot be detected. For example, a region hidden by a cliff or the like corresponds to the occlusion region. By detecting this occlusion region as an obstacle region, it is possible to improve the safety of the work machine 10.
Traveling Obstacle Region Detection Processing
[0062]
[0063] Subsequently, the exception region detection unit 181 detects an exception region of a height obstacle region (Step S152). For example, a region having width smaller than a predetermined threshold corresponds to the exception region. This is because the groove 26 or the like having relatively narrow width can be climbed over. The detected exception region is excluded from the height obstacle region.
[0064] Subsequently, the object detection unit 182 detects an object (Step S153). A region of the detected object is added to the traveling obstacle region.
[0065] Subsequently, the occlusion region detection unit 183 detects an occlusion region (Step S154). The detected occlusion region is added to the traveling obstacle region.
Occlusion Region
[0066]
Traveling Obstacle Detection Processing
[0067]
[0068] As explained above, the traveling obstacle region detection system 100 of the present disclosure detects the traveling obstacle region based on the angle and the height of the traveling surface. Therefore, it is possible to improve the ability of detecting an obstacle region.
[0069] Note that the traveling obstacle region detection system 100 of the present disclosure can be applied to, for example, a general vehicle, a construction machine, an agricultural machine, a forestry machine, a disaster rescue machine, a home machine for nursing care and cleaning, a seabed probe, and a planet probe besides a work device at a construction site.
Other Modifications
[0070] The traveling obstacle region detection system 100 in the present embodiment may be implemented by a dedicated computer system or may be implemented by a general-purpose computer system.
[0071] For example, a program for executing the operation explained above is stored in a computer-readable recording medium such as an optical disk, a semiconductor memory, a magnetic tape, or a flexible disk and distributed. Then, for example, the control device is configured by installing the program in a computer and executing the processing explained above.
[0072] The communication program explained above may be stored in a disk device included in a server device on a network such as the Internet to make it possible to download the communication program to a computer. The functions explained above may be implemented by cooperation of an OS (Operating System) and application software. In this case, a portion other than the OS may be stored in a medium and distributed or the portion other than the OS may be stored in the server device to make it possible to download the portion to the computer.
[0073] Among the kinds of processing explained in the embodiment, all or a part of the processing explained as being automatically performed can be manually performed or all or a part of the processing explained as being manually performed can be automatically performed by a publicly-known method. Besides, the processing procedures, the specific names, and the information including the various data and parameters explained in the document and illustrated in the figures can be optionally changed except when specifically noted otherwise. For example, the various kinds of information illustrated in the figures are not limited to the illustrated information.
[0074] The illustrated components of the devices are functionally conceptual and are not always required to be physically configured as illustrated in the figures. That is, specific forms of distribution and integration of the devices are not limited to the illustrated forms and all or a part thereof can be functionally or physically distributed and integrated in any unit according to various loads, usage situations, and the like. Note that this configuration by the distribution and the integration may be dynamically performed.
[0075] The embodiments explained above can be combined as appropriate in a range for not causing processing contents to contradict one another. The order of the steps illustrated the flowchart in the embodiment explained above can be changed as appropriate.
[0076] For example, the present embodiments can be implemented as any component configuring a device or a system, for example, a processor functioning as a system LSI (Large Scale Integration) or the like, a module that uses a plurality of processors or the like, a unit that uses a plurality of modules or the like, or a set obtained by further adding other functions to the unit (that is, a component as a part of the device).
[0077] Note that, in the present embodiments, the system means a set of a plurality of components (devices, modules (components), and the like). It does not matter whether all the components are present in the same housing. Therefore, both of a plurality of devices housed in separate housings and connected via a network and one device in which a plurality of modules is housed in one housing are systems.
[0078] For example, the present embodiment can adopt a configuration of cloud computing in which one function is shared and processed by a plurality of devices in cooperation via a network.
[0079] Although the embodiments of the present disclosure are explained above, the technical scope of the present disclosure is not limited to the embodiments per se. Various changes can be made without departing from the gist of the present disclosure. Components in different embodiments and modifications may be combined as appropriate.
[0080] The processing procedure explained in the embodiments may be regarded as a method including these series of procedures and may be regarded as a program for causing a computer to execute these series of procedures or a recording medium storing the program. As this recording medium, for example, a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magnet optical) disk, a DVD (Digital Versatile Disc), a Blu-ray (registered trademark) disc, a magnetic disk, a semiconductor memory, a memory card, and the like can be used.
[0081] Note that the effects described in this specification are only illustrations and are not limited. Other effects may be present.
[0082] Note that this technology can also take the following configurations.
[0083] (1) A system that detects a traveling obstacle region that is a region that obstructs traveling of a work machine, the system comprising: [0084] a region angle detection unit that detects an angle of a traveling surface based on a distance image around the work machine; [0085] a region height detection unit that detects a height of the traveling surface based on the distance image; [0086] an angle obstacle region detection unit that detects an angle obstacle region that is an obstacle region based on the detected angle; [0087] a height obstacle region detection unit that detects a height obstacle region that is an obstacle region based on the detected height; and [0088] a traveling obstacle region detection unit that detects the traveling obstacle region based on the detected angle obstacle region and the detected height obstacle region.
[0089] (2) The system according to the above (1), wherein the angle obstacle region detection unit detects the angle obstacle region based on a threshold of the angle.
[0090] (3) The system according to the above (2), wherein the angle obstacle region detection unit detects the angle obstacle region based on the threshold adjusted according to information concerning the work machine.
[0091] (4) The system according to the above (2), wherein the angle obstacle region detection unit detects the angle obstacle region based on the threshold adjusted according to information concerning a surrounding environment.
[0092] (5) The system according to any one of the above (1) to (4), wherein the height obstacle region detection unit detects the height obstacle region based on a threshold of the height.
[0093] (6) The system according to any one of the above (1) to (5), wherein the traveling obstacle region detection unit detects the traveling obstacle region by excluding a region having a predetermined height in the detected angle obstacle region.
[0094] (7) The system according to any one of the above (1) to (6), wherein the traveling obstacle region detection unit detects the traveling obstacle region by excluding a region having a predetermined width in the detected height obstacle region.
[0095] (8) The system according to any one of the above (1) to (7), further comprising [0096] an object detection unit that detects an object placed on the traveling surface, wherein [0097] the traveling obstacle region detection unit further detects a region of the detected object as the traveling obstacle region.
[0098] (9) The system according to any one of the above (1) to (8), further comprising [0099] an occlusion region detection unit that detects an occlusion region that is a region where the angle and the height on the traveling surface cannot be detected, wherein [0100] the traveling obstacle region detection unit further detects the detected occlusion region as the traveling obstacle region.
[0101] (10) A program for detecting a traveling obstacle region that is a region that obstructs traveling of a work machine, the program comprising: [0102] a region angle detection procedure of detecting an angle of a traveling surface based on a distance image around the work machine; [0103] a region height detection procedure of detecting height of the traveling surface based on the distance image; [0104] an angle obstacle region detection procedure of detecting an angle obstacle region that is an obstacle region based on the detected angle; [0105] a height obstacle region detection procedure of detecting a height obstacle region that is an obstacle region based on the detected height; and [0106] a traveling obstacle region detection procedure of detecting the traveling obstacle region based on the detected angle obstacle region and the detected height obstacle region.
REFERENCE SIGNS LIST
[0107] 10 WORK MACHINE [0108] 12 SENSOR [0109] 100 TRAVELING OBSTACLE REGION DETECTION SYSTEM [0110] 110 DATA CONVERSION UNIT [0111] 120 REGION ANGLE DETECTION UNIT [0112] 130 THRESHOLD GENERATION UNIT [0113] 140 ANGLE OBSTACLE REGION DETECTION UNIT [0114] 150 REGION HEIGHT DETECTION UNIT [0115] 160 THRESHOLD GENERATION UNIT [0116] 170 HEIGHT OBSTACLE REGION DETECTION UNIT [0117] 180 TRAVELING OBSTACLE REGION DETECTION UNIT [0118] 181 EXCEPTION REGION DETECTION UNIT [0119] 182 OBJECT DETECTION UNIT [0120] 183 OCCLUSION REGION DETECTION UNIT