OBJECT DETECTION PROBABILITY CALCULATING APPARATUS, OBJECT DETECTION PROBABILITY CALCULATING METHOD, AND PROGRAM

Abstract

An object detection probability calculating apparatus includes a processor; and a memory storing instructions that cause the processor to execute a process. The process includes calculating an object detection probability indicating a probability at which a person is able to detect a predetermined object appearing in an image, based on a parameter of image quality acquired from a camera provided at a moving body capable of automatic movement and distance information indicating a distance from the moving body or the camera to the predetermined object.

Claims

1. An object detection probability calculating apparatus, comprising: a processor; and a memory storing instructions that cause the processor to execute a process, the process including calculating an object detection probability indicating a probability at which a person is able to detect a predetermined object appearing in an image, based on a parameter of image quality acquired from a camera provided at a moving body capable of automatic movement and distance information indicating a distance from the moving body or the camera to the predetermined object.

2. The object detection probability calculating apparatus according to claim 1, wherein the object detection probability is calculated based on the parameter and the distance information by using a model of the object detection probability, the model indicating a relationship between a value of the parameter and the object detection probability.

3. The object detection probability calculating apparatus according to claim 2, wherein the model is a model in which the object detection probability increases as the parameter increases, and the object detection probability does not change when the parameter is equal to or greater than a first predetermined value; in which the object detection probability increases in a case where the distance is equal to or smaller than a second predetermined value, and the object detection probability is equal to or greater than a certain value even in a case where the parameter is equal to or smaller than a third predetermined value; and in which the object detection probability decreases as the distance increases, and the object detection probability is 0 in a case where the distance is equal to or greater than a fourth predetermined value.

4. The object detection probability calculating apparatus according to claim 1, wherein the parameter is a bit rate, a resolution, or a frame rate related to the image.

5. The object detection probability calculating apparatus according to claim 1, wherein the process further includes transmitting object detection probability information indicating the calculated object detection probability, to an operator terminal for remotely monitoring movement of the moving body.

6. The object detection probability calculating apparatus according to claim 1, wherein the object detection probability calculating apparatus is installed at the moving body or installed in a monitoring center in which an operator terminal for remotely monitoring movement of the moving body is installed.

7. An object detection probability calculating method comprising: calculating an object detection probability indicating a probability at which a person is able to detect a predetermined object appearing in an image, based on a parameter of image quality acquired from a camera provided at a moving body capable of automatic movement and distance information indicating a distance from the moving body or the camera to the predetermined object.

8. A non-transitory computer-readable recording medium having computer-readable instructions stored thereon, which when executed, cause a computer to perform the object detection probability calculating method according to claim 7.

Description

BRIEF DESCRIPTION OF DRAWINGS

[0014] FIG. 1 is an overall configuration diagram of a communication system according to an embodiment.

[0015] FIG. 2 is a diagram illustrating an example of a model of an object detection probability estimated by using a bit rate of an image and a distance to an object.

[0016] FIG. 3 is an electrical hardware configuration diagram of each device and a terminal according to the embodiment.

[0017] FIG. 4 is an overall configuration diagram of a communication system in a modification example according to the embodiment.

[0018] FIG. 5 is a flowchart illustrating processing of estimating and presenting the object detection probability.

[0019] FIG. 6 is a view illustrating a display screen example by an operator terminal in a monitoring center.

DESCRIPTION OF EMBODIMENTS

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

System Configuration of Embodiment

[0021] An outline of a configuration of a communication system according to an embodiment will be described with reference to FIG. 1. FIG. 1 is an overall configuration diagram of the communication system according to the embodiment. Note that values (1.2 Mbps, 20 m, 70%) illustrated in FIG. 1 are an example, and the embodiment is not limited to these values.

[0022] As illustrated in FIG. 1, a communication system 1 is constructed by a camera 3 and each device (object distance setting device 2, object detection probability calculating apparatus 5, and communication device 7), which are mounted on a moving body A such as a vehicle, and an operator terminal 8 placed on a remote monitoring center B side.

[0023] Basically, the moving body A performs automated driving, but when the operator terminal 8 in the monitoring center B displays images from a plurality of moving bodies as illustrated in FIG. 6, an operator in the monitoring center B can monitor the moving situation of each automobile, and can remotely operate a specific moving body according to the moving situation. As a result, the operator can remotely avoid an accident in a case where an object or a person that hinders driving jumps out at the time of viewing an image from a predetermined moving body.

[0024] Note that a plurality of the object distance setting devices 2, a plurality of the cameras 3, a plurality of the object detection probability calculating apparatuses 5, a plurality of the communication devices 7, and a plurality of the operator terminals 8 may be provided. In particular, four or more cameras 3 may be provided to capture images of the front side, the right side surface, the left side surface, and the rear side of the moving body A.

[0025] In the present embodiment, the moving body A is assumed to be, for example, a vehicle (automobile) having a communication function and an automated driving function. However, the moving body A is not limited to the automobile, and may be an agricultural machine such as a tractor having the communication function and the automated driving function. In addition to the vehicle, the moving body may be a ship, an aircraft, or the like.

[0026] Among the components, the object distance setting device 2 is a device that sets distance information used for the object detection probability calculating apparatus 5 to calculate an object detection probability. The distance information indicates a distance between the moving body A (or the camera 3) and an object, and is, for example, a predetermined fixed value such as 10 m, 20 m, or 30 m. However, the distance information may have a value designated by the operator in the monitoring center B, or may have a value set by the moving situation of the moving body A or the like.

[0027] The camera 3 is a monitoring camera or the like mounted on the moving body A. The camera 3 that captures an image of the outside of the moving body transmits image data obtained by image capturing to the operator terminal 8 in the monitoring center B via the communication device 7. Further, the camera 3 extracts a bit rate of the image from encoding information of the camera, and transmits bit rate information to the object detection probability calculating apparatus 5.

[0028] The object detection probability calculating apparatus 5 calculates the object detection probability based on the bit rate information acquired from the camera 3 and the distance information acquired from the object distance setting device 2 by using a model of the object detection probability as illustrated in FIG. 2. The object detection probability indicates a probability at which a person (healthy person) can detect a predetermined object appearing in an image. Then, the object detection probability calculating apparatus 5 transmits object detection probability information indicating the calculated object detection probability to the operator terminal 8 in the monitoring center B via the communication device 7.

[0029] FIG. 2 is a diagram illustrating an example of the model of the object detection probability estimated by using the bit rate of an image and a distance to an object. The horizontal axis of the graph represents the bit rate of the monitoring image, and the vertical axis represents the detection probability of the object, and a quality estimation graph is created for each distance between the moving body A (or the camera 3) and the object. In a case where the distance between the moving body A (or the camera 3) and the object is short, the object appears large, and thus the detection probability is increased. In addition, the detection probability is increased as the input image bit rate becomes higher.

[0030] Note that a detailed calculation method of the object detection probability performed by the object detection probability calculating apparatus 5 will be described later.

[0031] The communication device 7 can perform data communication with the operator terminal 8 via a communication network 100. The communication network 100 is constructed with a mobile network, the Internet, or the like. In addition, the Internet includes the space Internet through a space using an artificial satellite or the like.

[0032] The operator terminal 8 is a terminal such as a personal computer (PC) used by the operator in the monitoring center B.

[0033] With the configuration described above, the bit rate information transmitted from the camera 3 is transmitted to the object detection probability calculating apparatus 5. The distance information transmitted from the object distance setting device 2 is also transmitted to the object detection probability calculating apparatus 5. Further, the image data transmitted from the camera 3 is transmitted to the operator terminal 8 via the communication device 7. The object detection probability information indicating the object detection probability calculated by the object detection probability calculating apparatus 5 is transmitted from the object detection probability calculating apparatus 5 and transmitted to the operator terminal 8 via the communication device 7.

[0034] Note that the object distance setting device 2 may be provided on the monitoring center B side. In this case, the distance information transmitted from the object distance setting device 2 is transmitted to the object detection probability calculating apparatus 5 via a new communication device installed on the monitoring center B side and then the communication device 7 on the moving body A side.

Hardware Configuration

[0035] Next, electrical hardware configurations of the object distance setting device 2, the object detection probability calculating apparatus 5, the communication device 7, and the operator terminal 8 will be described with reference to FIG. 3. FIG. 3 is an electrical hardware configuration diagram of each device and the terminal according to the embodiment.

[0036] As illustrated in FIG. 3, the object detection probability calculating apparatus 5 includes, as a computer, a central processing unit (CPU) 101, a read only memory (ROM) 102, a random access memory (RAM) 103, a solid state drive (SSD) 104, an external device connection interface (I/F) 105, a network I/F 106, a medium I/F 109, and a bus line 110.

[0037] Among the components, the CPU 101 controls the overall operation of the object detection probability calculating apparatus 5. The ROM 102 stores a program such as an initial program loader (IPL) used for driving the CPU 101. The RAM 103 is used as a work area of the CPU 101.

[0038] The SSD 104 reads or writes various types of data under the control of the CPU 101. Note that a hard disk drive (HDD) may be used instead of the SSD 104.

[0039] The external device connection I/F 105 is an interface for connection with various external devices. Examples of the external devices in this case include a display, a speaker, a keyboard, a mouse, a universal serial bus (USB) memory, a printer, and the like.

[0040] The network I/F 106 is an interface for performing data communication via a communication network 100 or a Local area network (LAN). Note that a wireless communication device may be used instead of the network I/F 106.

[0041] The medium I/F 109 controls reading or writing (storing) of data from or in a recording medium 109m such as a flash memory. Examples of the recording medium 109m also include a digital versatile disc (DVD) and a Blu-ray Disc (registered trademark).

[0042] The bus line 110 is an address bus, a data bus, or the like for electrical connection with components such as the CPU 101 illustrated in FIG. 3.

[0043] Note that the object distance setting device 2, the communication device 7, and the operator terminal 8 have the same configurations as the configuration of the object detection probability calculating apparatus 5, and thus the description thereof will be omitted.

Modification Example of System Configuration of Embodiment

[0044] Here, a modification example of FIG. 1 will be described with reference to FIG. 4. Note that values (1.2 Mbps, 20 m, 70%) illustrated in FIG. 4 are an example, and the embodiment is not limited to the values. In the communication system 1 of FIG. 4, the object detection probability calculating apparatus 5 on the moving body A side is installed on the monitoring center B side, and accordingly, a communication device 9 is installed in the monitoring center B. The bit rate information transmitted from the camera 3 is transmitted to the object detection probability calculating apparatus 5 via the communication device 7 and the communication device 9. The distance information transmitted from the object distance setting device 2 is also transmitted to the object detection probability calculating apparatus 5 via the communication device 7 and the communication device 9. Further, the image data transmitted from the camera 3 is transmitted to the operator terminal 8 via the communication device 7. Note that the object distance setting device 2 may be provided on the monitoring center B side.

Processing or Operation of Communication System

[0045] Subsequently, processing or an operation of each device in the communication system 1 will be described with reference to FIGS. 5 and 6. FIG. 5 is a flowchart illustrating processing of estimating and presenting the object detection probability. FIG. 6 is a view illustrating a display screen example by the operator terminal in the monitoring center.

[0046] S11: The camera 3 captures an image of the outside of the moving body A to obtain image data, and then transmits the image data to the operator terminal.

[0047] S12: The camera 3 extracts the bit rate of the image by using encoding information of the image data, and transmits the bit rate information (for example, 1.2 Mbps) to the object detection probability calculating apparatus 5.

[0048] S13: The object detection probability calculating apparatus 5 calculates the object detection probability (for example, 70%) based on the bit rate information and the distance information (for example, 20 m) by using the model of the object detection probability as illustrated in FIG. 2.

[0049] S14: The object detection probability calculating apparatus 5 transmits object detection probability information indicating the object detection probability calculated by the object detection probability calculating apparatus 5 to the operator terminal 8.

[0050] As a result, as illustrated in FIG. 6, the operator terminal 8 displays, on a display, the image transmitted from each moving body in S11 and the object detection probability information related to the image of each moving body, which has been transmitted from the object detection probability calculating apparatus 5 in S14, in association with each other by screen division. Note that, in order to associate the image and the object detection probability information with each other, identification information such as an ID may be included in each of the image data and the object detection probability information. In this case, the camera 3 adds the identification information to the image data and the bit rate information to be output. Then, the object detection probability calculating apparatus 5 adds the identification information added to the bit rate information acquired from the camera 3 to the object detection probability information and outputs the result of the addition.

[0051] S15: In a case where the movement of the moving body A is not ended (NO), the process returns to S11. On the other hand, in a case where the movement of the moving body A is ended (YES), the processing illustrated in FIG. 5 is ended.

Detailed Calculation Method of Object Detection Probability

[0052] Next, the calculation method of the object detection probability performed by the object detection probability calculating apparatus 5 in the process of S13 will be described. Here, a method of obtaining an object detection probability dr using a bit rate br of an image and a distance d between the moving body A (or the camera 3) and the object will be described.

[0053] Image quality is correlated with the bit rate. The higher the bit rate, the higher the image quality. On the other hand, in a case where the bit rate reaches a predetermined value or greater, the quality of the image reaches almost an upper limit, and the object detection probability does not change even at a bit rate higher than the predetermined bit rate. Therefore, the object detection probability is calculated by using the model of the object detection probability in which the object detection probability increases as the bit rate increases, but the detection probability does not change when the bit rate is equal to or greater than a predetermined value (first predetermined value). In addition, calculation is performed by using the model of the object detection probability in which the object detection probability increases in a case where the distance between the moving body A (or the camera 3) and the object is equal to or smaller than a predetermined value (second predetermined value) (in a case where the distance is relatively short), and the detection probability is equal to or greater than a predetermined value even in a case where the bit rate is equal to or smaller than a predetermined value (third predetermined value) (in a case where the bit rate is relatively low). On the other hand, calculation is performed by using the model of the object detection probability in which the object detection probability decreases as the distance between the moving body A (or the camera 3) and the object increases, and the object detection probability is 0 in a case where the distance is equal to or greater than a predetermined value (fourth predetermined value) (in a case where the distance is too long). Using a mathematical expression satisfying these conditions, the object detection probability is calculated by any one of the following mathematical expressions. Note that the detection probability is defined in a range of 0 to 1, and, in each of the following expressions, the object detection probability is defined as 0 in a case where the object detection probability dr is equal to or smaller than 0, and the object detection probability is defined as 1 in a case where dr is equal to or greater than 1. In each expression, a to as are coefficients different for each expression, and e represents the base of the natural logarithm. Note that the expression used for calculating the object detection probability is not limited to the following expressions.

[00001] $(Expression 1)$ $\begin{matrix} d r = \frac{(a_{1} - a_{2})}{1 + \exp \frac{b r - a_{3}}{a_{4}}} + a_{2} - a_{5} e^{d} & [Math . 1] \end{matrix}$ $(Expression 2)$ $\begin{matrix} dr = a_{1} (1 - b^{(b r - a_{2})}) + a_{3} d & [Math . 2] \end{matrix}$ $(Expression 3)$ $\begin{matrix} dr = a_{1} + a_{2} (1 - \exp (- \frac{b r}{a_{3} d})) + a_{4} (1 - \exp (- \frac{b r}{a_{5} d})) & [Math . 3] \end{matrix}$ $(Expression 4)$ $\begin{matrix} dr = - (a_{1} + a_{2} \exp (- \frac{b r}{a_{3} d})) & [Math . 4] \end{matrix}$ $(Expression 5)$ $\begin{matrix} dr = a_{1} + \frac{(a_{2} - a_{1}) {(b r - a_{3} d)}^{a_{4}}}{a_{5}^{a_{4}} + {(b r - a_{3} d)}^{a_{4}}} & [Math . 5] \end{matrix}$ $(Expression 6)$ $\begin{matrix} dr = ((1 - a_{3} e^{d}) - a_{1}) * \frac{({br}^{a_{2}})}{a_{4}^{a_{2}} + (b r^{a_{2}})} & [Math . 6] \end{matrix}$ $(Expression 7)$ $\begin{matrix} dr = (1 - a_{3} e^{d}) - \frac{(1 - a_{3} e^{d}) - a_{4}}{{(1 + \frac{b r}{a_{1} e^{d}})}^{a_{2}}} & [Math . 7] \end{matrix}$ $(Expression 8)$ $\begin{matrix} dr = \frac{a_{1}}{1 + \exp A (d - x_{0})} & [Math . 8] \end{matrix}$

[0054] However, A and x.sub.0 in Expression 8 are expressed as follows.

[00002] $A = - c_{1} \log (c_{2} + b r) + c_{3}$ $x_{0} = b_{1} \log (b_{2} + b r) + b_{3}$

[0055] Alternatively, A may be expressed as follows.

[00003] $A = c_{1} \exp (- (c_{2} x_{0}) - c_{3}) + c_{4}$

[0056] Here, a.sub.1, b.sub.1 to b.sub.3, and c.sub.1 to c.sub.4 in (Expression 8) are coefficients, respectively.

Effects of Embodiment

[0057] As described above, according to the present embodiment, it is possible to obtain and present the detection probability of the object by the operator in the monitoring center B. As a result, since the monitoring image from each moving body and the object detection probability information of this monitoring image are displayed in association with each other on the operator terminal 8 in the monitoring center B as illustrated in FIG. 6, it is possible to eliminate an occurrence of a situation in which the operator does not notice that the image quality of the monitoring image is deteriorated.

Supplementary Notes

[0058] As described above, the present invention is not limited to the above-described embodiment, and various modifications and applications can be made, for example, as will be described below.

[0059] (1) In the above embodiment, the camera 3 outputs the bit rate information, and the object detection probability calculating apparatus 5 calculates the object detection probability by using the bit rate information, but the present invention is not limited thereto. For example, instead of the bit rate information, the camera 3 may output resolution information indicating the resolution of the image or frame rate information indicating the frame rate of the image, and the object detection probability calculating apparatus 5 may calculate the object detection probability by using the resolution information or the frame rate information. Note that the bit rate information, the resolution information, and the frame rate information are examples of a parameter of the image quality.

[0060] (2) Although each device can be implemented by a computer and a program, the program can also be provided by being recorded in a (non-transitory) recording medium or via the communication network such as the Internet.

[0061] (3) The number of CPUs 101 is not limited to one, and may be plural.

REFERENCE SIGNS LIST

[0062] 1 Communication system [0063] A Moving body [0064] B Monitoring center [0065] 2 Object distance setting device [0066] 3 Camera (example of image capturing unit) [0067] 5 Object detection probability calculating apparatus [0068] 7 Communication device [0069] 8 Operator terminal [0070] 9 Communication device [0071] 100 Communication network

OBJECT DETECTION PROBABILITY CALCULATING APPARATUS, OBJECT DETECTION PROBABILITY CALCULATING METHOD, AND PROGRAM

Inventors

Cpc classification

Classification Explorer

G05D2105/22

PHYSICS

Classification Explorer

G06V20/58

PHYSICS

Classification Explorer

G06V10/98

PHYSICS

Classification Explorer

G05D1/2247

PHYSICS

International classification

Classification Explorer

G05D1/224

PHYSICS

Classification Explorer

G06V10/98

PHYSICS

Classification Explorer

G06V20/58

PHYSICS

Abstract

Claims

Description