Pedestrian detecting system
09727780 · 2017-08-08
Assignee
Inventors
- Kuo-Ching Chang (Changhua County, TW)
- Yu-Sheng Liao (Changhua County, TW)
- Pang-Ting Huang (Taipei, TW)
- Yi-Ming Chan (Taoyuan County, TW)
- Li-Chen Fu (Taipei, TW)
- Pei-Yung Hsiao (Taipei, TW)
Cpc classification
G01B11/14
PHYSICS
G06V20/58
PHYSICS
G06V10/467
PHYSICS
G06V10/60
PHYSICS
G06V10/50
PHYSICS
International classification
H04N7/18
ELECTRICITY
G01B11/14
PHYSICS
Abstract
A pedestrian detecting system includes a depth capturing unit, an image capturing unit and a composite processing unit. The depth capturing unit is configured to detect and obtain spatial information of a target object. The image capturing unit is configured to capture an image of the target object and recognize the image, thereby obtaining image feature information of the target object. The composite processing unit is electrically connected to the depth capturing unit and the image capturing unit, wherein the composite processing unit is configured to receive the spatial information and the image feature information and to perform a scoring scheme to detect and determine if the target object is a pedestrian. The scoring scheme performs weighted scoring on a spatial confidence and an appearance confidence to obtain a composite scoring value to determine if the target object is the pedestrian.
Claims
1. A pedestrian detecting system, comprising: a radar detecting and obtaining spatial information of a target object; a camera capturing an image of the target object and recognizing the image, thereby obtaining image feature information of the target object; and a computer electrically connected to the radar and the camera, wherein the computer receives the spatial information and the image feature information and performs a scoring scheme to detect and determine if the target object is a pedestrian; wherein an appearance confidence is obtained through data transformation of the image feature information, and a spatial confidence is obtained through data transformation of the spatial information, and the scoring scheme performs weighted scoring on the spatial confidence and the appearance confidence to obtain a composite scoring value to determine if the target object is the pedestrian; wherein the appearance confidence is calculated using the following equations:
P.sub.spatial(B)=P.sub.associate(c.sub.I,c.sub.R*); where P.sub.associate is a probability corresponding to each coupled position (C.sub.I, C.sub.R), and C.sub.R* is a best matching that has a highest probability to each position C.sub.I; and the scoring scheme is calculated using the following equation:
P.sub.fusion(B)=(1−w.sub.r)×P.sub.appearance(B)+w.sub.r×P.sub.spatial(B); where Wr is a weighted ratio.
2. The pedestrian detecting system of claim 1, further comprising an image processing apparatus, wherein the image processing apparatus obtains an intermediately-illuminated image region of the target object recognized by the camera, and retains the image feature information of the intermediately-illuminated image region, wherein an illuminance of the intermediately-illuminated image region is from 10 to 80000 lux.
3. The pedestrian detecting system of claim 2, wherein the image processing apparatus obtains the intermediately-illuminated image region by using a Histogram of Oriented Gradient (HOG) and a Logarithm Weighted Pattern (LWP).
4. The pedestrian detecting system of claim 3, wherein the operation of the Logarithm Weighted Pattern comprises capturing an image intensity variance of the target object, performing a logarithm operation and obtaining a portion of the image feature information.
5. The pedestrian detecting system of claim 1, wherein the spatial information comprises a coordinate of the target object and a distance between the target object and a vehicle.
6. The pedestrian detecting system of claim 5, wherein the scoring scheme uses a Gaussian function to obtain a matching probability between the image of the target object and the distance, thereby obtaining the spatial confidence.
7. A pedestrian detecting system, comprising: a radar detecting and obtaining spatial information of a target object; a camera capturing an image of the target object and recognizing the image, thereby obtaining image feature information of the target object; an image processing apparatus obtaining an intermediately-illuminated image region of the target object recognized by the camera according to a relative luminance intensity of the image of the target object, and retaining the image feature information of the intermediately-illuminated image region, wherein an illuminance of the intermediately-illuminated image region is from 10 to 80000 lux; and a computer electrically connected to the radar and the camera, wherein the computer receives the spatial information and the image feature information, and performs a scoring scheme to detect and determine if the target object is a pedestrian; wherein an appearance confidence is obtained through data transformation of the image feature information, and a spatial confidence is obtained through data transformation of the spatial information, and the scoring scheme performs weighted scoring on the spatial confidence and the appearance confidence to obtain a composite scoring value to determine if the target object is the pedestrian; wherein the appearance confidence is calculated using the following equations:
P.sub.spatial(B)=P.sub.associate(c.sub.I,c.sub.R*); where P.sub.associate is a probability corresponding to each coupled position (C.sub.I, C.sub.R), and C.sub.R* is a best matching that has a highest probability to each position C.sub.I; and the scoring scheme is calculated using the following equation:
P.sub.fusion(B)=(1−w.sub.r)×P.sub.appearance(B)+w.sub.r×P.sub.spatial(B); where Wr is a weighted ratio.
8. The pedestrian detecting system of claim 7, wherein the intermediately-illuminated image region of the target object is obtained under a luminance greater than 10 lux.
9. The pedestrian detecting system of claim 7, wherein the image processing apparatus obtains the intermediately-illuminated image region through a Histogram of Oriented Gradient (HOG) and a Logarithm Weighted Pattern (LWP).
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) The present disclosure can be more fully understood by reading the following detailed description of the embodiment, with reference made to the accompanying drawings as follows:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
DETAILED DESCRIPTION
(12) Reference will now be made in detail to the present embodiments of the disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the description to refer to the same or like parts.
(13) Simultaneously referring to
(14) In
(15) In
(16) An image of the target object is captured by the image capturing unit 120, and is recognized to obtain image feature information 123. Operations of a Histogram of Oriented Gradient (HOG) 121 and a Logarithm Weighted Pattern (LWP) 122 are applied to obtain the image feature information 123 that are actually required.
(17) Through the composite processing unit 130, a data correlating process 131 and a scoring scheme 132 are performed on the spatial information 111 and the image feature information 123, and then a final detecting result is obtained.
(18) The aforementioned scoring scheme 132 performs weighted scoring on a spatial confidence and an appearance confidence to obtain a composite scoring value to determine if the target object is the pedestrian, thereby enabling a driver to react instantly to changes of the surrounding environment.
(19) In an embodiment,
(20) In the following embodiments, more details are described regarding how to use the pedestrian detecting system 100 to detect and determine if the target object is a real pedestrian. In the pedestrian detecting system 200, most of the components and operating processes are similar to those of the pedestrian detecting system 100. Main differences between the pedestrian detecting system 100 and the pedestrian detecting system 200 will also be described.
(21) Referring to
(22) Referring to
(23) In
(24) In
(25) In
(26) Under an insufficiently-illuminated environment, the image captured by the image capturing unit 120 will have different luminance levels. In this situation, some over-illuminated conditions or insufficiently-illuminated conditions will lead to overlarge voting values of some specified feature values, thus generating abnormal bins. The abnormal bins will cause the inaccurate image feature information 123.
(27) For solving the aforementioned problem caused by high contrast, the Logarithm Weighted Pattern 122 is introduced to reduce the overlarge voting values caused by the overlarge weight.
(28) The Logarithm Weighted Pattern 122 can be represented by the following formula:
P.sub.mag(x)=Σ.sub.i=0.sup.7|I.sub.y.sub.
(29) In the formula (1), w.sub.x represents a weight parameter in a point x, and I(.) represents intensity values corresponding to the point x and a point y.
(30) By using the logarithm function ln(P.sub.mag(x)+ε)|, the growth speed of the weight parameter can be suppressed. Therefore, the high contrast portion of the image under the insufficiently-illuminated environment can be restrained, and the abnormal problem of the image feature information caused by the overlarge weight can be reduced.
(31) The aforementioned embodiment shows that in the pedestrian detecting system 100, the correct image feature information 123 can be obtained by considering a shape feature extracted from the Histogram of Oriented Gradient (HOG), a material feature extracted from the Logarithm Weighted Pattern (LWP), and a luminance variance of the environment. In the following embodiments, how to collaborate the composite processing unit 130 with the depth capturing unit 110 for increasing the detecting accuracy of the pedestrian is shown. It is also shown how to use the dynamically illuminated object detector 240 of the pedestrian detecting system 200 to accurately detect the pedestrian even under the insufficiently-illuminated environment.
(32)
(33) For solving the aforementioned problem, the depth capturing unit 110 is introduced to obtain the depth information such as a distance between the pedestrian and the vehicle. Then, a composite processing unit 130 performs a data correlating process 131 to combine the spatial information 111 captured by the depth capturing unit 110 with the image feature information 123 captured by the image capturing unit 120.
(34) The depth capturing unit 110 can use a sensing device such as radar to detect the spatial information 111 of the pedestrian. For fusing the spatial information 111 and the image feature information 123, a data transformation process is first performed. In
(35)
(36) In the formula (2), (u.sub.p,v.sub.p) represents a position of the image point; K.sub.u and K.sub.v represent a horizontal focal length of the depth capturing unit 110 (e.g. a radar) and a vertical focal length of the depth capturing unit 110 respectively; D.sub.c represents a distance between the image capturing unit 120 (e.g. a camera) and the depth capturing unit 110; H.sub.c represents a height of the image capturing unit 120; and θ.sub.tilt and θ.sub.pan represents a tilt angle and a pan angle of the image capturing unit 120. After a coordinate transformation is performed according to formula (2), the subsequent process can be preceded.
(37) The aforementioned embodiments take the pedestrian detecting system 100 as an example; and similar processes can also be applied to the pedestrian detecting system 200. As mentioned previously, in the insufficiently-illuminated environment, the accuracy of detecting the pedestrian by the pedestrian detecting system 100 will be decreased. Therefore, in the pedestrian detecting system 200, the dynamically illuminated object detector 240 is introduced.
(38) The low-illuminated image region A1 is usually located in an upper-half portion of the target object. By dividing the target object into upper-half and lower-half portions, a boundary point can be defined by calculating a location that has a maximum difference between the average luminance of the two portions. The boundary of the low-illuminated image region A1 can be represented by the following formulas:
(39)
(40) In the formulas (3) and (4), p(u,v) represents a point on the detected image; and v.sub.PI represents a boundary point of low-illuminated, which classifies the target object into the low-illuminated image region A1 and the ideal-illuminated image region A2.
(41) The over-illuminated region A3 is usually located in a lower-half portion of the target object. An upper boundary of the over-illuminated region A3 is defined as an over-illuminated line. The boundary of the over-illuminated region A3 can be represented by the following formula:
(42)
(43) In the formula (5), v.sub.OE represents a boundary point of an over-illuminated region, which classifies the target object into the ideal-illuminated image region A2 and the over-illuminated region A3; d.sub.1 represents one dimensional magnitude of the image feature vector of the HOG 121; d.sub.2 represents another dimensional magnitude of the image feature vector of the LWP 122; d.sub.B represents a start dimension of each image feature; and d.sub.E represents a terminal dimension of each image feature.
(44) As mentioned previously, the confidence of the detecting result is dependent on the data correlation between the feature vector and the training model. In more detail, the confidence is defined by an inner product of the data of the feature vector and the training model. Therefore, the scoring value can be calculated by the following formula:
(45)
(46) In the formula (6), W=(w.sub.1.sup.T,w.sub.2.sup.T).sup.T represents a vector corresponding to the data of the training model; X=(x.sub.1.sup.T,x.sub.2.sup.T).sup.T represents a feature vector; and s(X) represents the scoring value corresponding to the feature vector X.
(47)
(48) By using the formula (6), the scoring value s(X) corresponding to the confidence can be obtained. For obtaining a more accurate detecting result, the image feature information captured by the image capturing unit 120 and the spatial information captured by the depth capturing unit 110 should be taken into consideration simultaneously. Commonly, a point on the detected image is obtained by the depth capturing unit 110, and a series of detected images are obtained by the image capturing unit 120. For fusing the two different mechanisms, in
Δu=u.sub.I−u.sub.R
Δv=v.sub.I−v.sub.R (7)
(49)
G.sub.u(•)˜N(0,σ.sub.u.sup.2)
G.sub.v(•)˜N(0,σ.sub.v.sup.2) (8)
P.sub.associate(c.sub.I,c.sub.R)=G.sub.u(Δu)×G.sub.v(Δv) (9)
(50) In the formula (9), a probability P.sub.associate corresponding to each coupled position (C.sub.I, C.sub.R) can be obtained. In more detail, to each position C.sub.I, a best matching C.sub.R* that has the highest probability can be defined. Therefore, a spatial confidence P.sub.spatial(B) of the pedestrian image boundary region B relative to the position C.sub.I can be obtained by the following formula:
P.sub.spatial(B)=P.sub.associate(c.sub.I,c.sub.R*) (10)
(51) Furthermore, an appearance confidence P.sub.appearance(B) of the pedestrian image boundary region B can be obtained by the following formula:
(52)
(53) In the formula (11), s(X.sub.B) represents a scoring value corresponding to the feature vector X in the pedestrian image boundary region B. Thereafter, a composite P.sub.fusion(B) can be obtained by combining the formula (10) and the formula (11):
P.sub.fusion(B)=(1−w.sub.r)×P.sub.appearance(B)+w.sub.r×P.sub.spatial(B) (12)
(54) In the formula (12), the spatial confidence P.sub.spatial(B) can be adjusted by the appearance confidence P.sub.appearance(B). In more detail, the scoring scheme provided by the composite processing unit 130 performs weighted scoring to fuse the spatial confidence P.sub.spatial(B) and the appearance confidence P.sub.appearance(B), thereby obtaining a composite scoring value. According to the composite scoring value, the object of accurately detecting a pedestrian under the insufficiently-illuminated environment can be achieved.
(55) To sum up, a pedestrian detecting system is provided in the present disclosure. By fusing the depth information captured by the depth capturing unit and the image feature information captured by the image capturing unit, the pedestrian detecting system is capable of performing an accurate detection to determine if the detected target object is a pedestrian. Furthermore, the Logarithm Weighted Pattern collaborates with the dynamically illuminated object detector to obtain the ideal-illuminated image region of the target object, thereby reducing overlarge voting value caused by high contrast under the insufficiently-illuminated environment. Moreover, the appearance confidence and the spatial confidence are scored in a specified weight ratio, thereby obtaining a composite scoring value for increasing the accuracy of detecting the pedestrian.
(56) Although the present disclosure has been described in considerable detail with reference to certain embodiments thereof, other embodiments are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the embodiments contained herein.
(57) It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the present disclosure without departing from the scope or spirit of the disclosure. In view of the foregoing, it is intended that the present disclosure cover modifications and variations of this disclosure provided they fall within the scope of the following claims.