SYSTEM FOR DETERMINING TYPE OF PASSENGER

Abstract

A system capable of determining a type of a passenger sitting on a seat according to a body type is disclosed. The system includes a seat detecting means detecting a position of a seat and an angle of a seat back, a passenger detecting means detecting a passenger based on a state where a specific passenger normally sits on the seat, and a controller detecting a passenger's body part along with a seat-back area where the seat back is positioned and a passenger area where the passenger is positioned using information detected by the seat detecting means and the passenger detecting means, and learning a parameter value that is input as a relationship of detected information and a classification result for a body type of the passenger that is output by the parameter value, thus setting a body-type classification model.

Claims

1. A system for determining a type of a passenger, the system comprising: a seat detector configured to detect a position of a seat and an angle of a seat back; a passenger detector configured to detect a passenger based on a first seating location of a specific passenger; and a controller configured to: detect, based on information detected by the seat detector and the passenger detector, a body part of the passenger, a seat-back area where the seat back is positioned, and a passenger area where the passenger is positioned, and learn (i) a parameter value that is input as a relationship of detected information and (ii) a classification result for a body type of the passenger that is output by the parameter value, to thereby establish a body-type classification model.

2. The system of claim 1, wherein the seat detector includes a seat position sensor and is configured to detect the seat-back area in a shape of a box by tracking the position of the seat back using the seat position sensor according to a movement of the seat, and wherein the passenger detector includes an image sensor configured to capture an image of the passenger and is configured to detect the passenger area including a whole-body area of the passenger and a face area of the passenger captured by the image sensor in a shape of a box.

3. The system of claim 2, wherein the parameter value comprises: a length of a vertical torso line of the passenger, and a distance between an upper line of the whole-body area and an upper line of the seat-back area.

4. The system of claim 2, wherein the parameter value comprises: an angle defined between a horizontal shoulder line and a vertical torso line of the passenger, a distance between an upper line of the whole-body area and an upper line of the seat-back area, a distance between an upper line of the face area and the upper line of the seat-back area, a size and position of the seat-back area, and a minimum distance between a center of the horizontal shoulder line of the passenger and a side line of the seat-back area.

5. The system of claim 4, wherein the controller is configured to: based on an unspecified passenger being detected, determine whether the passenger is seated at the first seating location using the body-type classification model, and based on the passenger being in an abnormal sitting state, perform control to induce self-correction for normal sitting.

6. The system of claim 5, wherein, based on the unspecified passenger being in the abnormal sitting state, a currently photographed image is transmitted to an audio, video, navigation (AVN) system along with a warning message indicating that an airbag is not to be deployed.

7. The system of claim 5, wherein the controller is configured to, based on (i) a child restraint system (CRS) in the seat not being detected, (ii) a shoe area of the passenger being detected in a shape of a box, and (iii) a length of the shoulder line in the shoe area being equal to or more than a threshold value, determine that the passenger is in the abnormal sitting state.

8. The system of claim 7, wherein, based on a determination that the passenger in the abnormal sitting state, the length of the shoulder line in the shoe area is corrected to be less than a length shown in an actual image, and a corrected state is maintained until a posture of the passenger is changed to a normal sitting state.

9. The system of claim 5, wherein the controller is configured to, based on the shoulder line of the passenger being higher than the upper line of the seat-back area and a length of the shoulder line being less than a threshold value, determine that the passenger is in the abnormal sitting state.

10. The system of claim 9, wherein, based on a determination that the passenger is in the abnormal sitting state, the length of the shoulder line of the passenger and the size of the seat-back area are corrected to be less than a size shown in an actual image, and a corrected state is maintained until a posture of the passenger is changed to a normal sitting state.

11. The system of claim 5, wherein the controller is configured to, based on the shoulder line of the passenger being higher than the upper line of the seat-back area and a length of the shoulder line being equal to or more than a threshold value, determine that the passenger is in a normal sitting state.

12. The system of claim 5, wherein the controller is configured to, based on a center line of the seat-back area being higher than the upper line of the face area, determine that the passenger is in the abnormal sitting state.

13. The system of claim 12, wherein, based on a determination that the passenger is in the abnormal sitting state, the seat-back area and the face area are corrected to be larger than a size shown in an actual image, and a corrected state is maintained until a posture of the passenger is changed to a normal sitting state.

14. The system of claim 5, wherein the controller is configured to, based on a distance between the vertical torso line of the passenger and the side line of the seat-back area being less than a threshold vehicle, determine that the passenger is in the abnormal sitting state.

15. The system of claim 14, wherein, based on a determination that the passenger is in the abnormal sitting state, the distance between the torso line of the passenger and the side line of the seat-back area is corrected to be less than a distance shown in an actual image, and a corrected state is maintained until a posture of the passenger is changed to a normal sitting state.

16. The system of claim 2, wherein the controller is configure to, based on a blanket being detected by the image sensor, detect an object around the blanket to determine (i) whether the passenger is sitting and (ii) the type of the passenger.

17. The system of claim 16, wherein the controller is configured to, based on a CRS being detected with the blanket, determine that a size of a passenger is less than a predetermined size.

18. The system of claim 16, wherein, based on the blanket being detected, a function of detecting a size of the body part of the passenger is deactivated.

19. The system of claim 16, wherein the controller is configured to, based on the face of the passenger being detected with the blanket and an upper line of the seat-back area being higher than an upper line of the face area, determine that a size of a passenger is less than a predetermined size.

20. The system of claim 16, wherein the controller is configured to, based on only the blanket being detected, determine that the passenger does not sit, and a photographed image is transmitted to an AVN system along with a warning message indicating that an airbag is not to be deployed.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0029] FIG. 1 is a block diagram schematically showing an example of a passenger recognition system.

[0030] FIG. 2 is a diagram illustrating a state in which an example of an image sensor is installed.

[0031] FIG. 3 is a diagram illustrating an example of a seat position and a seat back angle acquired by a seat position sensor.

[0032] FIG. 4 is a diagram illustrating an example of an image in which seat information and passenger information are detected in a state where a seat is positioned in front.

[0033] FIG. 5 is a diagram illustrating an example of an image in which seat information and passenger information are detected in a state where a seat is positioned in the middle.

[0034] FIG. 6 is a diagram illustrating an example of an image in which seat back information and passenger information are detected in a state where a seat is positioned in the rear.

[0035] FIG. 7 is a diagram illustrating an example of an image in which seat information and passenger information are detected in a state in which a seat back is reclined backward.

[0036] FIG. 8 is a diagram illustrating an example of an image in which seat information and passenger information are detected in an abnormal sitting state in which a child passenger is standing on a seat.

[0037] FIG. 9 is a diagram illustrating an example of an image in which seat information and passenger information are detected in an abnormal sitting state in which a child passenger is lying on a seat.

[0038] FIG. 10 is a diagram illustrating an example of an image in which seat information and passenger information are detected in an abnormal seating state in which a child passenger is leaning forward on a seat.

[0039] FIG. 11 is a diagram illustrating an example of an image in which seat information and passenger information are detected in an abnormal seating state in which a child passenger is kneeling on a front of a seat.

[0040] FIG. 12 is a diagram illustrating an example of an image in which a CRS is detected along with a blanket.

[0041] FIG. 13 is a diagram illustrating an example of an image in which a passenger's face is detected along with a blanket.

[0042] FIG. 14 is a diagram illustrating an example of an image in which a seat back and a face are detected in a state where an adult passenger is fully reclined.

[0043] FIG. 15 is a diagram illustrating an example of an image in which only a blanket is detected.

DETAILED DESCRIPTION

[0044] A system for determining a type of a passenger according to the present disclosure includes a seat detector detecting a position a of a seat and an angle b of a seat back, a passenger detector detecting a passenger based on a state where a specific passenger normally sits on the seat, and a controller 300 detecting a passenger's body part along with a seat-back area A1 where the seat back is positioned and a passenger area where the passenger is positioned using information detected by the seat detector and the passenger detector, and learning a parameter value that is input as a relationship of detected information and a classification result for a body type of the passenger that is output by the parameter value, thus setting a body-type classification model 330.

[0045] Here, the seat detector may be a seat position sensor 100, and may detect the seat-back area A1 in a shape of a box by tracking the position of the seat back using the seat position sensor 100 according to a movement of the seat, and the passenger detector may be an image sensor 200 photographing the passenger, and detects the passenger area including a passenger's whole-body area A2 and a passenger's face area A3 photographed by the image sensor 200 in a shape of a box.

[0046] Main components of the passenger recognition system according to the present disclosure will be described with reference to FIGS. 1 to 3. The seat position sensor 100 may acquire information about a position a of the seat and an angle b of the seat back. The position a of the seat may be a criterion for the sliding of the seat, the height of the seat, and a point at which the tilting action of the seat may be reflected.

[0047] The image sensor 200 may be a 2D camera-type vision sensor that detects IR (infrared rays) and RGB (color), and is installed in a vehicle to detect a passenger sitting on the seat.

[0048] The controller 300 may be an integrated image controller that detects the position of the seat back using information acquired by the seat position sensor 100, detects a passenger's body part on the basis of an image detected by the image sensor 200, and classifies and detects objects.

[0049] To be more specific, the controller 300 includes a passenger detector 310 that detects the size of the passenger's body part as well as the body shape and body part of the passenger through an AI-based image recognition algorithm, and an object detector 320 that individually detects an object shown in the image.

[0050] Here, the passenger detector 310 may detect the passenger's body part and estimate the body size using Skeleton-based body key point technology, but other algorithms that may detect the passenger's body part and estimate the body size may be used.

[0051] As long as the object detector 320 is a detector capable of detecting an object, it may be applied to the present disclosure regardless of how it detects and learns the object.

[0052] On the other hand, when a passenger is detected in a designated area photographed by the image sensor 200, the body size of the passenger may be estimated using the passenger detector 310 and the object detector 320.

[0053] However, if the position a of the seat is changed by sliding, height adjustment, or tilting movement of the seat, such a change causes an image distortion phenomenon in which the body size of the passenger shown in the image changes, so an adult passenger and a child passenger may be misrecognized.

[0054] Thus, as shown in FIGS. 4 to 7, according to the present disclosure, seat position information and body part information of the passenger are detected on the basis of the normal sitting state of a specific passenger (or dummy) according to various changes in the position of the seat, a classification result for the body type of the passenger is previously learned using the detected information, and a learned body-type classification model 330 is stored in the controller 300.

[0055] At this time, specific passengers may be classified into regular adult passengers, adult passengers whose body size is smaller than that of a regular adult by 5% or more, and passengers with body types representing child passengers and infant passengers.

[0056] As such, when a passenger gets into the vehicle in a state where the model in which the body type of the passenger has been learned is stored in the controller 300, the position of the seat back and the body information of the passenger are detected using the passenger detector 310 and the object detector 320.

[0057] Thus, when data on the body type of the passenger on the basis of the detected information is detected and input, the result of determining the body type of the passenger is output using the modeled body-type classification model 330, so it is possible to more precisely estimate and determine the age of the passenger.

[0058] As shown in FIGS. 5 and 7, according to the present disclosure, parameter values that are input into the controller 300 may be a length d of a passenger's torso line, and a distance e between an upper line of a whole body area A2 detecting the passenger's whole body and an upper line of a seat back area A1.

[0059] Referring to the drawings, the seat back area A1 is detected using information acquired by the seat position sensor 100.

[0060] Further, from the image captured by the image sensor 200, the object detector 320 detects the outline of the passenger's whole body in the form of a rectangular box, and detects the outline of the passenger's face in the form of a rectangular box.

[0061] Further, the passenger detector 310 detects a length d from a passenger's shoulder line to a torso line.

[0062] That is, the parameter value data is measured and learned using a value measured through the image and information acquired by the seat position sensor 100. Therefore, when the passenger is detected, the type of the passenger may be determined on the basis of the corresponding parameter value data.

[0063] In addition, the parameter values input into the controller 300 may be an angle c between a horizontal shoulder line and a vertical torso line of a passenger, a distance e between an upper line of a whole-body area A2 detecting the whole body of a passenger and an upper line of a seat-back area A1, a distance f between an upper line of a face area A3 detecting a passenger's face and an upper line of the seat-back area A1, a size and position g of the seat-back area A1, and a minimum distance h between a center of the horizontal shoulder line of the passenger and a side line of the seat-back area A1.

[0064] For example, when an object covers the passenger's torso and distortion occurs in the measurement of the passenger's torso length, the parameter value data is measured and learned using the value measured through the image and the information obtained from the seat position sensor 100. Therefore, when a passenger is detected, the type of the passenger may be determined using the corresponding parameter value data.

[0065] On the other hand, according to the present disclosure, the controller 300 may determine whether the passenger is sitting normally by the body-type classification model 330 when an unspecified passenger is detected, and may control to induce self-correction for normal sitting, when the passenger is in the abnormal sitting state.

[0066] That is, when the passenger is detected in a designated area, it is checked whether the passenger is sitting normally. When the passenger is not in the normal sitting state, a warning or message for inducing the passenger to sit normally may be generated.

[0067] As a preferred example, when an unspecified passenger is in the abnormal sitting state, a currently photographed image may be transmitted to an AVN system along with a warning message indicating that the airbag may not be deployed.

[0068] That is, the warning message such as Please sit in a correct posture. The airbag of the passenger seat is not deployed. along with the image of the passenger sitting in the abnormal sitting state are transmitted to the AVN system in the vehicle, thereby inducing the passenger to sit normally.

[0069] In addition, when the CRS in the seat is not detected, the controller 300 may detect a shoe area A4 of the passenger in a box shape. When the length of the shoulder line in the shoe area A4 is equal to or more than a threshold value, it may be determined as the abnormal sitting state.

[0070] For example, referring to FIG. 8, when it is determined that the passenger sitting on the seat is a small 3-year-old child passenger, it may be estimated that the child passenger is standing on the seat if the length between the shoe of the child passenger and the shoulder line is equal to or greater than a set threshold value. This is determined as the abnormal sitting state.

[0071] As such, when it is determined that the sitting state is abnormal, the length of the shoulder line in the shoe area A4 may be corrected to be smaller than a length shown in an actual image, and a corrected state may be maintained until the passenger's posture is changed to a normal sitting state.

[0072] For example, when the child passenger is sitting on the seat in the abnormal sitting state, the body size of the child passenger detected in the image may be detected as large due to the change in posture of the child passenger. As a result, the body size of the child passenger may be distorted to the body size of the adult passenger, so the child passenger may be mistakenly recognized as the adult passenger.

[0073] In the event of an accident in a state where the child passenger is misrecognized as the adult passenger, the airbag cushion is deployed based on the characteristics of the adult passenger rather than using a LRD mode. This increases the risk of injury to the child passenger.

[0074] Therefore, the body size of the child passenger detected in the image is kept small until it is determined that the child passenger is in the normal sitting state, thus preventing the risk of injury to the passenger when the airbag is deployed.

[0075] In addition, when the shoulder line of the passenger is higher than the upper line of the seat-back area A1 and the length of the shoulder line is less than the threshold value, this may be determined as the abnormal sitting state by the controller 300.

[0076] For example, referring to FIG. 8, when it is determined that the passenger sitting on the seat is a small 3-year-old child passenger, it may be estimated that the child passenger is standing on the front of the seat if the height of the child passenger's shoulder line is higher than the upper line of the seat-back area A1 and the length of the shoulder line is less than a set threshold value. This is determined as the abnormal sitting state.

[0077] As such, when it is determined that the sitting state is abnormal, the length of the shoulder line of the passenger and the size of the seat-back area A1 may be corrected to be smaller than a length shown in an actual image, and a corrected state may be maintained until the passenger's posture is changed to a normal sitting state. For example, when the child passenger is sitting on the seat in the abnormal sitting state, the body size of the child passenger detected in the image may be detected as large due to the change in posture of the child passenger. As a result, the body size of the child passenger may be distorted to the body size of the adult passenger, so the child passenger may be mistakenly recognized as the adult passenger.

[0078] In the event of an accident in a state where the child passenger is misrecognized as the adult passenger, the airbag cushion is deployed based on the characteristics of the adult passenger rather than using a LRD mode. This increases the risk of injury to the child passenger.

[0079] Therefore, the body size of the child passenger detected in the image is kept small until it is determined that the child passenger is in the normal sitting state, thus preventing the risk of injury to the passenger when the airbag is deployed.

[0080] However, the controller 300 may determine that the passenger is in the normal sitting state when the shoulder line of the corresponding passenger is higher than the upper line of the seat-back area A1 and the length of the shoulder line is equal to or greater than a threshold value.

[0081] That is, as shown in FIG. 8, when the height of the passenger's shoulder line is higher than the upper line of the seat-back area A1 and the length of the shoulder line is less than a preset threshold value, it is determined that the passenger is a child passenger. In contrast, when the height of the passenger's shoulder line is higher than the upper line of the seat-back area A1 and the length of the shoulder line is equal to or more than the preset threshold value, it is determined that the passenger is an adult passenger.

[0082] Therefore, in this case, it is determined that the passenger is in the normal sitting state.

[0083] On the other hand, according to the present disclosure, the controller 300 may determine that the passenger is in the abnormal sitting state when a center line of the seat-back area A1 is higher than the upper line of the face area A3 detecting the passenger's face.

[0084] For example, referring to FIG. 9, when it is determined that the passenger sitting on the seat is a small 3-year-old child passenger, it may be estimated that the child passenger is lying on the seat if the height of the center line of the seat-back area A1 is higher than the upper end of the face area A3 of the child passenger. This is determined as the abnormal sitting state.

[0085] As such, when it is determined that the sitting state is abnormal, the seat-back area A1 and the face area A3 may be corrected to be larger than a size shown in an actual image, and a corrected state may be maintained until the passenger's posture is changed to a normal sitting state.

[0086] For example, when the child passenger is sitting on the seat in the abnormal sitting state, the body size of the child passenger detected in the image may be detected as small due to the change in posture of the child passenger. As a result, the body size of the child passenger may be distorted to the body size of the infant passenger, so the child passenger may be mistakenly recognized as the infant passenger.

[0087] In the event of an accident in a state where the child passenger is misrecognized as the infant passenger, the airbag cushion is not deployed. This increases the risk of injury to the child passenger.

[0088] Therefore, the body size of the child passenger detected in the image is kept large until it is determined that the child passenger is in the normal sitting state, thus preventing the risk of injury to the passenger when the airbag is deployed.

[0089] Further, the controller 300 may determine that the passenger is in the abnormal sitting state when a distance between the vertical torso line of the passenger and the side line of the seat-back area A1 is less than a threshold vehicle.

[0090] For example, referring to FIGS. 10 and 11, when the distance between the vertical torso line of the passenger and the left or right line of the seat-back area A1 is less than a preset threshold vehicle, it may be estimated that the passenger is leaning forward or the upper body of the passenger is positioned forward. This is determined as the abnormal sitting state.

[0091] As such, when it is determined that the sitting state is abnormal, the distance between the torso line of the passenger and the side line of the seat-back area A1 may be corrected to be smaller than a distance shown in an actual image, and a corrected state may be maintained until the passenger's posture is changed to a normal sitting state.

[0092] For example, when the child passenger is sitting on the seat in the abnormal sitting state, the body size of the child passenger detected in the image may be detected as large due to the change in posture of the child passenger. As a result, the body size of the child passenger may be distorted to the body size of the adult passenger, so the child passenger may be mistakenly recognized as the adult passenger.

[0093] In the event of an accident in a state where the child passenger is misrecognized as the adult passenger, the airbag cushion is deployed based on the characteristics of the adult passenger rather than using the LRD mode. This increases the risk of injury to the child passenger.

[0094] Therefore, the body size of the child passenger detected in the image is kept small until it is determined that the child passenger is in the normal sitting state, thus preventing the risk of injury to the passenger when the airbag is deployed.

[0095] On the other hand, according to the present disclosure, when the blanket A5 is detected by the image sensor 200, the controller 300 may detect an object around the blanket A5 to determine whether a passenger is sitting and determine a type of the passenger.

[0096] That is, when the blanket is detected in a box shape by the object detector 320 in the designated area, it is determined whether the passenger sits. When the passenger sits, the type of the passenger is determined according to the body type.

[0097] At this time, by operating in a mode of detecting only an adjacent object around the blanket A5, hardware resources are saved.

[0098] As a preferred example, when the CRS (child restraint system) A6 is detected together with the blanket A5, it may be determined that a relatively small-sized passenger sits.

[0099] For example, as shown in FIG. 12, when a car seat is detected in the shape of a box around the blanket A5, the passenger of the seat may be determined to be a 1-year-old infant passenger. For reference, when it is determined that the passenger is a 1-year-old infant passenger, control may be performed such that the airbag is not deployed in the corresponding seat.

[0100] In addition, when the blanket A5 is detected, the function of detecting the size of the passenger's body part is deactivated.

[0101] That is, when the blanket A5 is detected, the body of the passenger may be covered by the blanket A5, so the passenger may be misrecognized.

[0102] Therefore, the function of measuring the passenger's body size by a body key point is deactivated.

[0103] Further, when the passenger's face is detected together with the blanket A5 and the upper line of the seat-back area A1 is higher than the upper line of the face area A3 detecting the passenger's face, it may be determined that a relatively small-sized passenger sits.

[0104] For example, as shown in FIG. 13, when both the blanket A5 and the passenger's face are detected, it is determined that the passenger sitting on the seat is a 3-year-old child passenger if the upper line of the seat-back area A1 is higher in height than the upper line of the face area A3.

[0105] This is because the upper line of the seat-back area A1 is higher in height than the upper line of the face area A3 even if the adult passenger is fully reclined as shown in FIG. 14. When the upper line of the seat-back area A1 is higher in height than the upper line of the face area A3, it is determined that the passenger is a child passenger.

[0106] In addition, when only the blanket A5 is detected, it is determined that the passenger does not sit, and the captured image may be transmitted to the AVN system along with the warning message indicating that the airbag may not be deployed.

[0107] That is, as shown in FIG. 15, when only the blanket A5 is detected and a surrounding object is not detected, the blanket A5 is recognized as the object and the airbag is not deployed.

[0108] Thus, the warning message such as Please do not completely cover with blanket. The passenger airbag is not deployed. along with an image showing that the passenger is covered with the blanket are transmitted to the AVN system in the vehicle.

[0109] Therefore, when the passenger is completely covered with the blanket, control is performed such that the passenger's face may be seen and the airbag may be deployed in a dangerous situation.

[0110] As described above, the present disclosure provides a system for determining a type of a passenger, which can more clearly classify adult, child, and infant passengers by learning various postures in which the passengers sit and then determining types of passengers according to their body types using a learned body-type classification model 330.

[0111] Further, the present disclosure provides a system for determining a type of a passenger, in which it checks whether a passenger is sitting normally, the distortion of the passenger's body size occurring when the passenger is not sitting normally is compensated for, thus reducing the risk of injury to the passenger due to the malfunction of an airbag, and causing the passenger who does not sit normally to change to a correct posture, thereby reducing the risk of injury to the passenger.

[0112] In addition, the present disclosure provides a system for determining a type of a passenger, in which passengers are classified according to scenarios for each situation when a blanket A5 is detected, a CRS legal test mode is satisfied with an optimal resource through a blanket surrounding detection mode, and the risk of injury to a passenger is reduced by inducing the passenger to perform self-correction when it is impossible to detect the passenger due to the incorrect use of the blanket.