System for interaction between an autonomous vehicle and a vulnerable user

Abstract

A system for implementing an interaction between a semi-autonomous or autonomous motor vehicle and a vulnerable user of a carriageway used by the motor vehicle and potentially crossed by a vulnerable user at a crossing, pedestrians and cyclists being considered to be vulnerable users. The system including an environment sensor, a computer processing the data received from the one or more environment sensors, and a display device arranged on the vehicle and directed to the outside of the vehicle, the display device in the form of a horizontal strip and occupying a longitudinal end of the vehicle, and providing vulnerable users present in the vehicle's environment in the vicinity of the crossing with a visual indication as to whether or not they may cross the carriageway road surface, so that pedestrians and cyclists can obtain unambiguous information as to their presence being taken into account by the vehicle.

Claims

1. A system for implementing an interaction between a semi-autonomous or autonomous motor vehicle and at least one vulnerable user of a carriageway used by the motor vehicle and potentially crossed by a vulnerable user at a crossing, which might or might not be marked, pedestrians and cyclists being considered to be vulnerable users, said system comprising: at least one environment sensor arranged on the vehicle; at least one computer on board the vehicle, processing the data received from the at least one environment sensor; and at least one display device arranged on the vehicle, and directed to the outside of the vehicle, and controlled by the computer, wherein, the display device is in the form of a horizontal strip and occupies at least one longitudinal end of the vehicle; the display device comprises a matrix panel having at least four pixels per cm.sup.2, and the display device being configured to provide vulnerable users present in an environment of the vehicle in the vicinity of the crossing with at least one visual indication as to whether or not they may cross the carriageway road surface, and delivers either an indication prompting them to cross if the vulnerable user is at a protected crossing or, conversely, an indication forbidding them from crossing if the vulnerable user is not at a protected crossing, so that pedestrians and cyclists can obtain unambiguous information as to their presence being taken into account by the semi-autonomous or autonomous motor vehicle, the display device also being configured to display a representation of a scene comprising the crossing, the users being symbolically represented by character avatars which may be stationary or moving, of which the rendering, by the display device, is performed in real time and of which the respective positions relative to the crossing reflect reality with a delay of less than 1 second, and of which a number corresponds to the number of people actually detected in the vehicle's environment.

2. The system as claimed in claim 1, wherein the strip occupies substantially an entire width of a front face of the vehicle and/or an entire width of a rear face of the vehicle.

3. The system as claimed in claim 1, wherein the vehicle comprises a display device on its front face and on its rear face, over the entire width.

4. The system as claimed in claim 1, wherein the computer is configured to classify entities perceived by the one or more environment sensors.

5. The system as claimed in claim 1, wherein the display device is arranged at a height defined by a bottom line of the display located above 1 m from the ground and its top line located below 1.40 m from the ground.

6. The system as claimed in claim 1, wherein the display device is configured to display an indication that the vehicle is about to start off and drive over the crossing.

7. A vehicle comprising a system as claimed in claim 1.

8. The system as claimed in claim 2, wherein the vehicle comprises a display device on its front face and on its rear face, over the entire width.

9. The system as claimed in claim 1, wherein the delay is less than 600 ms.

10. The system as claimed in claim 1, wherein the computer obtains information from a map database to determine if the crossing is a protected crossing.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Other features, details and advantages will become apparent from reading the following detailed description and from examining the appended drawings, in which:

(2) FIG. 1 is a general basic diagram of one embodiment of the invention,

(3) FIG. 2 shows one example of a basic diagram relating to the main display screen,

(4) FIG. 3 shows a functional schematic view of the main display screen,

(5) FIG. 4 illustrates two exemplary indications for allowing crossing or, conversely, forbidding crossing,

(6) FIGS. 5A and 5B illustrate the display configuration on the strip of the vehicle's front display device, with pedestrians crossing in front of the vehicle,

(7) FIG. 6 illustrates various types of avatars representing pedestrians, the avatars being inserted into the representation of the scene with their position and the change in each position over time,

(8) FIG. 7 illustrates the display toward the end of the crossing sequence indicating that the vehicle is about to start off, and

(9) FIG. 8 illustrates one particular variant embodiment, relating to a shuttle-type autonomous vehicle.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

(10) In the various figures, the same references have been used to reference identical or similar elements.

(11) With reference to FIG. 1, a semi-autonomous or autonomous motor vehicle Vh1 travels on a carriageway 9 which can be either a road or a street. FIG. 1 shows a pedestrian crossing referenced by 90, which in the present case is marked by lines painted on the ground (alternation of black and white strips 92, 93). However, it should be noted that in some cases, there may be pedestrian crossings which are not marked on the ground or else they are marked in another way, for example via crossing markings, pseudo crossing markings, lines and/or dots. In certain cases the crossing may be qualified as a “protected crossing”, but the invention is not limited to that case.

(12) Regarding the vehicle, it is an autonomous, or semi-autonomous or partially autonomous vehicle, that is to say that, in practice, this may correspond to a case where there is no one driving the vehicle, nor any person in any driving position (completely autonomous vehicle), this can also correspond to a case where there is someone in the driving position but who delegates driving functions to the driver assistance functions performed by the onboard electronics. It is noted that the level of delegation can change over time.

(13) It is noted that an aspect of the invention can also be applied to a vehicle driven in a conventional manner but with tinted windows that make visual contact between pedestrians and the driver impossible.

(14) It should be noted that the vehicle in question can be a private vehicle, a minibus, a van, a delivery truck, a taxi, or a self-driving taxi. The vehicle in question can also be a tram, a trolleybus, a tram-train, etc.

(15) The vehicle in question can be designed to travel on carriageways in either its own or a public area, or in mixed situations where it uses carriageways in its own area and then carriageways in a public area or vice versa.

(16) As will be seen in more detail below, the vehicle Vh1 is provided with one display device 1 at the front of the vehicle (in the normal direction of travel) and, optionally, another display device 101 located at the rear of the vehicle.

(17) The carriageway can be one-way or can be two-way. It is not excluded for there to be a plurality of parallel lanes in the same direction which corresponds, in everyday language, to a multi-lane traffic thoroughfare.

(18) Coming now to the users of the carriageway, “vulnerable users” are generally defined as those people present who are not in a vehicle's passenger compartment; the concept of “vulnerable users” includes pedestrians, cyclists, adults, children or old people, users of various strollers, scooters, skateboards, personal mobility devices, monocycles, etc. The concept of “vulnerable users” can also include pets.

(19) FIG. 1 shows a pedestrian U1 located to the left of the road from the vehicle's point of view, a cyclist U2 preparing to cross the road from the left, and an adult U4 and a child U3 who are both pedestrians and are preparing to cross the road from the right from the vehicle's point of view.

(20) The display device 1 can be presented as illustrated in FIG. 2 with a main strip 11 which extends over the entire width of the vehicle.

(21) In the example illustrated, the display device 1 is rectangular with a bottom edge 1a and a top edge 1d.

(22) The display device is arranged at least on one longitudinal end of the vehicle.

(23) A lateral extension 12 onto the side of the vehicle can be provided, either on one side only or on both sides symmetrically.

(24) The strip is horizontal, that is to say wider than it is high. It may be generally rectangular in shape.

(25) The typical dimensions of the horizontal strip are at least 1.60 m in width and 30 cm in height. Generally, the height H1 will be within a range [20 cm-40 cm]. Generally, the width L1 will be within a range [140 cm-200 cm].

(26) The lateral extension 12 may have the same height, or even a greater height, and it may have a length in the longitudinal direction of between 20 and 50 centimeters.

(27) The height position of the display device is defined by a bottom line of the display located above 0.9 m from the ground and its top line located below 1.50 m from the ground. Stated otherwise, the display device is located within the range [0.9 m-1.5 m].

(28) The display device comprises a matrix panel whose resolution is at least four pixels per cm.sup.2, or 25 pixels per square inch. Of course, a higher resolution is entirely possible.

(29) The display device can be in the form of an LED matrix, either a matrix of multicolor LEDs or interlaced matrices of monochromatic LEDs. The primary colors can be used as the basis for generating, in particular, the colors red and green but also other, additional colors.

(30) The vehicle is equipped with one or more environment sensors 3. Regarding the environment sensor, it can be a lidar sensor, for example with 360° coverage; it can be a radar; it can be a 3D camera; it can be a time-of-flight (TOF) camera; or it can be a combination of a plurality of sensors.

(31) In addition, provision is made to use an infrared camera which makes it possible to detect the temperature of each source and, in particular, facilitates the classification of entities in order to extract living beings from the environment.

(32) FIG. 1 shows, for example, a central sensor 3 arranged on the roof of the vehicle, for example a 360° lidar. In addition, another sensor 31 is provided at the front of the vehicle chosen from the technologies listed above. In addition, another sensor 32 is provided at the rear of the vehicle chosen from the technologies listed above.

(33) According to another configuration illustrated in FIG. 8, provision can be made for a sensor 3 located in the upper region of the vehicle's front portion, and sensors 31, 32 arranged in the vehicle's bumper, each of these sensors being able to be of any of the above types.

(34) The vehicle comprises at least one computer referenced by 4; said computer may also be referred to as an “electronic control unit” (“ECU”).

(35) The computer 4 processes the data received from the one or more environment sensors and performs one or more operations for classifying the entities that are detected in the images or, more generally, the data received from the sensors.

(36) In particular, an algorithm is provided which makes it possible to extract, from the data or images received from the sensors, entities corresponding to living beings, people or animals in the vehicle's immediate environment.

(37) The classification of the entities may involve image content analysis and/or spectral analysis.

(38) Certain elements can give rise to easier recognition, such as bicycle or stroller wheels; specifically, a continuous or discontinuous ellipse (a plurality of arcs of an ellipse) is recognized.

(39) The classification of the entities can also make use of a motion analysis, in particular to identify the motions of people in the data or images received from the sensors.

(40) After analyzing the data received from the sensors, if it is determined that one or more vulnerable users are about to cross the roadway, the computer can decide to stop the autonomous vehicle and give way to allow the vulnerable users to cross.

(41) It is noted that analysis and classification are performed continuously under the conditions where the vehicle has decided to stop to allow pedestrians to cross.

(42) It is noted that analysis and classification continue as vulnerable users cross the road.

(43) The display device 1 is configured to provide the vulnerable users present in the vehicle's environment in the vicinity of the protected crossing with at least one visual indication as to whether or not they may cross the carriageway road surface, and delivers either an indication prompting them to cross or, conversely, an indication forbidding them from crossing.

(44) Particularly, the visual indication may use a conventional color code, namely green to indicate a prompt to cross and red to indicate that crossing is forbidden.

(45) The visual indication can, in particular, use all or some of the pictograms commonly used at pedestrian crossings, namely a pictogram indicating a prompt to cross in green, illustrated by reference 21 in FIG. 4, and a pictogram indicating that crossing is forbidden in red, illustrated by reference 22 in FIG. 4. The color orange can also be used for the pictograms. Using effects involving changes of color from among the above colors or else other colors is not excluded either.

(46) The conventional pictograms 21, 22 can, in particular, be displayed on the lateral extensions 12 of the display device, as by analogy with the pictograms for crossings for pedestrians.

(47) The display is updated in real time, according to the stream of information received from the environment sensors.

(48) The display device is configured to display a representation of the scene comprising the protected crossing, as illustrated in FIGS. 5A, 5B and 7, with the users symbolically represented there by character avatars which may be stationary or moving.

(49) Each avatar reflecting the type of vulnerable user according to the classification can have characteristics representative of the classification of the entity: size of the character, direction of movement, stationary posture, low speed, high speed. Accessories used by the character can also be depicted; for example a bicycle, a skateboard, a stroller, a scooter, a pet, etc.

(50) More precisely, as illustrated in FIG. 6, the posture and size of the avatars represented reflects the presence and the posture of the people actually present in the vicinity of the vehicle, for example an adult in a stationary posture 51, a child in a stationary posture 52, two adults crossing together 53, 54, and a walking teenager 55.

(51) In FIGS. 5A and 5B, the character avatars are in the shape of cylinders 50.

(52) The computer 4 can access map databases which allow it to determine the position of the protected crossings listed, without this constituting an exhaustive list however. Thus, the level of filtering performed by the computer with regard to the human or animal entities classified may depend on the presence or absence of a listed protected crossing; in practice, the decision to stop to allow pedestrians to cross will be favored at the protected crossings listed.

(53) In the event that no pedestrian crosses despite the presence of entities classified as persons at the sides of the road, the autonomous vehicle switches back from the message prompting crossing to a message forbidding crossing and/or a message indicating that the vehicle is about to start off.

(54) As shown in FIG. 3, the computer 4 receives a continuous stream of data from the one or more environment sensors 3, 31, 32 via an onboard local area network or wirelessly, and it is connected to one or more displays via an onboard local area network or wirelessly.

(55) In addition, a sound source can be provided, which is used in addition to the display device and is controlled by the computer, to make it possible to strengthen the level of information for the users, complementing purely visual information which might be insufficient for certain vulnerable users focused on using their smartphone.

(56) In addition, a light source can be provided on the vehicle, used in addition to the display device, and controlled by the computer 4, to allow the crossing, which might or might not be marked on the ground, to be illuminated, in order to strengthen the perception and intelligibility of the message given by the display device to a vulnerable user crossing, or about to cross, the road.

(57) It is noted that the semi-autonomous or autonomous vehicle, when it stops to allow pedestrians to cross, observes a safety distance, referenced by DG, with respect to the marked crossing, as illustrated in FIGS. 5A and 5B. The safety distance DG can be configured to be between 1 m and 3 m; the safety distance can also be dependent on the circumstances, for example day or night, the type of carriageway, or the classification of the vulnerable users identified.

(58) With reference to FIG. 8, the front façade of the vehicle is almost vertical with, from bottom to top, a bumper in the bottom portion, then a front body region, then the main display strip 11, then a glazed region 18, above which is one of the sensors 3.

(59) It is noted that the lateral extensions 12 on the side of the vehicle have a position that is slightly higher than the front central strip 11. For example, the lateral extensions 12 each extend in the longitudinal direction over at least 30 cm, and over a height of 25 to 40 cm.

(60) In the example shown, the display device is continuous, i.e. there is no interruption between the lateral extensions 12 and the front central strip 11, which allows an avatar to move continuously from one side to the other without any singular point, as when the vulnerable user is crossing. However, it is not excluded for there to be a front strip and lateral displays which are not directly an extension of the front strip.