DEVICE AND METHOD FOR BIDIRECTIONAL COMMUNICATION BETWEEN A VEHICLE AND A PASSERBY

Abstract

A device and method for a vehicle to provide a bidirectional communication between the vehicle and at least one passerby. The device comprises a sensor for detecting a passersby near the vehicle and an array arranged on the vehicle and formed by a plurality of loudspeakers and a plurality of microphones for providing an acoustical beamforming focused in a predetermined region around the position of a detected passerby for the sending of acoustical signals to the passerby and for the receiving of acoustical information from the passerby. The device also has a computer-based recognition module coupled to the sensor and the array for recognizing gestures and/or acoustical messages from the passerby. An action of the vehicle, such as reproducing an acoustical signal and/or receiving a visual or acoustical message from the passerby, is performed based upon an identified passerby gesture and/or an identified acoustical message from the passerby.

Claims

1. A device for a vehicle to provide a bidirectional communication between the vehicle and at least one passerby, comprising: a sensor for detecting a passersby in a position near the vehicle, an array arranged on the vehicle and including a plurality of loudspeakers and a plurality of microphones, for providing an acoustical beamforming, focused in a predetermined region around the position of the passerby, and for sending acoustical signals to the passerby and for receiving acoustical information from the passerby, a computer-based recognition module in communication with the sensor for detecting the passerby, and in communication with the array for recognizing a gesture and/or an acoustical message in the acoustical information from the passerby, wherein the computer-based recognition module is configured to cause an action of the vehicle on the basis of an identified gesture and/or an identified acoustical message from the passerby, and wherein the action of the vehicle comprises a reproduction of an acoustical signal to the passerby and/or a receiving of a visual signal and/or the acoustical information from the passerby.

2. The device according to claim 1, wherein the vehicle is an autonomous vehicle.

3. The device according to claim 1, wherein the sensor for detecting the passersby comprises at least one camera.

4. The device according to claim 1, wherein the sensor for detecting the passersby is configured to determine position data of the passerby in relation to the vehicle.

5. The device according to claim 1, wherein the acoustical beamforming is dynamically focusable on the basis of a detected position data of the passerby.

6. The device according to claim 1, wherein the array formed with a plurality of loudspeakers and a plurality of microphones is arranged at least in the front region of the vehicle.

7. The device according to claim 1, wherein the array formed with a plurality of loudspeakers and a plurality of microphones is designed to fade out surrounding noises outside of a region of the acoustical beamforming focused around the position of the passerby.

8. The device according to claim 1, wherein the action triggered on the basis of a recognized passerby gesture and/or a recognized acoustical message of the passerby additionally involves a second action, wherein the second action is selected from a group of actions including a notification of a passenger of the vehicle, a stopping of the vehicle, a braking of the vehicle, an acceleration of the vehicle, an avoidance maneuver of the vehicle, an exiting of an autonomous driving mode of the vehicle and a shutting off of the vehicle.

9. A method for a vehicle to provide a bidirectional communication between the vehicle and at least one passerby, wherein the surroundings of the vehicle are detected by sensors and upon detecting a passerby at a given distance from the vehicle an acoustical beamforming emanating from the vehicle for the sending of acoustical signals to the passerby and for the receiving of acoustical information from the passerby is focused in a given region around the position of the passerby and upon recognizing a gesture and/or an acoustical message from the passerby, an action of the vehicle is triggered, wherein the action of the vehicle involves at least a reproduction of an acoustical signal to the passerby and/or a receiving of a visual signal and/or an acoustical message from the passerby.

10. The method as claimed in claim 9, wherein the position of a passerby is detected with at least one camera.

11. The method as claimed in claim 9, wherein the acoustical beamforming for the transmittal of acoustical signals is dynamically focused in a predetermined region around the position of the passerby on the basis of detected position data of the passerby.

12. The method as claimed in claim 9, wherein an acoustical message from the passerby and/or a gesture of the passerby is detected during and/or after an action of the vehicle.

13. The method as claimed in claim 9, wherein surrounding noises outside of a region of the acoustical beamforming focused around the position of the passerby are faded out.

14. The method as claimed in claim 9, wherein, upon recognizing a gesture and/or an acoustical message from the passerby, a further action of the vehicle is additionally triggered, wherein the further action is selected from a group of actions including a notification of a passenger of the vehicle, a stopping of the vehicle, a braking of the vehicle, an acceleration of the vehicle, an avoidance maneuver of the vehicle, an exiting of an autonomous driving mode of the vehicle and a shutting off of the vehicle.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Further details, features and advantages of designs of the invention result from the following description of embodiment examples in reference to the associated drawings.

[0032] FIG. 1 shows a schematic representation of a sample embodiment of the device for a vehicle, especially an autonomous vehicle, and schematic diagram of a first design of the display device according to the invention, and

[0033] FIG. 2 shows a flow chart of a sample embodiment of the method for a vehicle to provide a bidirectional communication between the vehicle and a passerby.

DETAILED DESCRIPTION

[0034] Recurring features are marked with identical reference numerals in the figures.

[0035] FIG. 1 a schematic representation of a sample embodiment of the device for a vehicle 1, especially an autonomous vehicle 1, to provide a bidirectional communication between the vehicle 1 and a passerby 2. The passerby 2 may be a pedestrian, or other person in the vicinity of the vehicle including, for example, person on a bicycle, scooter, motorcycle, horse, etc. The device comprises a sensor 3 for the detection of passersby 2 in a given surroundings of the vehicle 1. The sensor 3 involves a camera 3, which in the sample embodiment shown is arranged in the front region of the vehicle 1. The broken lines indicate the visual region 3.1 or viewing angle of the camera 3. The passerby 2 is located within the visual region 3.1 of the camera 3, as indicated by the broken lines, so that the camera 3 can detect the passerby 2. By means of a computer unit, which is designed to evaluate and interpret camera images of the camera 3 and which may be part of the sensor 3, position data of the passerby 2 in relation to the vehicle 1 can be determined. The computer unit may be outfitted with corresponding software for the detection and interpretation of camera images.

[0036] Furthermore, the device comprises an array 4 formed with a plurality of loudspeakers and microphones to provide an acoustical beamforming 5 focused in a given region around the position of the detected passerby 2, which is represented in FIG. 1 by a drop-shaped bubble emerging from the array 4 and surrounding the passerby 2. In the example shown, the array 4 formed with a plurality of loudspeakers and microphones is arranged in a front region of the vehicle 1. According to one advantageous embodiment, not shown, the array is arranged over the entire vehicle surface, with the plurality of microphones and the plurality of loudspeakers distributed in a regular arrangement or in an almost random arrangement over the entire structure of the vehicle 1.

[0037] The focusing of the acoustical beamforming 5 in a region around the passerby 2 is based on the passerby position data determined by the sensor 3 and it serves for the sending of acoustical signals to the passerby 2 and for the receiving of acoustical information from the passerby.

[0038] Furthermore, the device comprises a computer-based recognition module (not shown), coupled to, or in communication with, the sensor 3 and to the array 4 formed with a plurality of loudspeakers and microphones for the recognition of passerby gestures and/or acoustical messages from the passerby 2. The computer-based recognition module is configured to perform an action of the vehicle 1, on the basis of a recognized passerby gesture and/or a recognized acoustical message from the passerby 2. The action of the vehicle 1 comprises a reproduction of an acoustical signal to the passerby 2 and/or a reception of a visual signal and/or the acoustical information from the passerby 2.

[0039] The device, according to one modification, is configured such that, in response to an action triggered on the basis of a recognized passerby gesture and/or a recognized acoustical message from the passerby 2, a second action is performed in addition, wherein the second action is chosen from a group of actions including a notification of a passenger of the vehicle 1, a stopping of the vehicle 1, a braking of the vehicle 1, an acceleration of the vehicle 1, an avoidance maneuver of the vehicle 1, an exiting of an autonomous driving mode of the vehicle 1 and a shutting off of the vehicle 1. One such further action of the vehicle 1, especially the autonomous vehicle 1, during which a notification of a vehicle passenger is done, is shown in the sample embodiment of FIG. 1.

[0040] Acoustical signals of loudspeaker 6 arranged inside the vehicle and reproduced inside the vehicle 1 are indicated by lightly colored lines. It may be provided that the acoustical data 7 being acoustical messages emanating from the passerby 2 are reproduced inside the vehicle 1. In this way, it is possible to call the attention of vehicle passengers (not shown) to a situation outside the vehicle and/or to produce a communication between vehicle passengers and the passerby 2. Thanks to the acoustical beamforming 5 around the position of the passerby 2 provided with the array 4, surrounding noises outside the focused region of the acoustical beamforming 5 are faded out, so that an undisturbed communication can be assured. The acoustical signal reproduction in the direction of the passerby 2 within the acoustical beamforming 5 prevents passersby outside of the acoustical beamforming 5 from being drawn into the communication.

[0041] FIG. 2 shows a flow chart to explain the method for a vehicle 1, especially an autonomous vehicle 1, to provide a bidirectional communication between the vehicle 1 and a passerby 2. The method is to be used preferably with a device as shown in FIG. 1 or an embodiment of the device shown in FIG. 1. For the explanation of FIG. 2, therefore refer to the device shown in FIG. 1.

[0042] According to a preferred embodiment, the vehicle 1 is in an autonomous operating mode and can move by itself. This means that the vehicle 1 moves without direct intervention by human action. In the method for providing a bidirectional communication between the vehicle 1 and a passerby 2, first of all the surroundings of a vehicle 1 are detected by sensor, as indicated by step 10 of the method. If a passerby 2 is identified during the detecting of the vehicle's surroundings, as indicated by reference 20 in the flow chart, in the next step 30 of the method it is assessed whether an unchanged movement maneuver of the vehicle 1 is required based on the distance and/or the speed of movement and/or the direction of movement of the passerby 2 and taking in to account the movement of the vehicle 1 itself. Basically, it is to be assumed that an adapting of the movement of the vehicle 1 is required if an identified passerby 2 is already at a short distance from the vehicle 1. It may therefore be provided that a safety region is established at a given radius around the vehicle 1, in which a change in the movement of the vehicle 1 is triggered directly upon identifying a passerby 2. If an identified passerby 2 is located at a distance from the vehicle 1 that requires a communication, a decision 41 is made to establish a communication with the identified passerby 2. Furthermore, according to an alternative variant of the method not included in the flow chart, it may be provided that the decision on providing a communication with an identified passerby 2 is made based on a gesture and/or an acoustical message from the identified passerby 2. In this case, gestures can be detected by sensor or with a camera and/or acoustical signals may be detected with microphones and be evaluated and interpreted by means of software.

[0043] The next step of the method is a determination of specific position data of the passerby 2 on the basis of sensor data and/or camera images, as indicated in the flow chart by reference 50. For this, a software-supported target acquisition can be used. The determined passerby position is used in the next step 60 in order to focus an acoustical beamforming 5 emanating from the vehicle 1 for the sending of acoustical signals to the passerby 2 and for the receiving of acoustical information from the passerby 2 in a given region around the position of the passerby 2. The focus region of the acoustical beamforming 5 is chosen such that acoustical signals sent out from the vehicle 1 are only perceived by the affected passerby 2, without other passersby who are located outside the focus region of the acoustical beamforming 5 being able to perceive the acoustical signals sent out by the vehicle 1.

[0044] After reproduction of an acoustical signal 70 directed to the passerby 2, an action 80 of the vehicle 1 is triggered, while in the sample embodiment shown this involves the receiving of an acoustical information from the passerby 2 as a reaction to the signal 70 sent out by the vehicle 1. If the acoustical information from the passerby 2 does not require any further communication with the vehicle 1, the method continues with the decision 42 triggering a further action 90 of the vehicle 1, where the further action in the present example involves a driving on of the vehicle 1. The bidirectional communication can be maintained until the passerby 2 has left a safety region of the vehicle 1 or reached a given distance from the vehicle 1. Alternatively, the bidirectional communication is interrupted after a given length of time, as long as no acoustical signals affecting the vehicle 1 have been received during this length of time.

[0045] Furthermore, it may be provided that, based on a gesture and/or an acoustical message from the passerby 2, which can be detected by the vehicle 1 according to step 80 of the method, a further action of the vehicle 1 is additionally triggered, the further action being chosen from a group of actions including a notification of a passenger of the vehicle 1, a stopping of the vehicle 1, a braking of the vehicle 1, an acceleration of the vehicle 1, an avoidance maneuver of the vehicle 1, an exiting of an autonomous driving mode of the vehicle 1 and a shutting off of the vehicle 1.

[0046] If it is determined upon interpretation or recognition of a verbal reaction of a passerby 2 that a further communication with the passerby 2 is required, the bidirectional communication between the vehicle 1 and the passerby 2 will be maintained. It may be provided that the focus region of the acoustical beamforming 5 around the position of the passerby 2 is dynamically adapted when the vehicle 1 and the passerby 2 are moving relative to each other. For the dynamic adaptation of the focus region of the acoustical beamforming 5 a target acquisition for the determination of the passerby position data may be used.

[0047] The system, methods and/or processes described above, and steps thereof, may be realized in hardware, software or any combination of hardware and software suitable for a particular application. The hardware may include a general purpose computer and/or dedicated computing device or specific computing device or particular aspect or component of a specific computing device. The processes may be realized in one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors or other programmable device, along with internal and/or external memory. The processes may also, or alternatively, be embodied in an application specific integrated circuit, a programmable gate array, programmable array logic, or any other device or combination of devices that may be configured to process electronic signals. It will further be appreciated that one or more of the processes may be realized as a computer executable code capable of being executed on a machine readable medium.

[0048] The computer executable code may be created using a structured programming language such as C, an object oriented programming language such as C++, or any other high-level or low-level programming language (including assembly languages, hardware description languages, and database programming languages and technologies) that may be stored, compiled or interpreted to run on one of the above devices as well as heterogeneous combinations of processors processor architectures, or combinations of different hardware and software, or any other machine capable of executing program instructions.

[0049] Thus, in one aspect, each method described above and combinations thereof may be embodied in computer executable code that, when executing on one or more computing devices performs the steps thereof. In another aspect, the methods may be embodied in systems that perform the steps thereof, and may be distributed across devices in a number of ways, or all of the functionality may be integrated into a dedicated, standalone device or other hardware. In another aspect, the means for performing the steps associated with the processes described above may include any of the hardware and/or software described above. All such permutations and combinations are intended to fall within the scope of the present disclosure.

[0050] Obviously, many modifications and variations of the present invention are possible in light of the above teachings and may be practiced otherwise than as specifically described while within the scope of the appended claims.