METHOD FOR COORDINATING ROAD USERS VIA A SERVER DEVICE, AND SERVER DEVICE AND A CONTROL CIRCUIT FOR CARRYING OUT THE METHOD

Abstract

A method of coordinating road users via a server device which determines instruction data by a central model and transmits the instruction data as action instruction to respective control circuits of the road user, the central model being a virtual image of a real environment created by the server device from environment data, includes when a road user reaches the real environment a local model is created by onboard sensor data and delta data relating to the central model is determined. The delta data is transmitted to the server device by the control circuit of the road user which creates an updated central model by the delta data and the central model. The server device determines and transmits updated instruction data to the road user as readjustment of the instruction data.

Claims

1-10. (canceled)

11. A method of coordinating road users via a server device which, from environment data of an environment, repeatedly creates a digital central model of the environment and calculates respective instruction data for action instructions for at least one road user, from among a plurality of road users by the digital central model, an action instruction, from among the action instructions, defines an action to be executed by the at least one road user in the environment, the method comprising: by the server device, transmitting the action instruction to the at least one road user; repeatedly providing the at least one road user with the digital central model to perform a comparison of the digital central model with the environment; receiving from the at least one road user, delta data describing a difference between the environment and the digital central model, the difference being detected by the at least one road user; and updating the digital central model on basis of the delta data and using the updated digital central model to check whether the instruction data for the at least one user is to be corrected in accordance with a correction criterion, the digital central model, describes a plurality of traffic objects and a traffic object, from among traffic objects, is assigned to an object category, from a plurality of object categories, and at least one stipulation, from among the following stipulations is stipulated for the object category, a change frequency specifying a maximum rate at which the delta data is to be generated for the traffic object of the object category, and/or a priority stipulating an order with regard to determining the delta data, based on available computation time; and when the instruction data to be corrected is identified according to the error criterion, updating the instruction data by adapting the action instruction to the updated digital central model, and transmitting, to the at least one user, the updated instruction data of the action instruction for the at least one user.

12. The method as claimed in claim 11, wherein the adaptation of the action instruction is prioritized by the server device depending on a speed of movement of the at least one road user and/or a traffic situation that the at least one road user is in.

13. The method as claimed in claim 11, wherein the server device generates and/or updates the digital central model additionally on basis of sensor data from the at least one road user.

14. The method as claimed in claim 13, wherein the sensor data comprise at least some data, from among data including geoposition data of a current location of the road users, speed data of a speed of movement of the road users, or observation data concerning the environment from viewpoint of the road users.

15. The method as claimed in claim 11, wherein a geoposition and/or a relative position of the traffic object moving in the environment are/is extrapolated in the digital central model by the server device.

16. The method as claimed in claim 11, wherein the server device transmits temporally in advance the instruction data for the action instruction with a future execution time, thus resulting in a latency value between a transmission time of the instruction data and the execution time, wherein, by selection of the transmission time, the latency value is set depending on a speed of the at least road user and/or a traffic situation in a region in front of the at least one road user.

17. The method as claimed in claim 16, wherein the at least one road user carries out the action instruction in a manner shifted temporally ahead by a latency value in a simulation and estimates a future traffic situation resulting therefrom by the digital central model.

18. The method as claimed in claim 11, wherein to perform the comparison of the environment with the digital central model, the at least one road user carries out: by sensor data of a sensor circuit carried by the at least one road user , a digital local model of the environment is created and a difference between the digital local model and the digital central model is determined and described as the model difference by the delta data, wherein the at least one road user transmits the delta data to the server device.

19. A server device , comprising: a processor configured to, repeatedly create, from environment data of an environment, a digital central model of the environment and calculating respective instruction data for action instructions for at least one road user, from among the plurality of road users by the digital central model, an action instruction, from among the action instructions, defines an action to be executed by the at least one road user in the environment, transmit the action instruction to the at least one road user; repeatedly provide the at least one user with the digital central model to perform a comparison of the digital central model with the environment; receive from the at least one road user delta data describing a difference between the environment and the digital central model, the difference being detected by the at least one road user; and update the digital central model on basis of the delta data and using the updated digital central model to check whether the instruction data for the at least one user is to be corrected in accordance with a correction criterion, the digital central model, describes a plurality of traffic objects and a traffic object, from among traffic objects, is assigned to an object category, from a plurality of object categories, and at least one stipulation, from among the following stipulations is stipulated for the object category, a change frequency specifying a maximum rate at which the delta data is to be generated for the traffic object of the object category, and/or a priority stipulating an order with regard to determining the delta data, based on available computation time; and when the instruction data to be corrected is identified according to the error criterion, update the instruction data by adapting the action instruction to the updated digital central model, and transmit, to the at least one user, the updated instruction data of the action instruction for the at least one user.

20. The server device as claimed in claim 19, wherein the processor is further configured to generate and/or update the digital central model additionally on basis of sensor data from the at least one road user.

21. The server device as claimed in claim 20, wherein the sensor data comprise at least some data, from among data including geoposition data of a current location of the road users, speed data of a speed of movement of the road users, or observation data concerning the environment from viewpoint of the road users.

22. The server device as claimed in claim 19, wherein the processor is further configured to extrapolate a geoposition and/or a relative position of the traffic object moving in the environment in the digital central model.

23. The server device as claimed in claim 19, wherein the processor is further configured to control transmitting temporally in advance the instruction data for the action instruction with a future execution time, thus resulting in a latency value between a transmission time of the instruction data and the execution time, wherein, by selection of the transmission time, the latency value is set depending on a speed of the at least road user and/or a traffic situation in a region in front of the at least one road user.

24. The server device as claimed in claim 23, wherein the at least one road user carries out the action instruction in a manner shifted temporally ahead by a latency value in a simulation and estimates a future traffic situation resulting therefrom by the digital central model.

25. The server device as claimed in claim 19, wherein to perform the comparison of the environment with the digital central model, the at least one road user carries out: by sensor data of a sensor circuit carried by the at least one road user , a digital local model of the environment is created and a difference between the digital local model and the digital central model is determined and described as the model difference by the delta data, wherein the at least one road user transmits the delta data to the server device.

26. A control circuit for a road user, the control circuit configured to, receive model data of a digital central model of an environment of the road user from a server device, the server device including a processor configured to, repeatedly create, from environment data of an environment, a digital central model of the environment and calculating respective instruction data for action instructions for at least one road user, from among the plurality of road users by the digital central model, an action instruction, from among the action instructions, defines an action to be executed by the at least one road user in the environment, transmit the action instruction to the at least one road user; repeatedly provide the at least road user with the digital central model to perform a comparison of the digital central model with the environment; receive from the at least one road user delta data describing a difference between the environment and the digital central model, the difference being detected by the at least one road user; and update the digital central model on basis of the delta data and using the updated digital central model to check whether the instruction data for the at least one user is to be corrected in accordance with a correction criterion, the digital central model, describes a plurality of traffic objects and a traffic object, from among traffic objects, is assigned to an object category, from a plurality of object categories, and at least one stipulation, from among the following stipulations is stipulated for the object category, a change frequency specifying a maximum rate at which the delta data is to be generated for the traffic object of the object category, and/or a priority stipulating an order with regard to determining the delta data, based on available computation time; determine the delta data describing a difference between the environment and the digital central model, the difference being detected by the at least one road user; transmit the difference to the server device; and receive instruction data having an action instruction for an action in the environment from the server device and adapt the action instruction depending on updated instruction data from the server device.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0044] These and other aspects and advantages will become more apparent and more readily appreciated from the following description of the examples, taken in conjunction with the accompanying drawings of which:

[0045] FIG. 1 shows a traffic situation in which a road user is coordinated with respect to a real environment by the central model provided by a server unit;

[0046] FIG. 2 shows an overview of the process of updating the central model with delta data.

DETAILED DESCRIPTION

[0047] Reference will now be made in detail to the described examples which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout.

[0048] The examples explained below are examples of the invention. In the examples, the described components of the examples each constitute individual features which may be considered independently of one another and which also develop the examples independently of one another. Therefore, the disclosure is also intended to encompass combinations of the features of the examples other than those presented. Furthermore, the described examples are also able to be supplemented by further features of the examples from among those already described.

[0049] In the figures, identical reference signs designate in each case functional identical elements.

[0050] FIG. 1 shows a traffic situation in a real environment 11 in which a motor vehicle 2 as road user is coordinated by a central model 4 of the real environment 11, which central model is provided by a server device 1. In this example, an intersection with a building 8 as an advertising column is illustrated as the real environment 11. In the direction of the intersection in the real environment 11, an autonomously driving motor vehicle 2 in the role of a road user is moving toward the intersection from the left. In this case, the direction of travel of the motor vehicle is illustrated by an arrow. The autonomously driving motor vehicle 2 comprises a control circuit 10, which receives instruction data A2 from the server device 1. The motor vehicle 2 can likewise comprise a sensor circuit 13. In this case, the sensor circuit 13 denotes an interconnection of onboard sensors which can monitor the environment of the motor vehicle 2. They can be for example a camera sensor, a radar sensor and/or else general vehicle data such as, for example, a vehicle speed or a current steering angle. In this case, the control circuit 10 is connected to the sensor circuit 13 for data exchange. In this case, the instruction data A2 are action instructions that can be executed directly by the motor vehicle 2. The instruction data A2 can include for example a steering angle, here for example 0° for traveling straight ahead, and a constant speed of 50 km/h if for example the motor vehicle 2 is moving on the priority road across the intersection in the real environment 11 without traffic. In this case, the instruction data A2 are calculated by the server device 1 by the central model 4 and are transmitted to the control circuit 10 of the motor vehicle 2 with a latency before the intersection in the real environment 11 is reached.

[0051] The central model 4 of the real environment 11 is created and updated in the server device 1 by environment data U. In this case, the environment data U can originate from a number of sources. In this regard, sources of environment data U can be for example a digital road map, a traffic management system (traffic signs, traffic light systems, traffic flow, traffic density), weather data, geographic information, data of a building security system (surveillance camera system), geographic information from road users. The central model 4 can thus correspond to a current virtual representation of the real environment 11. The server device 1 thus has a macroperspective regarding the traffic events in the real environment 11 since the server device 1 can combine many perspectives of the individual sources of the environment data U with one another and therefore view the traffic situation in the real environment 11 from above from a kind of helicopter perspective. On this information basis, the server device 1 can calculate the instruction data A2 as action instruction to the motor vehicle 2. The motor vehicle 2 can thus drive particularly anticipatorily in autonomous driving operation by the instruction data A2.

[0052] One example of the method of coordinating a plurality of road users 2, 3 is shown in FIG. 2.

[0053] FIG. 2 shows one example of the update of an action instruction during the coordination of road users in road traffic in a real environment 11. The process shows two road users 2 and 3, wherein one road user is an autonomously driving motor vehicle 2 having the control circuit 10 and the sensor circuit 13 and the other road user is a visually impaired person 3 carrying, as assistance for orientation, a user device 9 having an integrated control circuit 10. By way of example, the user device 9 having the corresponding control circuit 10 can be designed as a smartphone with a corresponding application. This is referred to as user device 9 in the context of FIG. 2. Likewise, a road user denotes a unit which is a user of the presented method for coordinating road users and can thus receive an action instruction from the server device, and traffic object denotes a unit which takes on a participatory role in the traffic situation. Traffic objects can therefore be pedestrians, motor vehicles or cyclists.

[0054] The server device 1 creates a central model 4 of the real environment 11, which central model includes the road users, the motor vehicle 2 and the person 3, in this example. Furthermore, a cyclist as traffic object 7 is known to the server device 1 from the environment data U. Said cyclist may for example have been registered by a surveillance camera of a building in the real environment 11 or by a further road user, for example by the motor vehicle 2 as delta data, and be incorporated in the environment data U. Delta data can also be part of environment data U. The geoposition of the person 3 having the user device 9 is known to the server device 1. The motor vehicle 2 is moving on the priority road across the intersection in the real environment 11, which is indicated by a direction arrow.

[0055] In autonomous driving operation of the motor vehicle 2, the destination and the route are generally known. For the action instructions of autonomous driving operation, the method provides for the motor vehicle 2 to travel along the route in the real environment virtually in the central model 4. In this case, the motor vehicle 2 travels along the route in the virtual environment of the central model 4 in a manner ahead by a latency value in comparison with the real environment 11. Before the motor vehicle 2 reaches the real environment 11, the action instructions are calculated as instruction data A2 by the server device 1 for the motor vehicle 2 and are transmitted to the control circuit 10 of the motor vehicle 2. Likewise, before the motor vehicle 2 reaches the real environment 11, data of the central model 4 of the environment 11 are transmitted to the control circuit 10. This has the advantage that already concrete action instructions for the motor vehicle 2 in the real environment 11 are already present and the motor vehicle 2 therefore “subconsciously” knows, i.e. it is known in the motor vehicle 2 which actions said motor vehicle has to carry out in the environment 11. The motor vehicle 2 thus has a kind of artificial subconscious.

[0056] In the example shown, the original instruction data A2 according to the traffic situation in the central model 4 include as action instruction the fact that the motor vehicle 2 is intended to move analogously straight ahead along the priority road and is intended to stop at the right-hand pedestrian crossing in order that the person 3 behind the building 8, which is an advertising column in this example, can walk across the right-hand pedestrian crossing. The direction of the person 3 can be extrapolated by the server device on the basis of the person's geoposition. By way of example, on the basis of the last three geopositions of the person 3 the server device 1 can deduce that with a high probability the person 3 would like to cross the pedestrian crossing 12. Alternatively, the server device can assume that the person 3 would like to cross the pedestrian crossing 12 if the person 3 is situated within a minimum distance from the pedestrian crossing 12.

[0057] Since the person 3 is situated behind the building 8, the person is not visible from the perspective of the motor vehicle 2. The instruction data A2 can therefore be calculated by the server device 1 from a macroperspective since the server device 1, by the central model 4, can have a kind of helicopter view of the traffic situation in the real environment 11.

[0058] In some instances, however, the server device 1 does not represent the complete traffic situation in the real environment 11. The central model 4 can thus be supplemented by a local model 5 of the motor vehicle 2. When the motor vehicle 2 reaches the real environment 11, the motor vehicle 2, by onboard hardware connected to the control circuit 10, can create from its own perspective a local model 5 of the real environment 11 from an aggregate of sensor data of an onboard sensor circuit 13. The onboard sensor circuit 13 can comprise a camera system and/or a radar sensor system, for example. Furthermore the onboard sensor data of the motor vehicle 2 can comprise a quantity of vehicle state data, such as, for example, geoposition, steering angle or current vehicle speed.

[0059] From the onboard sensor data of the sensor circuit 13 the motor vehicle 2, by onboard hardware, can create a local model 5 of the real environment 11 in an analogous manner to the central model 4. In this case, additional traffic objects 7′ in the real environment 11 are visible from the perspective of the motor vehicle 2 in the local model 5, which were not visible in the central model 4 of the server device 1. In this regard, a motor vehicle 7′ in the middle of the intersection, a pedestrian 7′ crossing the road and a further cyclist 7′ as further traffic objects 7′ additionally appear in the local model 5 in contrast to the central model 4. The person 3 is not present in the local model 5 since the person is not visible from the perspective of the motor vehicle 2 because the person 3 is concealed by the building 8. This likewise applies to the cyclist 7 as traffic object in the central model 4 because said cyclist is still too far away from the intersection on the side road. From the local model 5 and the central model 4 with respect to the real environment 11, the control circuit can now determine delta data D for example as a difference between the local model 5 and the central model 4 and can transmit them to the server device 1. In this example, the delta data D comprise the additional traffic objects 7′.

[0060] The server device 1 updates the central model 4 with the delta data D and the environment data U and creates an updated central model 6, in which there appear the traffic objects 7 and 7′, and also the motor vehicle 2 and the person 3 as road users. On the basis of the updated central model 6, the server device 1 uses a predefined correction criterion to decide whether the action instructions for the road users 2 and 3 are in need of correction.

[0061] For the person 3 who would like to cross the pedestrian crossing 12, for example, the correction criterion may be undershooting of a distance between the cyclist on the right in the updated central model 6 as traffic object 7′ and the pedestrian crossing 12, which can cause the server device 1 to transmit instruction data A3 to the person 3, said instruction data including an acoustic and/or haptic warning message as action instruction. The instruction data A3 can be received by the user device 9 of the person 3 and the corresponding action instruction can be output as an acoustic and/or haptic output, for example as vibration. Particularly for visually impaired persons, this may be a supplementation for increasing safety in road traffic.

[0062] For the motor vehicle 2 it is likewise possible to transmit updated instruction data A2N on the basis of a correction criterion. In the case of the motor vehicle 2 this involves a readjusted action instruction, which can be a delta with respect to the previous instruction data A2. In this regard, a further motor vehicle 7′ as traffic object additionally appears in the image of the updated central model 6, which would result in undershooting of a minimum distance between the motor vehicle 2 and the motor vehicle 7′ on the basis of the previous instruction data A2. It is likewise necessary to reckon with the cyclist 7′ turning off to the left. Since the motor vehicle 7′ as traffic object can be assigned to an object category of motor vehicle and the cyclist 7′ as traffic object can be assigned to the object category of cyclist by the server device 1, the updating of the geoposition in a time interval and thus the respective observation intensity can be intensified by the server device 1 for the updated instruction data A2N. By way of example, in a predefined time interval, the updating of the geoposition of the motor vehicle 7′ relative to the cyclist 7′ can be prioritized firstly according to the object category and secondly according to the distance with respect to the motor vehicle 2. By way of example, pedestrian, motor vehicle and cyclist are conceivable as object category for the traffic objects 7 and 7′. In the aforementioned example, a priority may appear as follows: the motor vehicle 7′ is observed the most accurately by the server device 1 because it is at the shortest distance from the motor vehicle 2. The server device 1 observes the pedestrian 7′ next because he/she is at the next closest distance from the motor vehicle 2, and finally the cyclist 7 and 7′ since they are the furthest away from the motor vehicle 2.

[0063] For the updating of the instruction data A2 for the motor vehicle 2, consideration is given to undershooting of a minimum distance between the motor vehicle 2 and the motor vehicle 7′ and also a priority of the pedestrian 7′ when crossing the road on the pedestrian crossing 12. The instruction data can therefore be readjusted by the server device 1 with a braking maneuver and a reduced speed in order to avoid a collision between the motor vehicle 2 and the motor vehicle 7′ and the pedestrian 7′. The updated instruction data A2N are subsequently transmitted to the control circuit 10 of the motor vehicle 2, which implements the updated instruction data A2N as an updated action instruction. The aforementioned process is carried out repeatedly in traffic events.

[0064] In accordance with one aspect, therefore, the examples comprise a calculation and evaluation model analogous to human perception: the human “knows”, i.e. is aware of, his/her environment. The human registers and evaluates only the delta (difference or deviation from the model or the expectation). For an autonomous approach in the final development stage this means: with respect to a current (i.e. a few moments ago) environment captured and/or calculated locally (i.e. externally with respect to the backend/server device), the vehicle supplies to the backend delta information focused and/or concentrated on at least one region. Said backend supplements the precalculated scenario and checks the influence that this change has on the scenario already evaluated, and sends corresponding measures back to the vehicle. This “environment calculated externally a few moments ago” is comparable to the human subconscious (cf. the book by the author D. Kahneman: “Thinking, fast and slow”, ISBN: 9780141033570). This environment is calculated as “Model in the Middle” or “Digital Twin” and, besides taking into account the described environment data that are supplied directly to the backend (without a vehicle “detour”), also takes into account—if technically realizable—sensor data from vehicles which went past the actual object at the same location a few moments ago, and thus affords the possibility of not having to calculate and evaluate things in real time.

[0065] Present-day sensors in the vehicle may already suffice to implement the concept, and so it is possible to have recourse here to sensor circuits from the prior art.

[0066] Overall, this results in an avoidance of complexity in onboard hardware/onboard software of a motor vehicle, reduced consumption (e.g. as a result of reduced energy demand, lower vehicle curb weight), smaller amounts of data (significantly reduced demand for transmission capacity to the backend—particularly with regard to latency and bandwidth), lower costs in the case of damage (e.g. damage class, warranty), higher attractiveness of “nearly new vehicles” (i.e. increase in resell attractiveness).

[0067] In a computer environment (server device), the so-called backend, thus keeps environment information that is almost up to date in real time (for example, with an age of at most 10 minutes) for “the subconscious”. This information is kept up to date by triggers to be defined, e.g. according to the push and/or pull principle, from sources defined in advance. Graded according to the expected change frequency: streets/buildings/signaling systems and on the other hand weather data vs. road users on foot and on the other hand cycling road users and on the other hand road users (vehicles). Optionally focused on repeatedly traveled routes (which can be signaled by subscription data). The vehicle transmits position and may transmit predetermined further state data, and also delta data classified as relevant (e.g. road users, such as e.g. a vehicle), to the backend. The backend analyzes these changes vis-à-vis the “subconscious” already present and evaluates measures to be proposed, optionally also only adjustment of already transmitted and “preloaded” measures before the actual implementation by the vehicle (further reduction of the data to be transmitted in real time).

[0068] The examples related to a method of coordinating road users via a server device, which determines instruction data by a central model and transmits the instruction data as an action instruction to a control circuit of the respective road users. The central model is a virtual representation of a real environment and is created and/or updated by the server device from environment data and delta data of the road users. When the road user reaches the real environment, it creates a local model using onboard sensor data and determines delta data relating to the central model. The delta data are transmitted from the control circuit of the respective road user to the server device, which creates an updated central model using the delta data and the central model. The server device determines and then transmits updated instruction data as a readjustment of the instruction data to the road users.

[0069] Overall, the examples show how an external precalculation and pre-evaluation of environment scenarios by individual motor vehicles can be provided for a central mapping system.

[0070] A description has been provided with particular reference to examples, but it will be understood that variations and modifications can be effected within the spirit and scope of the claims which may include the phrase “at least one of A, B and C” as an alternative expression that means one or more of A, B and C may be used, contrary to the holding in Superguide v. DIRECTV, 358 F3d 870, 69 USPQ2d 1865 (Fed. Cir. 2004).