Method and System for Automatized Selection and Documentation of Personal Highlights During Journeys with a Motor Vehicle

Abstract

An embodiment method for automatized selection and documentation of personal highlights during journeys with a vehicle includes capturing journey data during a journey of the vehicle, the journey data including audio, image, or video data taken of an interior or an exterior of the vehicle, storing selected aspects of the journey data to a non-transient data storage in response to a trigger criteria being met during the journey in an associated driving situation, and generating a selection of personal journey highlights based on the selected aspects stored during the journey for presentation during or after the journey.

Claims

1. A method for automatized selection and documentation of personal highlights during journeys with a vehicle, the method comprising: capturing journey data during a journey of the vehicle, the journey data comprising audio, image, or video data taken of an interior or an exterior of the vehicle; storing selected aspects of the journey data to a non-transient data storage in response to a trigger criteria being met during the journey in an associated driving situation; and generating a selection of personal journey highlights based on the selected aspects stored during the journey for presentation during or after the journey.

2. The method according to claim 1, wherein the journey data is captured continuously during the journey by a vehicle sensor system of the vehicle, and wherein the selected aspects of the journey data comprise selected audio or video streams of limited length or selected images documenting the associated driving situation.

3. The method according to claim 1, wherein the journey data is retrieved from a personal electronic device of an occupant of the vehicle.

4. The method according to claim 1, wherein the trigger criteria is set based on pre-established or user defined content criteria.

5. The method according to claim 4, wherein the trigger criteria is successively refined based on machine learning algorithms taking user preferences or user interactions into account.

6. The method according to claim 1, wherein the trigger criteria comprises an occurrence of passing of points of interest along a route of the vehicle, notable traffic or driving situations involving the vehicle or happening near the vehicle, notable behavior of an occupant of the vehicle, notable interactions between multiple occupants of the vehicle, notable interactions between the occupant of the vehicle and the vehicle, trigger signals from a personal electronic device of the occupant, or ingress or egress of the occupant of the vehicle.

7. The method according to claim 1, wherein the journey data comprises a driving history of the vehicle, wherein the driving history comprises navigation data, environmental data, driving parameters, or status data of the vehicle, and wherein the selected aspects of the journey data are stored together with associated aspects of the driving history.

8. The method according to claim 1, wherein the journey data having sensitive content is blocked from storage.

9. The method according to claim 1, wherein the journey data comprises external data captured by other vehicles in an area of the vehicle during a respective driving situation, and wherein the method further comprises: wirelessly receiving the external data from the other vehicles and considering the external data for the selected aspects of the journey data; or wirelessly sharing the selected aspects of the journey data or data about user preferences or user interactions with the other vehicles.

10. A system for automatized selection and documentation of personal highlights during journeys with a vehicle, the system comprising: a control device configured to: access journey data captured during a journey of the vehicle, the journey data comprising audio, image, or video data taken of an interior or an exterior of the vehicle; store selected aspects of the journey data to a non-transient data storage in response to a trigger criteria being met during the journey in an associated driving situation; and generate a selection of personal journey highlights based on the selected aspects stored during the journey for presentation during or after the journey.

11. The system according to claim 10, further comprising a vehicle sensor system configured to capture the journey data continuously during the journey, wherein the selected aspects of the journey data comprise selected audio or video streams of limited length or selected images documenting the associated driving situation.

12. The system according to claim 10, wherein the control device is configured to retrieve the journey data from a personal electronic device of an occupant of the vehicle.

13. The system according to claim 10, wherein the control device is configured to set the trigger criteria based on pre-established or user defined content criteria.

14. The system according to claim 10, wherein the control device is configured to successively refine the trigger criteria based on machine learning algorithms taking user preferences or user interactions into account.

15. The system according to claim 10, wherein the trigger criteria comprises an occurrence of passing of points of interest along a route of the vehicle, notable traffic or driving situations involving the vehicle or happening in an area of the vehicle, notable behavior of an occupant of the vehicle, notable interactions between multiple occupants of the vehicle, notable interactions between the occupant of the vehicle and the vehicle, trigger signals from a personal electronic device of the occupant, or ingress or egress of the occupant of the vehicle.

16. The system according to claim 10, wherein the journey data comprises a driving history of the vehicle, wherein the driving history comprises navigation data, environmental data, driving parameters, or status data of the vehicle, and wherein the control device is configured to store the selected aspects of the journey data together with associated aspects of the driving history.

17. The system according to claim 10, wherein the control device is configured to block the journey data having sensitive content from storage.

18. The system according to claim 10, wherein the journey data comprises external data captured by other vehicles in an area of the vehicle during a respective driving situation, and wherein the control device is configured to: wirelessly receive the external data from the other vehicles and to consider the external data for the selected aspects of the journey data; or wirelessly share the selected aspects of the journey data or data about user preferences or user interactions with the other vehicles.

19. A vehicle comprising: a vehicle sensor system; and a control device configured to: access journey data captured during a journey of the vehicle, the journey data comprising audio, image, or video data taken of an interior or an exterior of the vehicle; store selected aspects of the journey data to a non-transient data storage in response to a trigger criteria being met during the journey in an associated driving situation; and generate a selection of personal journey highlights based on the selected aspects stored during the journey for presentation during or after the journey.

20. The vehicle according to claim 19, wherein the vehicle sensor system is configured to capture the journey data continuously during the journey, and wherein the selected aspects of the journey data comprise selected audio or video streams of limited length or selected images documenting the associated driving situation.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are included to provide a further understanding of embodiments of the present invention and are incorporated in and constitute a part of this specification. The drawings illustrate the embodiments of the present invention and together with the description serve to explain the principles of the invention. Other embodiments of the present invention and many of the intended advantages of embodiments of the present invention will be readily appreciated as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale relative to each other. In the figures, like reference numerals denote like or functionally like components, unless indicated otherwise.

[0040] FIG. 1 schematically depicts a motor vehicle equipped with a system for selection and documentation of personal highlights according to an embodiment of the invention.

[0041] FIG. 2 schematically shows a flow diagram of a method for automatized selection and documentation of personal highlights during journeys with the motor vehicle of FIG. 1.

[0042] Although specific embodiments are illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a variety of alternate and/or equivalent implementations may be substituted for the specific embodiments shown and described without departing from the scope of the present invention. Generally, this application is intended to cover any adaptations or variations of the specific embodiments discussed herein.

[0043] The following reference identifiers may be used in connection with the accompanying drawings to describe exemplary embodiments of the present disclosure. [0044] 1 system for selection and documentation of personal highlights [0045] 2 vehicle sensor system [0046] 2a interior camera [0047] 2b exterior camera [0048] 2C microphone [0049] 3 control device [0050] 4 non-transient data storage [0051] 5 driver interface [0052] 6 journey data [0053] 7 driving history [0054] 8 communication unit [0055] 9 machine learning algorithm [0056] 10 motor vehicle [0057] 11 other vehicles/vehicle fleet [0058] M method [0059] M1-M3 method steps

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

[0060] FIG. 1 schematically depicts a motor vehicle 10 equipped with a system 1 for selection and documentation of personal highlights according to an embodiment of the invention. FIG. 2 schematically shows a flow diagram of a method M for automatized selection and documentation of personal highlights during journeys with the motor vehicle 10 of FIG. 1.

[0061] The present method M and system 1 are provided to automatize the selection and documentation of highlights during car-related journeys. Conventionally, vehicle occupants need to take pictures/videos and audio recordings manually and then need to look through all recorded information later on to select and sort out any particularly worthwhile information for long-term storage, e.g., to produce a selection of highlights from a particular vacation. With the present system 1, integrated vehicle systems are used to automatically check if a current driving situation inside and/or around the motor vehicle 10 has the potential for a notable event worthwhile to be recorded. Only in case certain trigger conditions are met, audio and/or visual information are stored as part of the highlights of the respective journey and are then automatically selected for presentation to the users, who then can decide on the basis of these pre-selected data whether they want to keep them or not, whether to upload them to social media, etc. By adequately training and adjusting the configuration of the pre-selection, users may not even need to further select the acquired material at all anymore.

[0062] To this end, the present system 1 comprises a vehicle sensor system 2 configured to capture audio, image and/or video data of an interior and/or an exterior of the motor vehicle 10 during a journey of the motor vehicle 10. In the exemplary embodiment of FIG. 1, the sensor system 2 comprises an interior camera 2a, an exterior camera 2b, and a microphone 2C. It is to be understood of course that the sensor system 2 may comprise various other sensors and sensor technologies comprising amongst others ultrasound, radar, laser, lidar and so on. The sensor system 2 may also be communicatively coupled to and/or integrated in further vehicle systems, e.g., an assisted/autonomous driving system, an infotainment system, a navigation system and so on, and may thus also rely on the data acquired by and/or available to these systems.

[0063] The vehicle sensor system 2 may be particularly configured to capture the journey data 6 continuously during the journey. The data may be temporarily saved in a transient data storage, e.g., a memory, and be continuously overwritten there. Only in case certain requirements are fulfilled the data may then be transferred to a non-transient data storage for mid-term (e.g., until the end of the journey) or long-term storage.

[0064] Consequently, the system 1 further comprises a control device 3 configured to store selected aspects of the journey data 6 to a non-transient data storage 4 in case at least one of several given trigger criteria is met during the journey in an associated driving situation. The non-transient data storage 4 may be integrated in the vehicle 10 itself. However, also distributed and/or cloud storage solutions may be employed for the present purpose. To this end, the vehicle 10 may be provided with a communication unit 8 that provides a wireless connection to the corresponding data networks.

[0065] The system 1 then is adapted to generate a selection of personal journey highlights based on the selected aspects stored during the journey for presentation to a travel passenger (also including the driver) of the motor vehicle 10 during and/or after the journey, e.g., on a driver interface 5 as shown in FIG. 1. These selected aspects of the journey data 6 may comprise amongst others selected audio and/or video streams of limited length and/or selected images documenting the associated driving situation.

[0066] Additionally, or alternatively, the journey data may also be taken from personal electronic devices of the travel passengers of the motor vehicle. In that case, the control device 3 may access the respective devices of the occupants, e.g., a smartphone or the like.

[0067] The corresponding method M shown in FIG. 2 thus comprises under M1 capturing the journey data 6 with the vehicle sensor system 2, under M2 storing the selected aspects of the journey data 6 to the non-transient data storage 4 in case at least one of several given trigger criteria is met during the journey in an associated driving situation, and under M3 generating the selection of personal journey highlights based on the selected aspects stored during the journey for presentation to a travel passenger of the motor vehicle 10 during and/or after the journey.

[0068] The journey data 6 may further comprise a driving history 7 of the motor vehicle 10 comprising, for example, navigation data (GPS coordinates, routes, etc.), environmental data (e.g., weather information, etc.), driving parameters (acceleration, speed profiles, etc.) and status data (sports mode, autonomous mode, etc.) of the motor vehicle 10. The selected aspects of the journey data 6 may be stored together with associated aspects of the driving history 7. For example, information about a passed point of interest may be stored together with the audio and visual information captured during the passing. Moreover, the recorded audio and/or visual information may be accompanied by further entertainment data, e.g., background music and/or sound effects in case of a video (e.g., of a video recorded during sports mode).

[0069] The trigger criteria may, for example, comprise the occurrence of at least one of the following during the journey: passing of points of interest along a route of the motor vehicle 10 (e.g., a scenic view, a certain building or monument, etc.); notable traffic and/or driving situations involving the motor vehicle 10 or happening in the vicinity of the motor vehicle 10 (e.g., an exotic car passing, cornering, rough off-road driving, a spectacular and/or thrilling route with cliffs, bridges, rivers, etc.); notable behavior of a travel passenger of the motor vehicle 10 (e.g., telling of a joke or of a story, laughter, singing, etc.); notable interactions between travel passengers of the motor vehicle 10 (e.g., singing, laughter, discussions, etc.); notable interactions between the travel passengers of the motor vehicle and the motor vehicle (e.g., activation of sports driving mode, opening of a sunroof, etc.); trigger signals from personal devices of the travel passengers (e.g., signals from personal electronic devices like smartphones, smartwatches, etc.); and ingress and/or egress of travel passengers of the motor vehicle 10.

[0070] However, certain sensitive contents may also be selectively blocked from storage under specific circumstances, e.g., visual data in case of an accident.

[0071] The trigger criteria may be set based on pre-established and/or user defined content criteria, e.g., certain topics of interest. At the end of the journey, the selected data may then be summarized, e.g., in the form of a video clip, and may be uploaded to various social media or personal data repositories. Users may also be able to decide whether to keep certain elements, edit them, and/or delete them. The trigger criteria may also be successively refined based on machine learning algorithms 9 taking such user preferences and/or user interaction into account.

[0072] The journey data may be supplemented and/or substituted by wirelessly received external data captured by other motor vehicles 11 in the vicinity of the motor vehicle 10 during a respective driving situation. These external data may then be considered for the selected aspects of the journey data 6.

[0073] In a similar vein, the present vehicle 10 may wirelessly share aspects of the journey data 6 and/or data about user preferences and/or user interaction with other motor vehicles 11. The vehicles 10, 11 may thus share any relevant information, which may be particularly used to further optimize the data selection and documentation process (e.g., with regards to the employed machine learning algorithms 9).

[0074] In the foregoing detailed description, various features are grouped together in one or more examples with the purpose of streamlining the disclosure. It is to be understood that the above description is intended to be illustrative, and not restrictive. It is intended to cover all alternatives, modifications, and equivalents of the different features and embodiments. Many other examples will be apparent to one skilled in the art upon reviewing the above specification. The embodiments were chosen and described in order to explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated.