Responsive human machine interface

Abstract

The human machine interface (HMI) of a vehicle, especially an automobile is responsive to outside stimuli. Signals relevant to the amount of concentration required by a driver are received by the HMI, which determines, based on the values of these signals, a driver distraction level. As the driver distraction level increases, areas of a structured display on a display screen are dynamically removed, to present the driver with a reduced distractions when greater concentration is required.

Claims

1. An automobile comprising a human machine interface, the human machine interface comprising at least one display screen configured to simultaneously display a plurality of areas as a structured display; the human machine interface operable to determine a driver distraction level based on at least one signal received by the human machine interface and to remove predetermined areas from the plurality of areas of the structured display on the screen in response to an increase in the driver distraction level beyond a predetermined threshold; wherein the human machine interface is operable to remove areas from the structured display dynamically, such that predetermined areas gradually recede from the screen, appear to move off the edge of the plurality of areas of the structured display, or become covered by other areas of the screen.

2. The automobile comprising a human machine interface according to claim 1 which is operable to increase the plurality of areas remaining on the structured display as predetermined areas are removed and/or wherein one or more of the plurality of areas are selectable areas and selectable areas are removed from the screen.

3. The automobile comprising a human machine interface according to claim 1 wherein at least one of the plurality of areas comprises an image of a navigation instruction and wherein the human machine interface is configured to increase a size of the area comprising the navigation instruction on the structured display in response to an increase in the driver distraction level beyond the predetermined threshold.

4. The automobile comprising a human machine interface according to claim 1 wherein the driver distraction level is determined based on a speed of travel of the vehicle.

5. The automobile comprising a human machine interface according to claim 1 wherein the driver distraction level is calculated based on signals indicative of one or more of the following factors, each factor optionally being weighted: speed, driver drowsiness, road condition, traffic conditions, navigation data, upcoming manoeuvres, status of Autonomous Cruise Control (ACC), status of Automatic Emergency Braking system (AEB), status of automatic Lane Keeping System (LKS), and/or Lane Departure Warning system (LDW), status of a Traffic Pilot, or Auto Pilot, telephone status, radio/video status and/or volume, and/or based on the state of driving controls.

6. The automobile comprising a human machine interface according to claim 5 in which driver distraction level is calculated based on the state of driving controls and each factor is weighted, and which is operable such that when it is determined that a gear lever is in neutral and/or a hand brake is activated, the driver distraction level is set to zero regardless of other factors.

7. The automobile comprising a human machine interface according to claim 1 which is operable to receive a signal indicative of road condition from sensors on a vehicle, and to calculate the driver distraction level based on a road condition value obtained from the sensors.

8. The automobile comprising a human machine interface according to claim 1 which is operable to receive a signal comprising navigation data and/or upcoming manoeuvres from a satellite navigation system and to calculate the driver distraction level based on a value indicative of the difficulty and/or proximity of one or more upcoming manoeuvres.

9. The automobile comprising a human machine interface according to claim 1 which is operable to receive a signal indicating the status of one or more safety systems from the vehicle's central computer, or Engine Control Unit (ECU) and to calculate the driver distraction level based on a value associated with the status of the one or more safety systems.

10. The automobile comprising a human machine interface according to claim 5 which is operable to calculate the driver distraction level by assigning a value based on the level of one or more of the factors and applying a weighting to the factor.

11. The automobile comprising a human machine interface according to claim 1 wherein the human machine interface dynamically removes areas from the screen in response to an increase in the driver distraction level; wherein the signal indicative of the driver distraction level is indicative of the speed of the vehicle and a distance until a next navigation manoeuvre; the display screen being configured to simultaneously display an area comprising an image of navigation instructions, an area comprising an image of a map and one or more selectable areas, wherein the human machine interface is operable such that as speed increases and as the distance until the next manoeuvre decreases past a first predetermined threshold, the display screen is operable to remove one or more selectable areas from the structured display on the screen, then as a second predetermined threshold is passed, the human machine interface is operable to increase the size of the area comprising navigation instructions, then, as a third predetermined threshold is passed, the human machine interface is operable to remove the area comprising the image of the map from the screen such that only the image comprising a navigation instruction remains and its size is significantly increased.

12. The automobile comprising a human machine interface according to claim 11 which is operable to only remove the area comprising an image of the map from the structured display when it is determined that the automobile is traveling above a certain predetermined threshold speed and within a certain predetermined distance of the next navigation manoeuvre.

13. The automobile comprising a human machine interface according to claim 1 wherein the areas recede from the screen by fading away, and moving off the visible area of the screen.

14. The automobile comprising a human machine interface according to claim 1 wherein as speed, or driver distraction level, gradually increases, so predetermined areas gradually recede from the screen.

15. The automobile comprising a human machine interface according to claim 14 wherein the predetermined areas include first selectable areas which gradually recede from the screen and at a certain predetermined speed, or at a certain predetermined driver distraction level, are no longer selectable.

16. The automobile comprising a human machine interface according to claim 15 wherein once certain first predetermined selectable areas have receded and are no longer selectable, as speed, or driver distraction level increases, so second predetermined areas recede, and when the speed or driver distraction level reaches a second predetermined speed, the second predetermined areas are no longer selectable either.

17. The automobile comprising a human machine interface according to claim 16 wherein once the second predetermined selectable areas are no longer selectable, one or more third predetermined area recedes, until at a certain predetermined speed or a certain predetermined driver distraction level, only a predetermined minimum level of information is provided on the screen.

18. The automobile comprising a human machine interface according to claim 14 wherein the extent to which the areas recede is directly proportional to the speed of the vehicle, and/or wherein the areas return to the screen as speed, or driver distraction level, reduces, gradually appearing, and optionally becoming selectable at the same predetermined threshold speed, or driver distraction level, at which they stopped being selectable.

19. The automobile comprising a human machine interface according to claim 15 wherein the selectable areas gradually move off the visible area of the screen and begin to fade at the moment that they are no longer selectable; or gradually fade and begin to move of the screen at the point that they are no longer selectable.

20. A method of controlling a human machine interface in the automobile according to claim 1, the method comprising: a. receiving at least one signal relating to a factor relevant to the amount of concentration required of a driver; b. calculating the driver distraction level based on the at least one signal; c. determining whether the driver distraction level is above a predetermined threshold; and d. if the driver distraction level is determined to exceed a predetermined threshold, controlling the display screen so as to dynamically remove predetermined areas from the structured display on the display screen.

21. A human machine interface for a vehicle, the human machine interface comprising at least one display screen configured to simultaneously display a plurality of areas as a structured display; the human machine interface operable to determine a driver distraction level based on at least one signal received by the human machine interface and to remove predetermined areas from the plurality of areas of the structured display on the screen in response to an increase in the driver distraction level beyond a predetermined threshold; wherein the human machine interface is operable to remove areas from the structured display dynamically, such that predetermined areas gradually recede from the screen, appear to move off the edge of the plurality of areas of the structured display, or become covered by other areas of the screen.

Description

DETAILED DESCRIPTION OF THE INVENTION

(1) In order that the invention may be more clearly understood embodiments thereof will now be described, by way of example only, with reference to the accompanying drawings, of which:

(2) FIG. 1 shows an automobile incorporating a Human Machine Interface (HMI) according to the invention;

(3) FIG. 2 shows schematically the HMI of FIG. 1 and its connections to apparatus in the automobile;

(4) FIG. 3 shows a first embodiment of an arrangement of images on the display screen of the HMI of FIG. 1 in one configuration;

(5) FIG. 4 shows the display screen of the HMI of FIG. 3 in another configuration;

(6) FIG. 5 shows the display screen of the HMI of FIG. 3 in a third configuration;

(7) FIG. 6 shows a second embodiment of the arrangement of images on the display screen of the HMI of FIG. 1 in a first configuration;

(8) FIG. 7 shows the display screen of the HMI of FIG. 6 in a second configuration;

(9) FIG. 8 shows the display screen of the HMI of FIG. 6 in a third configuration;

(10) FIG. 9 shows the display screen of the HMI of FIG. 6 in a fourth configuration; and

(11) FIG. 10 shows a graph of visibility and selectability as a function of speed.

(12) Referring to FIG. 1, an automobile 1 comprises a Human Machine Interface 2 shown in more detail in FIG. 2. The HMI 2 comprises a display screen 3 mounted in interior of the automobile 1 so as to be visible to a driver, a user actuable input device 4, such as a rotary controller, or touch sensitive screen covering the display screen 3, a processor 5 which receives input signals from the input device 4 and outputs display signals to the display screen 3. The HMI 2 also comprises memory 6, which the processor may read from and store to and a distraction level determination module 7, which is also able to read data from, and store data to, the memory 6.

(13) The processor 5 is also able to receive and process signals from outside the HMI 2, which are received by a communications unit 14. In particular, the processor 5 is arranged to receive, from the communications unit 14 signals from a satellite navigation module 8, a vehicle speed sensor 9, a driver drowsiness estimating apparatus 10, a road condition sensor 11 (e.g. a sensor or combination of sensors arranged to measure wetness and temperature), and from the automobile's ECU 12.

(14) Those skilled in the art will appreciate that the processor 5 need not receive all of these signals directly from the apparatuses 8, 9, 10, 11, 12 defined, for example, the ECU alone may provide signals indicative of the vehicle's speed, or road conditions, indeed, an indicative speed could even be included in the signal from the satellite navigation module 8. It will also be appreciated that receiving modules (not shown) etc. may be introduced to interface with the various external apparatuses 8-12 to receive and optionally format the signals before they are passed to the communications unit 14.

(15) The processor 5 is arranged to request and subsequently process signals received from the various external apparatuses 8-12 and send them to the distraction level determination module 7 and in response to the distraction level determined by the distraction level module 7 and the input from the input device 4, to output a display to the display device.

(16) Of course, the processor 5 is also responsible for processing and forwarding commands from the input device 4 to further external hardware 13 such as radio or climate control devices etc., via the communications unit 14 in the HMI 2 (which will also feed back signals to the processor concerning the devices it controls), and optionally via the automobile's ECU 12.

(17) According to the method of an embodiment of the invention, the processor 5 receives input signals from the external apparatus 8-12 and the internal control unit 14 as set out in table 1 below:

(18) TABLE-US-00001 TABLE 1 Satellite navigation module 8 Traffic conditions Navigation data Upcoming manoeuvres Speed sensor 9 Vehicle speed Drowsiness estimating Driver drowsiness level apparatus 10 Road condition sensor 11 Road condition ECU 12 Autonomous Cruise Control (ACC) status Automatic Emergency Braking system (AEB) status Automatic Lane Keeping System (LKS) status Lane Departure Warning system (LDW) status Traffic Pilot status Auto Pilot status HMI Control Unit 14 Telephone status (e.g. on/off/active call) Radio status Video status Volume

(19) It will be appreciated that the signals may be provided by other devices in the vehicle, for example, the satellite navigation module may be able to provide information on the status of the radio, and the ECU or instrument cluster may provide speed information.

(20) The signals are processed and sent to the distraction level determination module 7, which determines based on the value of these signals, the distraction level.

(21) Of course various methods may be used to weight the different results, to conditionally take certain values into account and so on, and such methods may be optimised with improved algorithms, but as an exemplary specific embodiment, distraction level determination module 7 determines the distraction level by assigning weighted values to the results of the input signals as shown in table 2 below:

(22) TABLE-US-00002 TABLE 2 Traffic conditions Clear = 0 Moderate = 5 Busy = 10 Navigation data Straight road = 0 Moderate curves = 5 Very curvy = 10 Upcoming manoeuvres None within 2 minutes = 0 Easy manoeuvre within 2 minutes = 1 Easy manoeuvre within 30 seconds = 5 Complex manoeuvre within 2 minutes = 5 Complex manoeuvre within 30 seconds = 10 Vehicle speed 0 Kph = 1000 0-20 kph = 0 20-50 kph = 5 50-90 kph = 10 90-130 kph = 20 130-200 kph = 50 200-300 kph = 90 300+ kph = 1000 Driver drowsiness level Fully awake = 0 Moderately drowsy = 10 Drowsy = 30 Road condition Dry = 0 Wet = 5 Foggy = 30 Potentially Icy = 30 Autonomous Cruise Control On = 0 (ACC) status Off = 5 Automatic Emergency On = 5 Braking system (AEB) status Off = 5 Automatic Lane Keeping On = 10 System (LKS) status Off = 5 Lane Departure Warning On = 5 system (LDW) status Off = 2 Time of day day = 0 Dawn/dusk = 10 Night = 10 Telephone status (e.g. Off = 0 off/active call/incoming Active call = 10 call/interaction with device) Incoming call = 20 Interaction = 30 Radio status Inactive = 0 Channel browsing = 30 Video status Off = 0 On = 20 Volume Off = 0 Quiet = 2 Loud = 10

(23) Once a value has been assigned to each factor, the distraction level determination module 7 sums the weighted values to produce a total distraction level, which is output to the processor to determine which images should be displayed on the screen and which to dynamically remove. This distraction level is constantly updated as new signals are received.

(24) The value assigned to the distraction level, based on this is used to determine which images are shown on the display screen 3. Thus the memory 6 stores information concerning which images to display on the display screen 3 dependent on the value of the distraction level. In one example, the memory 3 indicates that that the display screen 3 should be fully operational, with all the selectable images displayed, provided the distraction level is 0 or lower. Hence, when the vehicle is stationary, because the value of 1000 is assigned to the speed, regardless of the value of any other signals, because the speed=0, the value of the distraction level will be below 0 and therefore the display screen 3 will be fully operational, with all selectable items shown on the display screen 3.

(25) A first exemplary embodiment of a possible structured display, in which particular areas displaying particular information are arranged in particular places on a display screen 3, in accordance with the invention is described with reference to FIGS. 3-5. As shown in FIG. 3, the structured display on the display screen 3, when fully operational (e.g. when speed=0) comprises a rectangular display area 15, on which a circular information display area 16 is centrally disposed. Information, for example in relation to the mode currently chosen, e.g. radio, climate control, navigation system etc. is displayed in the information display area 16. To either side of the information display area 16, three low-distraction selectable areas 17 are situated, in a region which defines an oval around the circular central section. These low-distraction selectable areas 17 (six in total) are for selecting items which do not require a great deal of attention, for example, volume control, or temperature control. Outside the oval region, extending to the edges of the visible area of the display screen 3, the area on each side of the oval is divided into four high-distraction selectable areas 18 (eight in total). These high-distraction selectable areas 18 are for selecting items which may require significant concentration from the driver, for example inputting a destination to a satellite navigation system, making a telephone call, watching a video or altering the settings of the automobile 1.

(26) In the example, the memory 6 stores a reference table indicating that where the distraction level is between 1 and 20, the high distraction selectable areas 18 should be removed from the screen, but the low distraction selectable areas 17 may remain.

(27) Thus, for example, when the traffic conditions are clear (=0), the road is straight (=0), there are no upcoming maneuvers (=0), the vehicle is being driven at 40 kph (=5), by a driver who the apparatus 10 determines to be moderately drowsy (=10) on a road which the road condition sensor 11 indicates to be dry (=0), with the signal from the ECU 21 indicating that all the driver assistance systems (ACC, AEB, LKS, are turned on (=15), and with the telephone status being off with no active call (=0), the radio inactive (=0) with volume at a relatively low level (=2) and the video off (=2), the distraction level determination module 7 will determine a distraction level of 5, the total of the values for each factor that is being monitored and will output that value to the processor 5, the processor 5 will determine based on a comparison with the reference table stored in the memory 6 that the value has passed the predetermined threshold of 1, at which the high distraction selectable areas 18 should be removed from the screen and will dynamically remove the high distraction selectable areas 18 from the screen.

(28) This is done by altering the display signal sent from the processor 5 to the display screen 3, so as to show the central information display image 16 expanding laterally, to occupy an oval shaped central region as shown in FIG. 4, simultaneously, the outer high-distraction selectable areas 18 are deleted from the screen (a dynamic display can for example show them shrinking towards the edges of the visible area of the rectangular display area 15), and the six low-distraction selectable areas 17 are then displayed occupying six sectors (three either side of the oval central information display image 16).

(29) The processor 5 continuously, or periodically, sends updated signals to the distraction level determination module 7, to update the value of the distraction level and, if the level passes the second threshold of 20 and enters a second region stored in the reference table in the memory, of between 20 and 100, the display signal sent by the processor 5 is again modified to alter the image shown on the display screen 3.

(30) Thus, for example, if the traffic conditions remain clear (=0), there are no upcoming manoeuvres (=0), the vehicle is being driven by a driver who the apparatus 10 determines to be moderately drowsy (=10) on a road which the road condition sensor 11 indicates to be dry (=0), with the signal from the ECU 21 indicating that all the driver assistance systems (ACC, AEB, LKS,) are turned on (=15), (note that LKS and LDW are mutually exclusive) and with the telephone status being active call (=10), the radio inactive (=0) with volume at a relatively low level (=2) and the video off (=2), but the speed sensor determines that the speed is 140 kph (=30), the distraction level determination module 7 will determine a distraction level of 39, the total of the values for each factor that is being monitored.

(31) Accordingly the distraction level determination module 7 will output that value to the processor 5, the processor 5 will determine based on a comparison with the reference table stored in the memory 6 that the value has passed the second predetermined threshold of 20, at which the low distraction selectable areas 17 should be removed from the display screen 3 and will dynamically remove the low distraction selectable areas 17 from the screen 3 to display the image shown in FIG. 5, in which none of the selectable items are visible.

(32) In a further step, if the continuous/periodic monitoring of the signals that determine the distraction level continue to increase and pass a third threshold of 100, to enter a third region stored in the memory of between 100 and 2000, the processor 5 sends a signal to the display screen 3 modifying the image to show only a very basic image e.g. the next manoeuvre if satellite navigation is active, or no image at all. At this stage, the processor may send additional signals to the control unit 14 or ECU 12 to shut down any further potential distractions.

(33) As will be appreciated, in this example, this final, third threshold cannot be exceeded, since the highest possible value for the distraction level would be 1207 and the circumstances where that total would be reached are unrealistic, involving a speed in excess of 300 kph on a busy road with a rapidly approaching complex manoeuvre, potentially icy weather and whilst interacting with the telephone and radio. Of course, the final threshold of 100 will always be reached when travelling at very high speed (in excess of 300 kph) and can be relatively easily reached if travelling at over 200 kph with additional potential distractions.

(34) A second exemplary embodiment of a structured arrangement of images on a display screen is shown with reference to FIGS. 6-9. This embodiment relates in particular to an arrangement including a satellite navigation system and in which the only factor taken into account in determining the distraction level is the speed of the vehicle. As shown in FIG. 6, the structured display on the display screen 3, when fully operational (e.g. when speed=0) comprises a rectangular display area 15, on which, in the satellite navigation mode about one third of the screen, in the top left, comprises a rectangular navigation instruction image 19. This is a simple display of the next manoeuvre required, e.g. an arrow, optionally also indicating the distance in which the manoeuvre will be necessary. To the right of the navigation instruction image 19, in this embodiment, occupying most of the remaining two thirds of the screen, is a map image 20, which, as is conventional, may show the position of the vehicle on a map. In the top right hand corner, three low-distraction selectable areas 21 are disposed, overlaying the map image 20. Again, these low-distraction selectable areas 21 (three in total) are for selecting items which do not require a great deal of attention. In this particular arrangement (which those skilled in the art will appreciate is just exemplary) four high-distraction selectable areas 22 are located along the bottom of the display area 15. Again, the high-distraction selectable areas 22 are for selecting items which may require significant concentration from the driver, for example inputting a destination to a satellite navigation system, making a telephone call etc.

(35) In the second example, the memory 6 stores a reference table which correlates speed with distraction level. Thus, as shown in FIG. 10, as speed increases visibility of certain areas, shown in the figure by the line graph reduces, and, as shown in the figure by the bar chart, selectability, i.e. the function of the areas when selected, similarly reduces as speed increases. Hence, at between 0 and 10 kph, all of the display areas, 19, 20, 21 and 22 are completely visible and selectable. However, when the speed reaches 10 kph, the high distraction selectable areas 22 are no longer selectable and as shown in the visibility line, they start to become less visible, being gradually be removed from the screen and fading away, whilst low distraction selectable areas 21, map image 20 and navigation instruction image remain entirely visible.

(36) Thus, as speed is output to the processor 5, the processor 5 will determine based on a comparison with the reference table stored in the memory 6 that the value has passed the predetermined threshold of 10 kph, at which the high distraction selectable areas 22 should no longer be selectable and should be removed from the screen and will dynamically remove the high distraction selectable areas 22 from the screen, in response to further increase in speed, by outputting an image signal which causes the high distraction selectable areas 22 to appear to fade and move downwards off the visible area 15 of the screen 3 as speed increases.

(37) At the same time, with increasing speed, as shown in FIG. 7, the length of the navigation information area 19 and the map image 20 expand downwardly, to occupy the space where the high-distraction selectable areas 22 were formerly disposed.

(38) The processor 5 continuously sends updated signals to the distraction level determination module 7, to update the value of the distraction level (i.e. in this example the speed) and, if the level passes the second threshold of 100 kph and enters a second region stored in the reference table in the memory, of between 100 kph and 200 kph, in which the display signal sent by the processor 5 is again modified to alter the image shown on the display screen 3.

(39) Accordingly the distraction level determination module 7 will output a value to the processor 5, the processor 5 will determine based on a comparison with the reference table stored in the memory 6 that the value has passed the second predetermined threshold of 100 kph, at which the low distraction selectable areas 21 should no longer react to being selected and should also be removed from the structured display on the display screen 3 and again, in accordance with the graph shown in FIG. 10, as speed increases above 100 kph, the image signal will be output in order to dynamically remove the low distraction selectable areas 21 from the screen 3, by gradually moving them off the screen as speed increases and fading them away, such that once the speed reaches 200 kph the display screen 3 shows the image shown in FIG. 8, in which none of the selectable items are visible.

(40) In a further step, if the continuous monitoring of the signal that determines the distraction level (i.e. speed) continues to increase and the value passes a third threshold of 200 kph, to enter a third region stored in the memory of between 200 and 500 kph, the processor 5 sends a signal to the display screen 3 modifying the image to remove the map image 20 and expand the navigation information image 19 to occupy substantially the entire visible area on the display screen 3 as shown in FIG. 9. Once again, the map image is removed responsively to the speed, being gradually moved off the visible area of the screen and faded away as speed increases above 200 kph, until at 250 kph it is completely invisible. At this stage, the processor may send additional signals to the control unit 14 or ECU 12 to shut down any further potential distractions.

(41) The above embodiments are described by way of example only. Many variations are possible without departing from the scope of the invention as defined in the appended claims.