CONTROL DEVICE, SYSTEM, AND METHOD FOR DETERMINING A COMFORT LEVEL OF A DRIVER

Abstract

A control device, system, and method for a vehicle to determine a driver's comfort level, configured to receive a first sensor output of a first physiological sensor measuring at least one first physiological feature of the driver, and receive a second sensor output of a second physiological sensor measuring at least one second physiological feature of the driver, create at least one reference data set by recording the first sensor output over a first predetermined reference time period and recording the second sensor output over a second predetermined reference time period being different than the first predetermined reference time period, determine at least one reference index for the comfort level of the driver based on the reference data set, and determine a comfort level index value of the driver, the index value determined as a function of the reference index and the current first and/or second sensor output.

Claims

1. A control device for a vehicle for determining a comfort level of a driver, the control device being configured to: receive a first sensor output of a first physiological sensor measuring at least one first physiological feature of the driver, and receive a second sensor output of a second physiological sensor measuring at least one second physiological feature of the driver, create at least one reference data set by recording the first sensor output over a first predetermined reference time period and recording the second sensor output over a second predetermined reference time period being different than the first predetermined reference time period, determine at least one reference index for the comfort level of the driver based on the at least one reference data set, and determine a comfort level index value of the driver, the index value being determined as a function of the at least one reference index and the current first and/or second sensor output.

2. The control device according to claim 1 configured to: receive driving context information from a driving context detection unit, determine at least one reference index for the comfort level of the driver based on the at least one reference data set and, based on the current driving context information, determine the comfort level index value as a function of a first reference index and the current first sensor output or a second reference index and the current second sensor output, the first reference index being based on the first sensor output recorded over the first predetermined reference time period and the second reference index being based on the second sensor output recorded over the second predetermined reference time.

3. The control device according to claim 1 configured to: define a first fixed-length sliding window and a second fixed-length sliding window during which the first and second sensor outputs are recorded.

4. The control device according to claim 3 configured to: define a first over-lap percentage (SWO1) and a second over-lap percentage (SWO2) that is different from the first over-lap percentage (SWO1).

5. The control device according to claim 1 configured to: sample the first and second sensor outputs with the first and second fixed-length sliding windows, respectively, in order to create the at least one reference data set.

6. The control device according to claim 1 configured to: determine the comfort level index value based on the product of the set of selected features within the first and second sliding windows, in particular based on the product of a first physiological feature and a second physiological feature, as: $\begin{array}{cc}{\mathrm{CI}}_{\left(\mathrm{SWL},\mathrm{SWO}\right)}=X\left[\begin{array}{c}{x}_1\\ \mathrm{.Math.}\\ x_n\end{array}\right].Math.Y\left[\begin{array}{c}{y}_1\\ \mathrm{.Math.}\\ y_n\end{array}\right],& \left(1\right)\end{array}$ where x.sub.n the latest computed value of the first physiological feature X within the first sliding window and y.sub.n the latest computed value of the second physiological feature Y within the second sliding window.

7. The control device according to claim 1 configured to: determine a threshold for the comfort level index value based on the at least one reference data index, detect a discomfort state of the driver when the comfort level index value exceeds the threshold, and in particular, trigger a driver assistance system based on the detected discomfort state.

8. The control device according to claim 1 configured to determine: a mean value (M) and a standard deviation (SD) of the at least one reference index, in particular, over the first and/or second predetermined reference time period, and the threshold for the comfort level index value by the equation:
DT=M+SD.

9. The control device according to claim 1, wherein the first sensor output includes a physiological measurement output, the physiological measurement output comprising a skin conductance level signal of the driver, the second sensor output comprising an electrocardiographic signal of the driver.

10. The control device according to claim 1 configured to: low-pass filter the skin conductance level signal, in particular at 1 Hz or less, more particularly by using a zero time-lag second-order Butterworth filter, and/or normalize the skin conductance level signal by dividing it by a predetermined reference skin conductance level.

11. The control device according to claim 3 configured to: determine the amplitude of the normalized SCL signal based on the difference between the maximal and minimal values of the normalized skin conductance level signal within the first fixed-length sliding window.

12. The control device according to claim 3 configured to: determine the root mean square value of the normalized SCL signal based on the following Equation (3): $\begin{array}{cc}\mathrm{RMS}=\sqrt{\frac{1}{n}.Math.\left(x_1^2+x_2^2+\mathrm{.Math.}+x_n^2\right)},& \left(3\right)\end{array}$ where n is the number of normalized SCL values over the total duration of the first fixed-length sliding window and x represents the corresponding normalized SCL values.

13. The control device according to claim 3 wherein the first sensor output includes a physiological measurement output, the physiological measurement output comprising a skin conductance level signal of the driver, the second sensor output comprising an electrocardiographic signal of the driver, the control device being configured to determine: a heart rate of the driver based on a mean heart rate value of the ECG signal within the second fixed-length sliding window, and in particular a heart rate variability based on the beat-to-beat alterations in the heart rate.

14. The control device according to claim 2, wherein the driving context detection unit is configured to detect driving conditions including vehicle speed, vehicle acceleration, vehicle deceleration, lateral position, steering wheel reversal rate, and/or steering wheel angle.

15. The control device according to claim 14 configured to determine a first discomfort event if the duration of the activity is 30s or less, and determine a second discomfort event if the duration of the activity is at least 10 minutes.

16. The control device according to claim 15, wherein when the control device determines that a first discomfort event has occurred, the comfort level index value is determined as a function of the first sensor output and the first reference data set.

17. The control device according to claim 15, wherein when the control device determines that a second discomfort event has occurred, the comfort level index value is determined as a function of the second sensor output and the second reference data set.

18. The control device according to claim 1, wherein the at least one reference data set recording is online.

19. The control device according to claim 15, wherein during a calibration of the threshold value, the control device acquires at least 10 mins of driving that is not determined to be a first discomfort event or a second discomfort event.

20. A system for a vehicle for determining a comfort level of a driver, the system comprising: a control device according to claim 1, a first physiological sensor for measuring at least one first physiological feature of the driver, and a second physiological sensor for measuring at least one second physiological feature of the driver.

21. The system according to claim 10, the first physiological sensor comprising: a conductance level sensor, in particular with unpolarizable A.sub.g or A.sub.gC.sub.l electrodes, more particularly with a surface of at least 30 mm.sup.2, the second physiological sensor comprising an electrocardiographic sensor, in particular with gold-plated active electrodes.

22. A vehicle comprising: a control device according to claim 1.

23. A method of determining a comfort level of a driver, the method comprising the steps of: receiving sensor output of a first physiological sensor and a second physiological sensor, the first physiological sensor measuring at least one physiological feature of the driver and the second physiological sensor measuring at least one physiological feature of the driver, creating at least one reference data set by recording the first sensor output over a first predetermined reference time period and recoding the second sensor output over a second predetermined reference time period, determining at least one reference index for the comfort level of the driver based on the at least one reference data set, and determining a comfort level index value of the driver, the index value being determined as a function of the current first and/or second sensor output, and the at least one reference index.

24. The method according to claim 23, wherein a reference skin conductance level is determined before driving based on recording before starting driving the skin conductance level signal of the driver at the driver position for a predetermined time period, in particular for at least 5 min.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0062] FIG. 1 shows a block diagram of a control device according to embodiments of the present disclosure;

[0063] FIG. 2 shows a schematic diagram of the data flow and data processing in the control device according to the embodiments of the present disclosure;

[0064] FIG. 3 shows a graph of the results of physiological features in terms of sensitivity and as a function of the driving event according to the embodiments of the present disclosure;

[0065] FIG. 4 shows a graph of the SCL as a function of time according to the embodiments of the present disclosure; and

[0066] FIG. 5 shows a graph of the HRV obtained before and after driving situations according to the embodiments of the present disclosure.

DESCRIPTION OF THE EMBODIMENTS

[0067] Reference will now be made in detail to exemplary embodiments of the disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

[0068] FIG. 1 shows a block diagram of a control device 1 according to embodiments of the present disclosure. The control device is part of a system 20. The system 20 is comprised by a vehicle 10.

[0069] The control device 1 may be connected to or comprise a data storage for storing a reference data set. The control device 1 may comprise an electronic circuit, a processor (shared, dedicated, or group), a combinational logic circuit, a memory that executes one or more software programs, and/or other suitable components that provide the described functionality. The control device 1 may additionally carry out further functions in the vehicle 1. For example, the control device may also act as the general purpose ECU (electronic control unit) of the vehicle.

[0070] The control device 1 receives data output of two or several physiological sensors 2, 3. Each of the physiological sensors 2, 3 measure at least one physiological feature of the driver. For example, the physiological sensor 2 measures the skin conductance level (SCL) and sensor 3 measures the electrocardiographic signal (ECG). Both may be recorded at 2000 Hz sampling rate for example. The SCL may be recorded with 30 mm.sup.2 unpolarizable Ag/AgCl electrodes (e.g., from Clark Electromedical Instruments). It may be desirable to place the SCL sensors at least at the level of the second phalanx of the index and the third digit of the non-dominant hand of the driver. For ECG recording, gold-plated active electrodes may be used.

[0071] The control device 1 may low-pass filter the skin conductance level (SCL) signal, in particular at 1 Hz or less, for example by using a zero time-lag second-order Butterworth filter. The skin conductance level (SCL) signal may be normalized by dividing it by a predetermined reference skin conductance level (SCL). The control device 1 may also determine the amplitude (AS) of the normalized SCL signal based on the difference between the maximal and minimal values of the normalized skin conductance level (SCL) signal within a fixed-length sliding window or a time window having a fixed length and moves over the real time monitored electrophysical data. The SCL may be first recorded at rest during 5 min before driving, while the driver sits at the driving wheel without any stimulation and then averaged to be considered the reference. The control device 1 may also determine the root mean square value (RMS) of the normalized SCL signal based on the following Equation (3):

[00003]$\begin{array}{cc}\mathrm{RMS}=\sqrt{\frac{1}{n}.Math.\left(x_1^2+x_2^2+\mathrm{.Math.}+x_n^2\right)},& \left(3\right)\end{array}$

where n is the number of normalized SCL values over the total duration of the fixed-length sliding window and x represents the corresponding normalized SCL values. The heart rate (HR) of the driver may be determined based on the mean heart rate value of the ECG signal within a different and distinct fixed-length sliding window. In particular, the heart rate variability (HRV) may be determined based on the beat-to-beat alterations in the heart rate.

[0072] Furthermore, the control device receives driving context information from a driving context detection unit 4. The driving context information describes the current driving situation. Such driving situations may include, e.g., a situation in which the vehicle comprising the control device 1 overtakes another vehicle, a situation in which the vehicle comprising the control device 1 is approaching a traffic light, a situation in which the vehicle comprising the control device 1 approaches a pedestrian or a pedestrian approaches the vehicle, a situation in which the vehicle comprising the control device 1 is driven in the dark, a situation in which the vehicle comprising the control device 1 makes a left-turn (in right-hand traffic), or other driving situations which may challenge the comfort level of the driver.

[0073] In order to determine the driving context information, the driving context detection unit 4 receives information from a plurality of measurement units which monitor the environment of the vehicle, e.g. a GPS 5, a radar 6, one or several cameras 7, etc. The driving context detection unit 4 may be an electronic control device, e.g., a processor. It may also be provided by the general purpose ECU (electronic control unit) of the vehicle. It is further possible that the control device 1 comprises the driving context detection unit 4.

[0074] The control device 1 determines a driving context event based on the driving context information. In the examples mentioned above, such driving context events may be overtaking another vehicle, approaching a traffic light, approaching a pedestrian, driving in the dark, or making a left-turn (in right-hand traffic) among others.

[0075] As described in the following, the control device 1 determines a comfort level index value of the driver, in particular it detects a discomfort state of the driver, when the comfort level index value exceeds a predetermined threshold DT. In such a case, it may trigger an advanced driver assistance system (ADAS) 8. Generally the ADAS may assist driving based on the received comfort level index value. For example the ADAS may increasingly assist driving, i.e., increasingly overtake driving control tasks, in case the comfort level index value decreases, i.e., the driver's discomfort increases.

[0076] FIG. 2 shows a schematic diagram of the data flow and data processing in the control device according to embodiments of the present disclosure. The control device 1 determines a comfort level index value of the driver. Said value represents the current (dis)comfort state.

[0077] The comfort level index value is calculated based on the current sensor outputs, for example, a first and a second sensor, and may be based on a set of features including the amplitude (AS) of the normalized SCL, the root mean square value (RMS) of the normalized SCL signal, the heart rate (HR) of the driver, and/or the heart rate variability (HRV).

[0078] The electrophysiological features (heart rate variability and amplitude of normalized skin conductance level (SCL)) required for the index computation are computed within separate fixed-length sliding-windows moving over the real time monitored electrophysiological data. For each feature, some parameters have to be specified: [0079] The sliding-window fixed-length (SWL). [0080] The sliding windows over-lap percentages (SWO).
The window length determines the number of data points per signal to be considered for a single window. The overlap factor determines the time offset between the first data points of two successive windows. While window length influences how much historical information is contained in a single window, the overlap factor influences how much historical information is shared among successive windows.

[0081] Sliding windows of 10 seconds with over-lap percentages of 99% moving over the SCL signal are used for the computation of the amplitude of normalized SCL. First the SCL may be low-pass filtered at 1 Hz using a zero time-lag second-order Butterworth filter for example to remove high frequency noise. Then, the filtered SCL signal may be divided by the reference to obtain the normalized SCL signal. The SCL may be first recorded at rest during 5 mins before driving, while the driver sits at the driving wheel without any stimulation and then averaged to be considered the reference. AS may correspond to the difference between the maximal and minimal values of the normalized SCL signal within the sliding window.

[0082] Sliding windows of 600 seconds (10 minutes) with over-lap percentages of 50% moving over the ECG signal may be used for the computation of the heart rate variability (HRV). HRV is computed as the standard deviation of the RR intervals over the total duration of the sliding window. This feature refers to the beat-to-beat alterations in heart rate that is widely used as an important marker of emotion regulatory ability.

[0083] Skin conductance level (SCL) and electrocardiographic signal (ECG), may both be recorded at 2000 Hz for example. SCL may be recorded with 30 mm2 unpolarizable Ag/AgCl electrodes (e.g., using devices from Clark Electromedical Instruments). The SCL sensors may be located at least at the level of the second phalanx of the index and the third digit of the non-dominant hand. For ECG recording, gold-plated active electrodes may be used.

[0084] At each time (each iteration), the index of (dis)comfort can be directly obtained from the product (or multiplication) of the two electrophysiological features. The latest value of each feature may be considered at each time.

[0085] The selection of the features used for the index value computation desirably depends on the type of driving situation (e.g., short term discomfort events where AS is used and/or long term discomfort events where HRV is used).

[0086] With continued reference to FIG. 2, the control device 1 further creates at least one reference data set by recording the multiple sensor outputs and the driving context information over a predetermined reference time period, in particular by recording the formerly determined comfort level index value of the driver. However, as mentioned above, the predetermined reference time period is different and distinct for each sensor and, further, may be dependent on the type of sensor used. For example, if a SCL sensor is used, an appropriate predetermined reference time period may be a sliding window of 10 seconds.

[0087] Based on the at least one reference data set, the control device 1 determines a reference index for the comfort level of the driver. The reference index represents the basic (dis)comfort state.

[0088] The predetermined reference time period may also comprise a plurality of driving sessions, e.g., the operating time of the vehicle during several weeks. Accordingly, a long-term monitoring of the driver can be achieved that is suitable to anticipate any negative long-term impact of the experienced (dis)comfort. Hence, the control device may take into account at least the accumulated operating time of the vehicle within one month as a reference time period. In other words, the comfort level index value of the driver can be correlated against historical data (e.g., collected in the last 1 month) represented by the reference index.

[0089] The reference index may be used to calculate a threshold for the comfort level index value. It is also possible that the threshold is calculated based on a predetermined selection of the set of features. The mean value (M) and the standard deviation (SD) of the index time series can be computed over a predetermined reference time period. Then, discomfort threshold (DT) will be computed following Equation (2):

DT=M+SD  (2),

[0090] Accordingly, the index value is determined as a function of the current sensor outputs and the reference index, i.e., by correlating the current measurements against historical measurements. In this way it is possible to calibrate the currently determined (dis)comfort level. In particular the uncorrelated index value can be compared to the discomfort threshold (DT).

[0091] In addition, the index value is determined also as a function of the current driving context information. In this way it is possible to compare the currently determined (dis)comfort level only with those historical data which have been recorded in a corresponding driving event. Moreover, it is possible to determine the length of the current driving event such that the determination of the comfort level index value can be based on sensor outputs of the time before, during and after the event.

[0092] Turning to FIG. 3, a group of forty test drivers were subjected to a driving simulator experiment which included three driving scenarios: a safety-driving scenario used to compute a threshold beyond which discomfort in driving is considered, and two discomfort driving sessions. In the discomfort driving sessions, the test drivers were systematically exposed to different driving situations producing discomfort (truck overtaking, left-turn, sudden event).

[0093] The driving situations were repeated twice by varying the level of discomfort intensity (high level and high level plus one). For each level of intensity of each driving situation, 40 measures of the behavioural and physiological parameters investigated were obtained. In addition, after the driver's exposure to the driving situations, the feeling of subjective comfort/discomfort was monitored.

[0094] The results from the experiment are shown in FIG. 3. The results suggested that it could be more desirable to use the amplitude of the SCL (AS) signal for the computation of the (dis)comfort index for short terms and unexpected driving events such as the sudden insertion/braking of a car or the sudden crossing of a pedestrian. For longer situations, such as overtakings or left-turns under high density traffic conditions, it could be more pertinent to use the heart rate variability (HRV) physiological feature. Indeed, despite the fact that both the heart rate variability and the amplitude of the SCL signal provide us useful information about the level of stress and/or mental workload, the accuracy and sensitivity to the feeling of discomfort of these features differ as a function of the time period considered. It may also be desirable to combine heart rate variability and the amplitude of the SCL.

[0095] A complementary analysis was performed while using the data recorded during the high level plus one overtaking situations. For this high level plus one situation, the overtaking was repeated twice in order to produce a cumulative effect of discomfort. The results statistically confirmed that a residual effect of discomfort is observed for the skin conductance level. Indeed, the skin conductance level did not return to its baseline level during the very short time period (around 15-20 seconds) between the two consecutive trucks. However, such a residual effect of discomfort was not observed for heart rate variability (HRV).

[0096] FIG. 4 shows the results from a second complementary analysis that was performed over a longer time period. This time, the baseline physiological measures (skin conductance level and HRV) obtained during the very beginning and the very ending of the driving scenarios were compared. The mean skin conductance level was found to significantly decrease over the total duration of the scenario (around 10 min). Thus, a global long-term decrease in skin conductance level can be observed, whereas discomfort can be assessed through short-term changes. This last observation confirms the interest of using the amplitude of skin conductance within short temporal windows, especially for sudden event (duration shorter than 30 seconds).

[0097] As shown in FIG. 5, the results also demonstrated that HRV significantly increased during the total duration of the scenario (around 10 min). This result brought the statistical evidence of the interest to use HRV when the objective is to detect the discomfort within a long period of time or to assess the cumulative effect of a series of uncomfortable situations.

[0098] Throughout the description, including the claims, the term “comprising a” should be understood as being synonymous with “comprising at least one” unless otherwise stated. In addition, any range set forth in the description, including the claims should be understood as including its end value(s) unless otherwise stated. Specific values for described elements should be understood to be within accepted manufacturing or industry tolerances known to one of skill in the art, and any use of the terms “substantially” and/or “approximately” and/or “generally” should be understood to mean falling within such accepted tolerances.

[0099] Although the present disclosure herein has been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present disclosure.

[0100] It is intended that the specification and examples be considered as exemplary only, with a true scope of the disclosure being indicated by the following claims.