Sensor system and method which makes use of multiple PPG sensors

Abstract

A sensor system comprises first and second PPG sensors. A monitoring system monitors detection by at least one of the first and second detectors an optical calibration signals, for performing time calibration between the first and second PPG sensors. This system makes use of two PPG sensors. To enable these sensors to be independent units, rather than being fully integrated into a combined system, a calibration system is provided. Based on detected optical signals, the behavior over time of each PPG sensor can be monitored and thus calibration can take place.

Claims

1. A sensor system, comprising: a first PPG sensor having a first light source a first optical detector and a first processor with a first clock domain; a second PPG sensor having a second light source a second optical detector and a second processor with a second clock domain; a docking station, which provides a first optical coupling between the first light source and the first optical detector and a second optical coupling between the second light source and the second optical detector; a monitoring system adapted to monitor detection of first and second optical calibration signals, wherein the first optical calibration signal is adapted to be detected by the first optical detector of the first PPG sensor and is generated by the first optical coupling and the second optical calibration signal is adapted to be detected by the second optical detector of the second PPG sensor and is generated by the second optical coupling, wherein the monitoring system is adapted to perform time calibration between the first and second clock domains of the first and second PPG sensors by determining a time offset between the detected signals.

2. A system as claimed in claim 1, wherein the first and second optical couplings have an optical reflectance or transmittance function with known absolute timing relationship.

3. A system as claimed in claim 2, wherein the first and second optical couplings each comprise a reflector with a reflection characteristic which is varied over time.

4. A system as claimed in claim 3, wherein the reflectors comprise a display device such as an LCD display.

5. A system as claimed in claim 4, wherein the reflectors together form a common display device.

6. A system as claimed in claim 2, wherein the first and second optical couplings each comprise structure with a transmission characteristic which is adapted to be controlled over time.

7. A pulse delay measurement device, comprising: a sensor system as claimed in any preceding claim; and a controller for obtaining PPG signals from the first and second PPG sensors, and deriving a pulse delay using the time calibration.

8. A sensor calibration method for calibrating first and second PPG sensors, the first PPG sensor having a first light source a first optical detector and a first processor with a first clock domain and the second PPG sensor having a second light source a second optical detector and a second processor with a second clock domain, the method comprising: placing the first and second PPG sensors on a docking station, which provides a first optical coupling between the first light source and the first optical detector and a second optical coupling between the second light source and the second optical detector, generating a first optical calibration signal using the optical coupling and generating a second optical calibration signal using the second optical coupling by controlling the first and second optical couplings with known absolute timing relationship; and performing time calibration between the first and second clock domains of the first and second PPG sensors by determining a time offset between detected signals at the first and second optical detectors.

9. A pulse delay sensing method comprising: performing sensing calibration using the method as claimed in claim 8; and after the sensing calibration, obtaining PPG signals from the first and second PPG sensors, and deriving a pulse delay.

10. A computer program product comprising computer program code means which is adapted, when run on a computer which forms part of the system of claim 1.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) Embodiments of the invention will be described, by way of example only, with reference to the drawings, in which:

(2) FIG. 1 shows a known PPG sensor;

(3) FIG. 2 shows a first example of a sensor arrangement using multiple PPG sensors;

(4) FIG. 3 is a set of timing waveforms for explaining how the timing calibration may take place;

(5) FIG. 4 shows a second example of a sensor arrangement using multiple PPG sensors;

(6) FIG. 5 shows a third example of a sensor arrangement using multiple PPG sensors;

(7) FIG. 6 shows a fourth example of a sensor arrangement using multiple PPG sensors;

(8) FIG. 7 shows a fifth example of a sensor arrangement using multiple PPG sensors;

(9) FIG. 8 shows timing diagrams for a synchronization signal received at two PPG sensors at the same time, but with different clock domains;

(10) FIG. 9 shows a cross correlation of the signals of FIG. 8;

(11) FIG. 10 shows a zoomed in part of the cross correlation signal of FIG. 9; and

(12) FIG. 11 shows a zoomed in part of the cross correlation signal of FIG. 9 after up-sampling.

DETAILED DESCRIPTION OF THE EMBODIMENTS

(13) The invention provides a sensor system, comprising first and second PPG sensors. A monitoring system monitors detection by at least one of the first and second detectors of an optical calibration signal, for performing time calibration between the first and second PPG sensors.

(14) This system makes use of two PPG sensors. To enable these sensors to be independent units, rather than being fully integrated into a combined system, a calibration system is provided. Based on detected optical signals, the behavior over time of each PPG sensor can be monitored and thus calibration can take place.

(15) FIG. 1 shows a known PPG sensor.

(16) It comprises a light source 10 and an optical detector 12. The light source may comprise a single multi-wavelength light source, or multiple light units with different output wavelengths. The sensor has a processor 14 and it generates an output 16 in the form of a PPG signal which varies over time. The processor includes an internal clock and a memory. The output may be provided in real time and/or it may be logged and stored as a trace over time during a monitoring period. The stored trace may then be analyzed after the monitoring period is over.

(17) The PPG sensor for example comprises a pulse oximeter. The output 16 may be interpreted to provide a pulse rate.

(18) The invention is based on the combination of two PPG sensors which may be of the general type shown in FIG. 1.

(19) FIG. 2 shows a first example of system to explain the principles of the synchronization approach. FIG. 2 relates to a cross coupled architecture.

(20) A first PPG sensor 20 has a first light source 22 and a first optical detector 24.

(21) A second PPG sensor 30 has a second light source 32 and a second optical detector 34.

(22) Each PPG sensor has its own processor 26, 36 generating a PPG output 28, 38.

(23) In this example, a calibration system comprises a controller 40 which is adapted to operate the first and second light sources 22, 32 at a controlled time or times. The controller 40 communicates with the processors 26, 36 as well as receiving the sensor signals.

(24) The detection by the first and second detectors is monitored, and the optical path between each light source and its corresponding detector is known. Timing calibration is based on the monitored detection.

(25) The version of FIG. 2 has the two PPG sensors 20, 30 mounted against each other, so that the output of the light source 22 of the first PPG sensor is fed directly to the detector 34 of the second PPG sensor, and the output of the light source 32 of the second PPG sensor is fed directly to the detector 24 of the first PPG sensor. The known optical path between each sensor-detector pair is thus a direct optical coupling.

(26) The two optical PPG sensors may be independent units, rather than being fully integrated into a combined system. There is no timing information shared between them. Their outputs 28, 38 comprise waveforms which vary along a time axis, but the time axes of the two sensors are not synchronized.

(27) The behavior over time of each PPG sensor can be monitored and thus calibration can take place.

(28) In the example of FIG. 2, no additional hardware to the two PPG sensors themselves is required. The calibration controller 40 may be implemented in software.

(29) With the devices coupled in this way a synchronization process can be triggered either by the user, or by another means.

(30) During synchronization, one (or both) of the PPG sensors is controlled to produce a known test signal, which can later be analyzed for the time correction. This signal may take the form of a call and response in that the first device 20 produces a known light pattern, which is detected by the second device 30. When the second device 30 receives this pattern, it in turn produces a known pattern with a known time delay. When the data captured from the two devices is analyzed in conjunction with each other, the synchronization patterns will be evident in the signal, and the time offset can be calculated.

(31) In this way, an optical synchronization pattern can be used to synchronize the data recordings from the two devices. This synchronization may thus be carried out after the recordings have been made, rather than in real time, or else it may be carried out in real time.

(32) By performing a time calibration at both the beginning and the end of a data recording event, it is possible not only to correct for any offset between the two logging device clocks, but also to calculate and correct for any drift between the two clocks. Thus, synchronization can occur either within a given time period prior to a measurement taking place, after a measurement has taken place, or both.

(33) Instead of having one controller 40 that can operate both the first light source 22 and the second light source 32, the controller 40 could also be connected to or integrated in only the first PPG sensor 20, while the other PPG sensor 30 could have its own controller or no controller at all. For example, by pressing a button on PPG sensor 20, the controller could operate the first light source 22 such that it would flash its lights in a certain pattern in time. In the signal of PPG sensor 30, which might be read out instantly or afterwards when analyzing the data, this light pattern can be recovered.

(34) FIG. 3 shows in simplified schematic form the approach explained above, and shows time traces for the first and second light sources LS1 and LS2 and the detected signals for the first and second detectors D1 and D2.

(35) FIG. 3(a) shows the use of an initial pulse to determine the delay between the time clocks.

(36) A first pattern is generated by the first light source LS1. This results in a detected output from the second detector D2, and this appears in the recorded trace provided as the output of the second PPG sensor. After a known delay d1 the second PPG sensor sends a second pattern (which may in fact be the same as the first) which is received by the first detector D1. This appears in the trace recorded of the first PPG sensor.

(37) The time delay between one light source emitting a pattern and a detector detecting it may either be negligible or else it may be taken into account when analyzing the traces.

(38) The absolute value of the time shift between the signal D2 and D1 is thus known. In this way, if the local clocks of the two PPG sensors represent these times with an incorrect relative time shift, the timing can be calibrated. Time point t1 is on the time axis of the second PPG sensor whereas time point t2 is on the time axis of the first PPG sensor.

(39) One or other of the traces D1 and D2 can be time shifted to make the time difference between the time points t1 and t2 equal to the delay d1 or else equal to the delay d1 plus the latency introduced between LS2 and D1 if it is relevant.

(40) The example of FIG. 3(a) shows how the offset between two clocks can be determined based on a single synchronization action. When drift between the two clocks also needs to be determined, two (or more) synchronization actions may be performed.

(41) This is shown schematically in FIG. 3(b), in which the same signals are shown. This time, there is a first synchronization operation at the beginning of the device operation, and a second synchronization at the end of the device operation. The first synchronization is as described above, and the second is the same, with device actuation at time t3 and reception of the returned signal at time t4 (of the two different clock domains). The same delay d1 gives rise to a different time difference between the two clock domains (i.e. t2t1 is different to t4t3 even though d1 is the same). This is the result of the relative drift between the two clocks. The time delay d1 looks different at the start and end because it is a time difference as measured between two clock domains.

(42) The example of FIG. 2 above makes use of a direct coupling between the two PPG sensors, by placing them back to back. FIG. 4 shows a system which works in exactly the same way, but uses a docking station 50, which provides a first optical coupling 52 between the first light source and the second optical detector and a second optical coupling 54 between the second light source and the first optical detector. The optical path is again a direct optical connection for example using optical fibers. However, a mirror arrangement may instead be used.

(43) FIG. 5 shows an example in which two transmissive PPG sensors 20, 30 such as ear clips or finger clips are used. The clips have a respective LED 22, 32 and a photodetector 24, 34 as in the examples above, but they are for mounting on opposite sides of tissue being examined. The system again comprises a passive docking station 50 on which the finger and/or ear clips 20, 30 are placed. A controller operates the light source 22 of the first sensor 20 to send a light signal. The light enters the docking station, where it is reflected by a mirror 55, after which it reaches the photodetector 34 of the second sensor 30. Similarly, the LED 32 of the second sensor 30 is optically coupled to the photodetector 24 of the first sensor 20.

(44) FIG. 6 shows an example for reflective PPG sensors that do not have a flat surface and/or for which the light source(s) of the one sensor cannot easily be laid on top of the light detector(s) of the other as in FIG. 2. In FIG. 6, each sensor 20, 30 is in the form of a wrist watch, with the LEDs 22, 32 and photodetectors 32, 34 at the base of the watch face. In this example, each sensor has one photodetector 24, 34 in the middle and two LEDs 22, 32 on the sides.

(45) The sensors 20, 30l have a curved bottom plate to fit against the skin of the wrist of the user. The docking station 50 has a shape that fits the curvatures of sensors 20 and 30. It has a translucent inner part 56 and an opaque outer part 57. The opaque outer part 57 prevents ambient light entering the optical coupling between the two sensors which takes place via translucent inner part 56. The way the light signals are generated and processed is similar to the examples described above.

(46) The examples of FIGS. 2 and 4 to 6 require the PPG sensors to send pulses at one time and to receive pulses at a later time (when the light source is turned off). This may not be possible for some most basic designs of PPG sensor. For example the sensor may only process samples received while the light source is turned on. When the light source is turned off, the device is expecting only to see ambient light. In some designs, this ambient light level is measured so that it can be filtered out from the sensor measurements when the light source is turned on. In this way, ambient light compensation is carried out by which the optical detector signal received when the light source is turned off is subtracted from the optical detector signal received when the light source is turned on. Thus, the examples above are suitable for certain types of PPG sensor only, in which there is for example continuous monitoring of the received light signal.

(47) Optionally, as soon as the optical calibration signal is recognized, the ambient light compensation may be turned off temporarily. This ambient light compensation involves subtracting the light measured when the light source is turned off from the light measured when the light source is turned on. By turning off the ambient light compensation, the optical calibration signal becomes directly visible in the PPG signal.

(48) Thus, a calibration signal can be detected by a sensor while its light source is turned on if there is no ambient light compensation. For example, if a calibration signal is detected during a period with the light source turned off, the ambient light compensation may then be turned off so that the calibration signal can continue to be detected while the light source is turned on. Alternatively, a suitably chosen calibration signal may be used, for example one which has different calibration signal levels during the light on and off periods. This can be achieved using a random noise signal or a sinusoidal calibration signal.

(49) If the calibration signal is instead analyzed only during periods when the light source is turned off, then the issue of possible cancellation of the calibration signal by ambient light compensation is avoided.

(50) The examples above make use of two calibration signals, one sent from the first PPG sensor to the second, and the other sent from the second PPG sensor to the first. However, in a more basic implementation, only one sensor (e.g. the second PPG sensor) provides a calibration signal to the other (e.g. the first PPG sensor). The second PPG sensor may then mark in its own data (e.g. in a memory or in the streamed data) the time at which it sent out the calibration signal, or else it may detect its own optical signal at the same time.

(51) In this way, a marker can be set in the data of the second PPG sensor to mark the time at which the calibration signal is sent out (at the start of the calibration signal, or at a recognizable point in the calibration signal, or at any point during or before or after the calibration signal but with a known timing with respect to the calibration signal).

(52) If there is no optical coupling between the second PPG sensor and its own signal, the marker may be used instead of a second optical calibration signal. As mentioned above, another option (instead of marking its own data or receiving a second calibration signal from the first sensor) is that the second PPG sensor can detect its own optical calibration signal as well for example in the manner explained with reference to FIG. 5. An acknowledgement signal may also be sent by the first PPG sensor to acknowledge that it has received the calibration signal. This may be an optical signal or could be sent e.g. via Bluetooth.

(53) An acknowledgement signal acknowledging that the calibration signal has been received may be provided. Such an acknowledgement could however also be done by wireless communication such as using Bluetooth.

(54) The monitoring system may also be for monitoring detection of a second optical calibration signal, wherein the second optical calibration signal is for detection by the second PPG sensor and is generated by the first light source.

(55) The second optical calibration signal for example also functions as an acknowledgement signal.

(56) The examples above are based on a cross coupling approach. This invention relates to a self-coupling approach.

(57) FIG. 7 shows an example in which the PPG sensors are again placed onto a docking station 50, but rather than connecting the optical pathways as in the examples above the devices are placed such that they are each facing a controllable reflector. The first PPG sensor 20 faces a first reflector 58 and the second PPG sensor 30 faces a second reflector 59. They may be regions of a single device. The reflectors (or single reflector) have a controllable reflectivity over time. An advantage of this approach is that it enables the most basic sensor PPG to be used without adaptation. The sensor measurements can be made when the light source is turned on.

(58) The reflector may be implemented using a passive-matrix LCD display for example. By varying the reflectivity of the display over time, the amount of light reflected from the light source to the light detector will also vary. By doing this to both PPG sensors at the same time, the time offset between the recorded signals can be detected. In particular, the received signal provides a measure of the absolute time period since a common event (which is the absolute time at which the reflectivity starts being controlled).

(59) Control of the reflection results in generation of the calibration signals.

(60) The nature of the reflection function can be determined from the detected signal, so that absolute timing information can then be allocated to the detected signals.

(61) In this example, a first optical coupling is between the first light source and the first optical detector and a second optical coupling is between the second light source and the second optical detector. Thus, instead of cross coupling the two PPG sensors together, the detector of each PPG sensor is used to detect the light output from the light source of that same PPG sensor.

(62) By enabling timing calibration in the manner explained above, pulse delay measurement is possible. The optional controller 40 as shown in FIG. 2 may provide a pulse delay output 42.

(63) Another approach is to provide a light source which provides a synchronization pulse sequence to both PPG sensors. The detection of this known pulse sequence in the recorded traces again enables the relative timing to be determined. In particular, the start and/or end of the known pulse sequence can be identified on the clock domain of each PPG sensor.

(64) This concept can be applied using the same structure shown in FIG. 7. The units 58 and 59 are then implemented as light sources delivering synchronized pulse sequences, or else as a single shared light source. The pair of PPG sensors may be docked for synchronization measurement before and after capturing a set of data, so that the time offset and offset drift may be calculated.

(65) The waveforms of FIG. 3 are simplified for the purposes of explaining the underlying principles. In practice, the low sampling frequency of a typical PPG sensor will not allow a direct (human) measurement of the time delay to the required accuracy. Instead, signal processing is need to derive the time delay to the desired level of accuracy, which is typically of the order of ones of milliseconds.

(66) FIG. 8 shows two traces of signal amplitude versus time. Each has approximately 5 seconds of normal signal at the beginning, followed by a 30 second period of a 2.4 Hz sync pulse, and approximately 15 seconds of normal signal at the end. The signals are measured using a sampling frequency of 256 Hz.

(67) For the purposes of explanation, the bottom plot has been selected to lag the first plot by 474.6 ms (486 samples at 1024 Hz sampling frequency). This approximate 0.5 s lag can be seen when looking at the raw plots, but of course the lag cannot be manually determined more accurately.

(68) FIG. 9 shows a cross correlation of the two signals, which plots the cross correlation versus the selected delay between them. It is centered about zero delay.

(69) FIG. 10 shows the same cross correlation function, but shows the peak of the function in enlarged form, with the cross correlation value versus the delay expressed as a number of samples (i.e. the sample difference used to calculate the function). It indicates that the peak is at either 121 or 122 samples delay (472.7 or 476.6 ms at Fs=256 Hz).

(70) Thus, it can be seen that the cross correlation function enables the delay to be determined to millisecond accuracy.

(71) FIG. 11 shows the same cross correlation peak as FIG. 10, but after first up-sampling the two input signals to 1024 Hz sample frequency. The peak is now at exactly 486 samples delay, or 474.6 ms (at Fs=1024 Hz).

(72) As well as up-sampling the input signals it is possible to first take the cross-correlation of the two low sample rate signals, and then up-sample the resultant. The output will be the same.

(73) Other signal processing approaches may be employed. Thus, any known approach for processing a pair of signals with near identical content but time-shifted can be used which enables the time shift to be extracted.

(74) This cross correlation approach may be used in all examples above, including the reflection version, in which a known amplitude function is present in the two signals but with a time shift. It will now be understood that the pulse sequences shown schematically in FIG. 3 may in practice comprise a signal of known frequency for a particular duration, but they may also comprise a particular pulse sequence or amplitude function.

(75) A further feature which may be used is to synchronize the device with a device measuring an electrical signal such as ECG. The electrical signal modulating the optical couplings in the example of FIG. 6 may for example be output directly to positive and negative leads of an ECG device (or scaled to form a suitable output within the signal range of the ECG device such as a maximum 8 mV peak-peak signal). In this way, a calibration signal will also show up in the ECG signal, at the same time as in the optical signals. This can be very useful if not only pulse delay is to be measured, but also pulse arrival time (PAT).

(76) The example of FIG. 6 makes use of a controlled reflectance. However, other modulation approaches may be used, such as a controlled transmittance. This may be of interest for a transmissive PPG sensor, for example based on two finger clips which may be clipped on opposite sides of a docking station (in this case a rod) which could change its transmission properties in order to implement the calibration signal. The changed optical coupling may be implemented using liquid crystals, so again using display technologies.

(77) The calibration (i.e. the extraction of the time lag) may be implemented by a computer program which forms or is part of the controller 40. The controller can be implemented in numerous ways, with software and/or hardware, to perform the various functions required. A processor is one example of a controller which employs one or more microprocessors that may be programmed using software (e.g., microcode) to perform the required functions. A controller may however be implemented with or without employing a processor, and also may be implemented as a combination of dedicated hardware to perform some functions and a processor (e.g., one or more programmed microprocessors and associated circuitry) to perform other functions.

(78) Examples of controller components that may be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs).

(79) In various implementations, a processor or controller may be associated with one or more storage media such as volatile and non-volatile computer memory such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed on one or more processors and/or controllers, perform at the required functions. Various storage media may be fixed within a processor or controller or may be transportable, such that the one or more programs stored thereon can be loaded into a processor or controller.

(80) The examples above are based on synchronizing two PPG sensors. However, the same approach may be used for synchronizing the signals from more than two sensors. For example a set of three or more sensors may be daisy-chained using a docking station, or else there may be three or more of the controllable reflectors.

(81) Furthermore, the examples above are based on monitoring synchronization signals at both PPG sensors. There may be only one optical synchronization signal that needs to be monitored. For example, a button may be pressed on the first PPG sensor to send out a light pattern to be detected by the second optical detector (of the second PPG sensor) for calibration purposes. It is not then necessary to make the second PPG sensor also send out a light pattern to be detected by the first optical sensor. A time stamp can be inserted into the data captured by the first optical detector at the moment the button is pressed. This time stamp then serves as the time calibration of the first PPG sensor, to be matched with the detected calibration light pattern as measured by the photodetector of the second PPG sensor. This approach can also take into account a known delay between pressing the button and sending out the light pattern.

(82) Essentially, what is needed is an identifiable signal in each detector trace in which the relative absolute time difference between those two identifiable signals is known. The various examples given above provide examples of how to implement this.

(83) A calibration at the start of a measuring cycle may synchronize the clocks, but by performing a second calibration at the end of a measurement period (say 8 hours overnight) any relative drift can also be compensated.

(84) As explained above, the invention enables the two PPG traces to be synchronized with respect to each other. This involves placing time markers in both traces. By matching these time markers from the two devices, the time base of the two devices can then be aligned with respect to offset and drift between the two independent clocks. This involves for example shifting the time axis of one trace and/or stretching the time axis.

(85) For offset and drift correction between the clocks of the two devices, at least two of those marker pairs are present and a relative shift and scaling of the time axes takes place. With a single marker pair, correction for offset is possible which involves only a shift of the time axes relative to each other.

(86) The examples above enable PD to be determined. There are other possible measurements based on multiple PPG measurements, such as arterial stiffness. The invention is not limited to these two examples, and is applicable whenever multiple PPG signals need to be synchronized.

(87) The PPG sensors may for example make use of a camera flash as the optical source, and a camera image sensor as the sensor device. In this way, with a suitable application running on a mobile phone, a PPG sensor and PD sensor can be implemented.

(88) While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.

(89) Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word comprising does not exclude other elements or steps, and the indefinite article a or an does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measured cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.