Physio-sensory transduction method and device
11596345 · 2023-03-07
Assignee
Inventors
Cpc classification
A61B5/7282
HUMAN NECESSITIES
A61B5/7278
HUMAN NECESSITIES
International classification
A61B5/00
HUMAN NECESSITIES
Abstract
A method and device allowing a physiological signal, typically representative of brain activity, to be transcribed in the form of sensory signals perceptible to a human user, typically acoustic signals is provided. For this purpose, a physiological signal is acquired and then analysed in such a way as to detect therein patterns that are then parameterised in the time domain. One or more parameters of these patterns are used to determine one or more parameters of the generated sensory signals and/or to determine one or more parameters of temporal envelopes used to modulate the sensory signals. This method and device can be applied, in particular, to neuro-acoustic transduction.
Claims
1. A physio-sensory transduction method comprising: an acquisition of a physiological signal of an organism using acquisition means; and a detection of one or more patterns in the physiological signal using detection means; for each pattern detected: an extraction of at least one pattern parameter from the pattern detected using extraction means; and a generation of a sensory signal associated with this pattern comprising: a determination of at least one parameter of the sensory signal as a function of the at least one pattern parameter using determination means; and a modulation of the sensory signal as a function of a temporal envelope associated with this pattern using modulation means, the temporal envelope having at least one envelope parameter determined as a function of the at least one pattern parameter.
2. The method according to claim 1, in which the detection of the one or more patterns comprises: a detection of an amplitude of the physiological signal becoming greater than or equal to a first predetermined amplitude at a first time; a detection of the amplitude of the physiological signal becoming less than a second predetermined amplitude at a second time ts2; and a validation test according to which a pattern is detected if the elapsed duration between said first and second times is greater than or equal to a predetermined duration.
3. The method according to claim 1 in which, for each pattern detected, the at least one pattern parameter results from a detection, a calculation or a measurement of: a pattern start time; and/or a pattern end time; and/or a maximum amplitude of the physiological signal in this pattern; and/or a maximum amplitude time; and/or a total duration corresponding to an elapsed duration between said pattern start time and said pattern end time; and/or a duration of ascent corresponding to an elapsed duration between said pattern start time and said maximum amplitude time; and/or a duration of descent corresponding to an elapsed duration between said maximum amplitude time, and said pattern end time; and/or a first and/or second derivative of the physiological signal between said pattern start time and said maximum amplitude time; and/or a first and/or second derivative of the physiological signal between said maximum amplitude time and said pattern end time.
4. The method according to claim 1, in which for each pattern detected, the at least one envelope parameter comprising: an attack amplitude corresponding to a maximum amplitude of the temporal envelope associated with this pattern; and/or an attack duration corresponding to an elapsed duration between a start time and a maximum amplitude time of the temporal envelope associated with this pattern; and/or a release duration corresponding to an elapsed duration starting from a release start time up to an end time of the temporal envelope associated with this pattern; and/or a decay duration corresponding to an elapsed duration starting from the maximum amplitude time up to a decay end time; and/or a sustain amplitude; and/or a sustain duration corresponding to an elapsed duration between the decay end time and a release start time.
5. The method according to claim 4, in which the at least one envelope parameter comprises the attack amplitude, the attack duration and an extinction duration, and in which: the attack amplitude depends on the maximum amplitude of the physiological signal in this pattern; and/or the attack duration depends on a duration of ascent; and/or the extinction duration depends on a duration of descent.
6. The method according to claim 4, in which for each pattern detected: the at least one pattern parameter comprises a maximum amplitude of the physiological signal in this pattern; the at least one envelope parameter comprises: the attack amplitude being a function of said maximum amplitude of the physiological signal in this pattern; and/or the attack duration being a function of said maximum amplitude of the physiological signal in this pattern.
7. The method according to claim 1, in which for each pattern detected, the at least one parameter of the sensory signal comprising: an oscillation frequency; and/or a cutoff frequency; and/or an amplitude.
8. The method according to claim 1, in which for each pattern detected: the at least one pattern parameter comprises a maximum amplitude of the physiological signal in this pattern, and the sensory signal: is filtered with a filter having a cutoff frequency being a function of said maximum amplitude of the physiological signal in this pattern; and/or oscillates with a frequency being a function of said maximum amplitude of the physiological signal in this pattern; and/or has an amplitude being a function of said maximum amplitude of the physiological signal in this pattern.
9. The method according to claim 1, in which for each pattern detected, the at least one parameter of the sensory signal varies as a function of a variation of the at least one pattern parameter, the at least one pattern parameter being a maximum amplitude of the physiological signal in this pattern.
10. The method according to claim 1, in which several patterns are detected in the physiological signal, each pattern comprising as a one of the at least one pattern parameter pattern start time, each sensory signal generated for each pattern detected comprising an initiation time of this sensory signal, the elapsed duration between the initiation time of each pair of consecutive sensory signals generated contiguously being proportional or equal to the elapsed duration between the pattern start time of each pair of consecutive patterns detected contiguously.
11. The method according to claim 10, in which each pattern also comprises as one of the at least one pattern parameter a pattern end time, each sensory signal generated for each pattern detected also comprising an expiration time of this sensory signal, the amplitude of each sensory signal generated for each pattern detected being constant between the initiation time and the expiration time of this sensory signal.
12. The method according to claim 1, in which the generation of said sensory signal is carried out in a deferred manner with respect to the acquisition of the physiological signal.
13. The method according to claim 1, in which the detection of the one or more patterns, the extraction of said at least one pattern parameter, and the generation of said sensory signal are carried out in real time with respect to the acquisition of the physiological signal.
14. The method according to claim 1, in which the sensory signal generated is a periodic function.
15. The method according to claim 1, in which no step is implemented in order to transform the physiological signal from a time domain to a frequency domain.
16. The method according to claim 1, in which: the organism is a human being; the physiological signal is representative of a brain activity of said human being; and the sensory signal is an acoustic signal.
17. A physio-sensory transduction device comprising: an acquisition means arranged and/or programmed in order to acquire a physiological signal from an organism; and a detection means arranged and/or programmed in order to detect one or more patterns in the physiological signal; an extraction means arranged and/or programmed in order to extract at least one pattern parameter from each pattern detected; and a generation system arranged and/or programmed in order to generate a sensory signal for each pattern detected, this generation system comprising: a determination means arranged and/or programmed in order to determine at least one parameter of the sensory signal as a function of the at least one pattern parameter; and a modulation means arranged and/or programmed in order to modulate the sensory signal as a function of a temporal envelope associated with this pattern, having at least one envelope parameter determined as a function of the at least one pattern parameter.
Description
BRIEF DESCRIPTION OF THE DRAWINGS
(1) Other advantages and features of the invention will become apparent on reading the detailed description of implementations and embodiments that are in no way limitative, and from the following attached drawings:
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
DETAILED DESCRIPTION
(14) In the different figures, the same references denote identical or similar characteristics or steps.
(15) As the embodiments described hereinafter are in no way limitative, variants of the invention can be considered comprising only a selection of the characteristics described hereinafter, in isolation from the other characteristics described (even if this selection is isolated within a phrase containing other characteristics), if this selection of characteristics is sufficient to confer a technical advantage or to differentiate the invention with respect to the state of the art. This selection comprises at least one, preferably functional, characteristic without structural details, or with only a part of the structural details if this part alone is sufficient to confer a technical advantage or to differentiate the invention with respect to the state of the prior art.
(16) The present invention typically aims to transcribe a physiological signal in the form of a sensory signal perceptible to a human user.
Physiological Signal
(17) The physiological signal is preferably representative of a human brain activity.
(18) The physiological signal is a temporal signal, i.e. a signal which is a function of a time.
(19) The physiological signal has temporal dynamics, i.e. it varies over time.
(20) A first example of a physiological signal 10 is represented in
(21) A second example of a physiological signal 11 is represented in
(22) These different cognitive states of deep sleep or of wakefulness, are characterized by the presence of waves which oscillate at specific frequencies. Typically, in a state of deep sleep, the brain can generate waves, called delta waves, having a frequency comprised between 0.3 and 4 Hz. In a state of wakefulness, the brain can generate waves, called beta waves, typically having a frequency greater than 14 Hz.
(23) The present invention is not limited to the acquisition or to the processing of the delta or beta waves and can be based on an acquisition of a physiological signal representative of a physiological activity, for example cerebral, cardiac, respiratory, ocular or else muscular—of any organism—for example an animal or a human being. This organism, in particular when the latter is an animal or a human being, can be placed in any cognitive state—for example in a state of deep or paradoxical sleep or of wakefulness or else of relaxation.
(24) With reference to
(25) Such an acquisition step E1 is carried out with the aid of a physio-sensory transduction device comprising an acquisition means arranged and/or programmed in order to acquire a physiological signal from an organism (not represented).
Pattern Detection
(26) The method according to the invention comprises detection of one or more patterns in the physiological signal.
(27) In the example of
(28) In the example of
(29) Alternatively or additionally, it is possible to detect one or more patterns in the physiological signal after partial acquisition of this signal.
(30)
(31)
(32) In order to carry out the detection step E20 or E21, the physio-sensory transduction device comprises a detection means arranged and/or programmed in order to detect one or more patterns in the physiological signal (not represented).
(33) In an embodiment, the detection E20 or E21 of a pattern 100 comprises the sub-steps SE1, SE2 and SE3 described below with reference to
(34) The sub-step SE1 comprises detection of the amplitude of the physiological signal 10 becoming greater than or equal to a first predetermined amplitude as1 at a first time ts1. For example, the first amplitude as1 can have a value of 7 μV.
(35)
(36) The sub-step 5E2 comprises detection of the amplitude of the physiological signal 10 becoming less than a second predetermined amplitude as2 at a second time ts2. For example, the second amplitude as2 can have a value of 0.89 μV.
(37) The value of the thresholds as1 and as2 can be proportional to a standard deviation of the physiological signal 10.
(38) The sub-step SE3 comprises a validation test according to which a pattern 100 is detected if the elapsed duration between said first ts1 and second times ts2 is greater than or equal to a predetermined duration. For example, this predetermined duration can have a value of 100 ms. Such a test makes it possible in particular to avoid considering as a pattern any simple variation in the amplitude of the signal which may be linked to the high-frequency noise.
Parameterization
(39) For each pattern 100 detected, the method comprises extraction of at least one pattern parameter from this pattern 100 detected.
(40) In the example of
(41) In the example of
(42) The extraction step is carried out by an extraction means comprised in the device of the invention, this extraction means being arranged and/or programmed in order to extract at least one pattern parameter from each pattern detected.
(43) With reference to
(44) In the present document, a first derivative is defined as a ratio of an amplitude variation to a time variation. For example, the first derivative of the physiological signal 10 between said pattern start time t1 and said maximum amplitude time t2 corresponds to the relationship between: on the one hand, the difference in amplitude al of the physiological signal 10 at the time t1 and the maximum amplitude a2, and on the other hand, the difference between t1 and t2.
(45) Similarly, the first derivative of the physiological signal 10 between said maximum amplitude time t2 and said pattern end time t3 corresponds to the relationship between: on the one hand, the difference between the maximum amplitude a2 and the amplitude a3 of the physiological signal 10 at the time t3, and on the other hand, the difference between t2 and t3.
(46) In the present document, a second derivative is defined as a variation of said first derivative with respect to said corresponding time variation.
(47) By way of indicative example, the maximum amplitude a2 of the physiological signal 10 in the pattern 100 can be equal to 27 μV, the duration of ascent can be equal to 268 ms and the duration of descent can be equal to 348 ms.
(48) In an embodiment (not shown), the detection means comprises a calculator or a measurement tool arranged and/or programmed in order to carry out, for each pattern detected, said calculation or said measurement and in order to assign the result of this calculation or of this measurement to said at least one pattern parameter.
(49) Thus the method of the invention comprises, during the extraction step, a parameterization of the physiological signal 10 in the time domain.
(50) Preferably, no step is implemented in order to transform the physiological signal 10 from a time domain to a frequency domain. In particular, the method according to the invention does not carry out any processing step in order to obtain a frequency representation of the physiological signal in order to extract the pattern parameter(s) therefrom. According to the invention, at least one and preferably all of the pattern parameters are extracts from the physiological signal in the time domain.
(51) By means of this parameterization carried out in the time domain, the sensory signal can be generated so as to transcribe the physiological signal relatively faithfully from a perceptual point of view, in particular from the point of view of time perception.
Sensory Signal
(52) According to the invention, a sensory signal is generated.
(53) This sensory signal is preferably an acoustic signal or a wave but can alternatively or additionally be a signal or a wave of any other kind (visual, tactile, olfactory etc.) provided that it is perceptible to said organism.
(54) The sensory signal is preferably a periodic function. For example, the sensory signal can consist of a sine wave or of a sum of sine waves.
(55) For each pattern 100 detected, the method of the invention comprises generation of a sensory signal associated with this pattern 100 by means of a generation system arranged and/or programmed in order to generate such a signal.
(56) In the example of
(57) In the example of
(58) Irrespective of the embodiment, for example that illustrated in
(59) In the example of
(60) In the box E401 of the diagram of
(61) The at least one sensory signal parameter is, for example, an oscillation frequency, and/or a cutoff frequency, and/or an amplitude.
(62) In order to carry out such a sub-step of determination of at least one parameter of the sensory signal, the generation system comprises a determination means arranged and/or programmed in order to determine at least one parameter of the sensory signal as a function of the at least one pattern parameter (not represented).
(63) Another example is shown in
(64) Such a temporal envelope consists of a specific signal constructed in order to modulate a sensory signal.
(65) An example of a temporal envelope 200 is represented in
(66) According to the invention, for each pattern 100, a temporal envelope 200 can be associated with this pattern 100, and have at least one envelope parameter determined as a function of the at least one pattern parameter.
(67) Thus, unlike the case of
(68) In other words, in the example of
(69) In order to carry out such a sub-step of modulation of the sensory signal, the generation system comprises a modulation means arranged and/or programmed in order to modulate the sensory signal as a function of a temporal envelope associated with the pattern that is the subject of the generation of this sensory signal, this temporal envelope having at least one envelope parameter determined as a function of the at least one pattern parameter.
(70) The sensory signal generation according to the diagram of
(71) In the box E403 of the diagram of
(72) By way of non-limitative example, the generation system can comprise an amplifier, one or more oscillators, for example: a monophonic oscillator capable of generating a wave of the sine, triangle-, square-wave type, etc., an oscillator of the “LFO” type (low-frequency oscillator), a module making it possible to carry out additive or subtractive or frequency-modulation synthesis. a wavetable reading module etc.
(73) The device of the invention can implement analogue and/or digital technologies.
Temporal Envelope
(74) A description will now be given of an example of a temporal envelope with reference to
(75) In the case of such an ADSR envelope associated with a given pattern 100, the at least one envelope parameter can comprise: an attack amplitude a21 corresponding to a maximum amplitude of this temporal envelope 200, and for example proportional to the maximum amplitude of the associated pattern, an attack duration corresponding to an elapsed duration between a start time t20 of this temporal envelope 200 and a maximum amplitude time t21, and for example proportional or equal to the duration of ascent of the associated pattern, or proportional to the maximum amplitude of the associated pattern, a release duration corresponding to an elapsed duration from a sustain duration end time t23 up to an end time t24 of the temporal envelope 200, and for example proportional to the maximum amplitude or to the duration of descent of the associated pattern, a decay duration corresponding to an elapsed duration starting from the maximum amplitude t21 up to a sustain start time t22, and for example proportional to the maximum amplitude or to the duration of descent of the associated pattern, a sustain amplitude a22, and for example proportional to the maximum amplitude of the associated pattern, a sustain duration corresponding to an elapsed duration between the decay end time t22 (which thus corresponds in this case to the sustain start time) and a release start time t23 (which thus corresponds in this case to the sustain duration end time), and for example proportional to the maximum amplitude or to the duration of descent of the associated pattern, a total duration corresponding to an elapsed duration between the decay start time t20 and the decay end time t22, and for example proportional or equal to the total duration of the associated pattern, or proportional to the maximum amplitude of the associated pattern, By default: the value of the amplitude a22 can be two-thirds of the amplitude a21, and/or the decay duration and the release duration can each be equal to half the sustain duration.
(76) The value of these envelope parameters can be defined by a value defined by default in the device according to the invention.
(77) Other types of envelope, not represented, can be used, for example envelopes of the following types: AR: comprising only attack and release phases; ASR: comprising only attack, sustain and release phases; ADR: comprising only attack, decay and release phases.
(78) In the case of these other types of envelope, the different envelope parameters available are deduced from the list of parameters given below by way of example for an ADSR envelope.
Multiplicity of Sensory Signals
(79) The preceding description essentially explains the generation of a sensory signal associated with a pattern detected in a physiological signal.
(80) Of course, a physiological signal generally comprises several patterns which can each give rise to generation of its own sensory signal.
(81) According to the variant of
(82) The steps of detection E20 of the pattern(s), of extraction E30 of said at least one pattern parameter, and of generation E40 of said sensory signal can thus be carried out in real time with respect to the acquisition E1 of the physiological signal 10. In order to do this, the generation system can be a real-time generator arranged and/or programmed in order to detect the pattern(s), to extract said at least one pattern parameter, and generate said sensory signal in real time with respect to the acquisition of the physiological signal.
(83) According to the variant of
(84) The generation of a sensory signal can thus be carried out in a deferred manner with respect to the acquisition of the physiological signal. In order to do this, the generation system can comprise a retarder arranged and/or programmed in order to generate the sensory signal in a deferred manner with respect to the acquisition of the physiological signal.
(85) Preferably, several patterns 100 are detected in the physiological signal 10. In an embodiment, it is considered that each pattern 100 comprises as pattern parameter said pattern start time t1, and that each sensory signal generated for each pattern detected comprises an initiation time of this sensory signal. Preferably, according to this embodiment, the elapsed duration between the initiation time of each pair of sensory signals generated contiguously is proportional or equal to the elapsed duration between the pattern start time t1 of each pair of patterns detected contiguously.
Specific Examples
(86) In an embodiment, the temporal envelope 200 comprises the following three envelope parameters: attack amplitude a21, the attack duration and the extinction duration. In this embodiment, for each envelope 200 associated with a given pattern 100: the attack amplitude a21 depends on—for example is proportional to the maximum amplitude a2 of the physiological signal 10 in this pattern 100, and/or the attack duration depends on—for example is proportional to—the duration of ascent and/or the extinction duration depends on—for example is proportional to—the duration of descent.
(87) In the embodiment MDR11 of
(88)
(89) The embodiment MDR11 thus illustrates a simple case in which each sensory signal has an identical amplitude irrespective of the shape of the pattern associated with this sensory signal—it is assumed hypothetically in this example that the patterns associated with the sensory signals 31, 32, 33 and 34 have a different shape, and have for example a different maximum amplitude.
(90)
(91) For this embodiment MDR11 it is also assumed that, for illustration purposes, the elapsed duration between the initiation time of each pair of sensory signals generated contiguously is equal to the elapsed duration between the pattern start time of each pair of associated patterns detected contiguously in the physiological signal 10. In other words, the successive generation of the sensory signals 31, 32, 33 and 34 respects the temporal dynamics of the part of the physiological signal comprising the four patterns associated with these sensory signals.
(92) The embodiment MDR12 of
(93) In other embodiments, the respective amplitude of the sensory signals 35, 36, 37 and 38 could be inversely proportional to the maximum amplitude of the physiological signal in the corresponding pattern, or be a function of any other kind, for example a function of this maximum amplitude and/or of one or more other pattern parameters.
(94)
(95) In this embodiment MDR13, for each pattern 100 detected in the physiological signal, the generation of the sensory signal associated with this pattern 100 comprises said modulation of this sensory signal as a function of said temporal envelope 200 associated with this pattern 100. As a consequence, it can be seen that the modulated sensory signals 39, 40, 41 and 42 each have a profile which evolves over time, in this instance have an amplitude which evolves over time due to the modulation of these signals by a temporal envelope, in this example an envelope of the AR type (see above for a description of different types of envelope). In this example, the at least one pattern parameter comprises a maximum amplitude a2 of the physiological signal 10 in this pattern 100, and the at least one envelope parameter comprises an attack duration a21 being proportional to said maximum amplitude a2 of the physiological signal 10 in the pattern associated with the corresponding envelope.
(96) The sensory signals 39, 40, 41 and 42 are respectively generated at an initiation time t39, t42, t45 and t48 and end at an expiration time t41, t44, t47 and t50 respectively. These signals have a maximum amplitude time t40, t43, t46 and t49 respectively. The maximum amplitude reached by each of the sensory signals 39, 40, 41 and 42 is identical in absolute value from one signal to another but occurs after a duration of ascent that differs from one signal to another. The duration of ascent is defined by the elapsed duration between the initiation time (for example t39 for the signal 39) and the maximum amplitude time (for example t40 for the signal 39).
(97) In another embodiment, not represented, the envelope parameter in the embodiment MDR13 is replaced by an envelope parameter comprising an attack amplitude a21 proportional to said maximum amplitude a2 of the physiological signal 10 in the pattern associated with the corresponding envelope.
(98) The embodiment MDR14 of
(99) More specifically, the sensory signals 43, 44, 45 and 46 of
(100) The example of
(101) In the embodiment MDR15, for each pattern 100 detected, the at least one pattern parameter comprises a maximum amplitude a2 of the physiological signal 10 in this pattern 100, and the sensory signal is filtered with a filter having a cutoff frequency being a function of said maximum amplitude a2 of the physiological signal 10 in this pattern 100.
(102) In other embodiments, not represented, which can be combined in particular with the embodiment MDR15, for each pattern 100 detected, the at least one pattern parameter comprises a maximum amplitude a2 of the physiological signal 10 in this pattern 100, and the sensory signal can: oscillate with a frequency being a function of said maximum amplitude a2 of the physiological signal 10 in this pattern 100 and/or have an amplitude being a function of said maximum amplitude a2 of the physiological signal 10 in this pattern 100.
(103) In another embodiment, not represented, for each pattern 100 detected, the at least one sensory signal parameter varies as a function of a variation of the at least one pattern parameter, the at least one pattern parameter preferably being a maximum amplitude a2 of the physiological signal 10 in this pattern 100. The at least one sensory signal parameter can comprise an oscillation frequency and/or a cutoff frequency and/or an amplitude.
(104) Of course, the invention is not limited to the examples which have just been described and numerous adjustments may be made to these examples without exceeding the scope of the invention
(105) For example, the method can comprise a step of detection of all of the patterns contained in a physiological signal followed by a classification step making it possible to select detected patterns representative of a particular cognitive state. Such a classification step can be useful, in particular in a situation in which the chosen pattern detection criteria are capable of detecting patterns representative of different cognitive states between which it is desired to make a distinction. This could be the case after acquisition of a physiological signal representing a brain activity over a prolonged duration, such as one night.
(106) Furthermore, the different characteristics, forms, variants and embodiments of the invention can be combined together in various combinations, to the extent that they are not incompatible or mutually exclusive.
(107) The physiological signal can be representative of a brain activity of the organism (alpha, beta, delta waves etc.) or of a cardiac or respiratory or ocular or muscular activity of the organism of an animal or human being. The physiological signal is not acquired from this animal or human being in an invasive manner, for example not by means of an intracerebral probe.
(108) As explained above, the sensory signal can be an acoustic or visual or tactile or olfactory or gustatory signal. The sensory signal is preferably received (for example heard) by the same animal or human being from which or from whom the physiological signal has originated. The term “personalized” sensory signal is then used.
(109) This sensory signal can be replayed in a personalized manner to the same individual who produced the sensory signal, or more generally to another user.
(110) Of course, the different characteristics, forms, variants and embodiments of the invention can be combined together in various combinations.