BREATHING ADAPTATION SYSTEM AND METHOD FOR INFLUENCING A BREATHING PARAMETER

Abstract

It is an object of the invention to improve a user's breathing for a specific application (e.g. diagnostic imaging, falling asleep improvement (shortening sleep onset), delivery, tinnitus control therapy, CORD, blood pressure control). This object is achieved by a breathing adaptation system configured for influencing a breathing parameter of a user's breathing pattern in order to meet a goal of decreasing or increasing the influenced breathing parameter to a certain extent in a user specific manner. The breathing adaptation system comprises a sensor, configured for monitoring a current value of the influenced breathing parameter of the user. The breathing adaptation system further comprises a feedback unit configured for providing feedback to the user about a preferred value of the influenced breathing parameter, wherein the preferred value is different from the current value of the influenced breathing parameter (310). The breathing adaptation system further comprises a control unit, comprising program code means for determining the preferred value of the influenced breathing parameter in a user specific manner by means of an intelligent agent, wherein the control unit is implemented such that the intelligent agent is rewarded (330) by means of a reward function after performing the determination of the preferred value (320) of the influenced breathing parameter, wherein the reward function is such that it balances the reward for obtaining the goal and the ability of the user to breathe according to a breathing pattern having the preferred value for the influenced breathing parameter.

Claims

1. A breathing adaptation system configured for influencing a breathing parameter of a user's breathing pattern based on provision of feedback to the user, in order to meet a goal comprising decreasing or increasing the influenced breathing parameter, wherein the breathing adaptation system comprising: a sensor configured to monitor a current value of the breathing parameter of the user; a feedback unit configured to provide sensory feedback to the user about a determined preferred value of the breathing parameter for achieving the goal, wherein the preferred value is different from the current value of the breathing parameter and; a control unit configured to determine the preferred value of the influenced breathing parameter by an artificial intelligence intelligent agent, and the control unit further configured for implementing a reward function configured to reward the intelligent agent after performing the determination of the preferred value of the breathing parameter, the rewarding being based on the monitored values of the breathing parameter, and wherein the reward is higher in response to the breathing parameter moving closer to the goal and lower in response to the parameter moving further from the goal, wherein the reward function is configured such that it balances reward for obtaining the goal and a detected ability of the user to breathe according to a breathing pattern having the preferred value for the breathing parameter.

2. A breathing adaptation system according to claim 1, configured for providing the feedback in the form of a multi-sensory feedback, wherein the frequencies of different sensory feedbacks are equal.

3. A breathing adaptation system according to claim 1, wherein the feedback unit is configured for providing a current value of the influenced breathing parameter to the user for a period of time before starting to provide the preferred value of the influenced breathing parameter.

4. A breathing adaptation system according to claim 1, wherein the feedback unit is configured for providing a current or preferred value of the influenced breathing parameter to the user by providing a breathing pattern, which is based on the current or preferred value of the influenced breathing parameter.

5. A breathing adaptation system according to claim 1, configured for repeating the determination of the preferred value of the influenced breathing parameter and providing this to the user until a predefined criterion is met.

6. A breathing adaptation system according to claim 1, wherein the feedback signal provided has a frequency below 1 Hz.

7. A breathing adaptation system according to claim 1, wherein the influenced breathing parameter is representative of breathing amplitude, breathing frequency and/or repeatability of breathing amplitude and/or breathing frequency, and/or breathe hold duration.

8. A breathing adaptation system according to claim 1, configured to be used for the purpose of improving diagnostic imaging or treatment delivery.

9. A breathing adaptation system according to claim 1, further comprising a diagnostic imaging system and/or a treatment delivery system.

10. A breathing adaptation system according to claim 9, wherein the breathing adaptation system comprises a diagnostic imaging system and the diagnostic imaging system is an MRI system.

11. A breathing adaptation system according to claim 5, a communication module configured to communicate with a remote computer of a diagnostic imaging or treatment center and configured in use for communicating a final preferred value of the influenced breathing parameter with the diagnostic imaging or treatment center.

12. A breathing adaptation system according to claim 10, further comprising a patient scheduling module configured for scheduling a timeslot for diagnostic imaging, wherein the length of the scheduled timeslot is dependent on the final preferred value of the influenced breathing parameter.

13. A breathing adaptation system according to claim 10, further comprising an image protocol optimizer, wherein the image protocol optimizer is configured for optimizing one imaging parameter of a diagnostic imaging system at least partly based on the final preferred value of the influenced breathing parameter, and configured to provide the optimized parameter as an input to a diagnostic imaging system.

14. A method for influencing a breathing parameter of a user's breathing pattern in order to meet a goal comprising decreasing or increasing the breathing parameter, wherein the method comprises the steps of monitoring a current value of the breathing parameter of the user and; providing sensory feedback to the user about a determined preferred value of the breathing parameter for achieving the goal, wherein the preferred value is different from the current value of the breathing parameter and; determining the preferred value of the breathing parameter by an artificial intelligence intelligent agent, wherein the intelligent agent is rewarded by a reward function after performing the determination of the preferred value of the breathing parameter, the rewarding based on the monitored values of the breathing parameter and wherein the reward is higher in response to the breathing parameter moving closer to the goal and lower in response to the parameter moving further from the goal, and wherein the reward function is such that it balances reward for obtaining the goal and a detected ability of the user to breathe according to a breathing pattern having the preferred value for the influenced breathing parameter.

15. A method according to claim 14, wherein the method is used as part of a diagnostic imaging or treatment delivery procedure.

16. A method according to claim 15, further comprising sharing a final preferred value of the preferred influenced breathing parameter with a diagnostic imaging or treatment center.

17. A method according to claim 14, further comprising scheduling a timeslot for diagnostic imaging, wherein the length of the scheduled timeslot is dependent on a final preferred value of the influenced breathing parameter.

18. A method according to claim 14, further comprising optimizing at least one imaging parameter at least partly based on a final preferred value of the breathing parameter.

19. A computer program product comprising program code configured to perform the method according to claim 14.

Description

BRIEF DESCRIPTION OF THE FIGURES

[0044] FIG. 1 diagrammatically shows an intelligent agent placed in a certain environment and

[0045] FIG. 2 diagrammatically shows a breathing adaptation system according to embodiments of the invention and

[0046] FIG. 3 displays the breathing frequency for one person over time, as well as the preferred value of the breathing frequency determined by the control unit and the corresponding reward figure and

[0047] FIG. 4 diagrammatically shows a breathing adaptation system for diagnostic imaging according to embodiments of the invention and

[0048] FIG. 5 diagrammatically shows a method for influencing a breathing parameter according to the invention.

DETAILED DESCRIPTION OF THE INVENTION

[0049] Reinforcement learning is a field in machine learning that is inspired by behaviorist psychology, and is concerned with how an intelligent agent can take actions in an environment so as to maximize its cumulative reward.

[0050] FIG. 1 diagrammatically shows an intelligent agent 111 placed in a certain environment 105. It can perceive the environment through sensors 110 to observe the current state of the environment. The agent can also act upon the environment using its actuators 112, and observes a certain reward 120, that is determined by the previous action and how good or bad the current state of the environment is. A good state yields a positive reward, whereas a bad state will yield a less positive or even negative reward. The reward is used to describe an implicit goal. Using the rewards, the intelligent agent 111 can determine what actions are preferred to achieve the goal. Reinforcement learning can be seen as the reasoning of the intelligent agent: it determines what action to take in a certain state. The intelligent agent does this by estimating the value of an action in this state, and by exploring states and actions for some iterations. This estimate becomes more accurate and an optimal policy can be defined: the action that maximizes the value of the next state will be picked.

[0051] To reach an optimal policy, the intelligent agent will often first go through a stage called exploration, where it will explore different actions regardless of the estimate of the value for each action. However, once this estimate has reached an acceptable accuracy, the agent should switch to a stage called exploitation. In this stage, it will only select the action with the highest value. In reality, there is no such clear boundary between the exploration and exploitation stages. It is more likely that the agent will start with a fully explorative policy and slowly move towards a more exploitative policy. However, preferably the agent should always keep some explorative actions in its policy.

[0052] Usually, the agent initiates the estimates of the values randomly. In order to improve on that, domain knowledge about the environment can be added to the reasoning to form an initial policy. This ensures that the agent will take less time to converge to an optimal policy.

[0053] There are several reasoning algorithms available in the prior art among one is Q-learning, which utilizes the Bellman equation to estimate the value of executing a certain action in a certain state (a state-action pair). This algorithm assigns a Q-value to each state-action pair by taking the reward obtained in the state into account, plus the expected Q-value of the best action in the next state, multiplied by a decay factor. This decay factor is a numeric value between 0 and 1. The expected Q-value of the next state is multiplied by the decay factor since the agent has to take one more action at a certain cost to reach this Q-value.

[0054] The best action in a certain state is computed by calculating the Q-values for all the possible actions, and choosing the action with the highest Q-value. The optimal policy corresponds to a policy where the expected value of the total reward return over all successive steps, starting from the current state, is the maximum achievable.

[0055] An alternative method to Q-learning is to search directly in policy space, instead of estimating a value for each possible action. Policy gradient is such a known method, based on optimizing parametrized policies by gradient descent.

[0056] In order to estimate an approximately correct value of a certain state, approximators can be used. These could for example be decision trees or random forest, both are known in the art.

[0057] When using an intelligent agent for influencing breathing parameters it is preferable to smooth the data that is sensed by the sensor in such a way that relevant patterns are captured while white noise is filtered out. Smoothing methods are known in the art among which are rectangular and triangular smoothing. It is further preferable to normalize the data sensed by the sensor.

[0058] Preferably, the data sensed by the sensor is modeled in order to extract the important breathing parameters. However, depending on the type of sensor and the goal, the one or more breathing parameters may also be measured directly. The breathing pattern could for example be modeled by sine or cosine function having parameters describing the amplitude, period and/or phase shift of the breathing pattern sensed by the sensor.

[0059] The current state of the breathing parameter(s) may be monitored by regular intervals. These intervals should not be too short, such that the user or patient has sufficient time to act upon the determined preferred value of the influenced breathing parameter. However, preferably this interval is neither too long, because this would make the method take too long.

[0060] The actions that the intelligent agent can take are preferably defined by the same parameters as a state is defined, which are for example amplitude, period and/or phase. Preferably, the actions that the intelligent agent is allowed to perform are dependent of the current state. In order to prevent too strong hyperventilation or too slow breathing, the value for the breathing frequency or interval may be constrained.

[0061] The reward function used is such that the reward balances the reward for obtaining the goal and the ability of the user to breathe according to a breathing pattern having the preferred value for the influenced breathing parameter. For example, the goal could be reducing the breathing parameter breathing frequency. In this example, the reward function should be designed such that it rewards lower breathing frequencies more than higher breathing frequencies. Also, alternatively or additionally the reward function in this example may be designed such that it rewards a decrease in breathing frequency compared to a previous state.

[0062] The ability of the user to breathe according to the breathing pattern having the preferred value for the influenced breathing parameter has to be taken into account in the reward function as well. This could for example be achieved by taking into account the difference between the preferred value of the influenced breathing parameter and the actually measured or sensed value for this influenced breathing parameter.

[0063] Alternatively, the ability of the user to breathe according to the breathing pattern could be determined in a broader sense. For example the breathing adaptation system could calculate an “ideal” breathing pattern using the measured values for the user's breathing frequency, amplitude and/or phase shifts. This could for example be achieved by using the above mentioned (co)sine function.

[0064] When calculating the ideal breathing pattern, preferably constraints could be provided to the non-influenced breathing parameters, such that they remain in a range that is assumed to be comfortable for the patient. When the measured non-influenced breathing parameters are within the allowed range, the breathing adaptation system will use the user's values when calculating the ideal breathing pattern. However, if the measured values are above or below the values set by the constraints, values taken from the allowed range will be used for calculating the ideal breathing pattern. Usually these values will be the upper or lower limit of the allowed range.

[0065] The control unit could then determine the difference between the calculated ideal breathing pattern and the actual breathing pattern of the patient/user. The difference could for example be expressed in a sum of squared differences, but alternatives are possible. In this way not only the breathing parameter that is being optimized (e.g. frequency) is taken into account, but also other parameters that may have an effect on the patient's comfort (e.g. breathing amplitude) when assessing the patient's comfort. Also, other irregularities in the measured breathing parameter could be detected in this way and be fed back to the intelligent agent by means of the reward function.

[0066] In order to obtain a value for the measured breathing parameter(s) it is advantageous to take into account a (weighted) average over multiple breathing cycles of the user or patient.

[0067] FIG. 2 diagrammatically shows a breathing adaptation system according to embodiments of the invention. A patient/user 210 is positioned on a bed 212. The bed could for example be the bed of a medical imaging system. The user is breathing according to a certain breathing pattern that is sensed by a sensor 214, which in this case is a camera. In this case this camera is displayed as a separate camera, which could for example be a camera inside a bore of a medical imaging system. However, in other applications it could also be for example the camera FIG. 4, 310 of a smartphone or tablet. The sensor does not need to be a camera. It could also be any other sensor suitable for determining breathing parameters, like for example a breathing bellow. The value for the breathing parameter may be directly measured by the sensor. Otherwise a mathematical function, e.g. a sine or cosine, may be fitted to the breathing pattern measured by the sensor. This measured breathing pattern is preferably smoothed and normalized. The data sensed by the sensor are sent to the control unit 250, which comprises the intelligent agent FIG. 1, 111. The control unit is connected to a feedback unit 216, configured for providing feedback to the user 210. In this embodiment the feedback is audiovisual. The visual feedback in this embodiment is provided by displaying a breathing pattern. This feedback reflects one or more of the breathing parameters, like for example the breathing period and/or the breathing amplitude 220. The line and circle 222 in this embodiment indicate a current position in the breathing cycle of the user. The part of the breathing pattern on the right side of this line and circle 222 display a breathing pattern according to the preferred value of the influenced breathing parameter. So, for example in case the influenced breathing parameter is breathing amplitude, the right hand side of the circle and line 222 displays a breathing pattern having a breathing amplitude according to the preferred value of the breathing period. According to embodiments of the invention, the breathing adaptation system could be configured such that at the beginning of the method not the preferred value of the influenced breathing parameter is displayed, but only a current value of the influenced breathing parameter. This would mean that the actual value of the breathing parameter of the user breathing cycle would be provided to the user. This would mean that the user would always breathe according to the breathing parameter displayed during this phase. This may give the user an increased feeling of being in control.

[0068] When the preferred value of the influenced breathing parameter is provided to the user, the sensor keeps on measuring the values of the influenced breathing parameter achieved by the user. Depending on how well the user is capable in adapting to the preferred value of the influenced breathing parameter, the control unit 250 will not change the value for the preferred value of the influenced breathing parameter at all or will increase or decrease it in a faster or slower fashion.

[0069] FIG. 3 displays the measured breathing frequency 310 for one person over time, as well as the preferred value of the breathing frequency 320 determined by the control unit FIG. 2, 250 and the corresponding reward FIG. 330. The measured breathing frequency starts with around 10 breaths per minute. The control unit has as a result determined that the preferred value of the influenced breathing parameter should be around 9.5 breaths per minute. The user is however not capable of adapting to this preferred breathing frequency and starts breathing with more than 12 breaths per minute. The control unit adapts accordingly and determines a preferred value for the breathing frequency of 12 breaths per minute. After this the user appears better capable to pick up the preferred breathing frequency and after one more slight increase of the measured breathing frequency, the measured breathing frequency goes more or less steadily downwards until a breathing frequency of about 3 breaths per minute is reached. This improvement of the influenced breathing parameter is also reflected in the reward 330, which increases over time.

[0070] FIG. 4 diagrammatically shows a breathing adaptation system configured to be used for the purpose of improving diagnostic imaging or treatment delivery. A patient scheduled for diagnostic imaging will be provided with a computer program product, e.g. an app. The computer program product can be used at home for example on a computer, tablet or phone 311. The patient may be provided with a sensor that is connectable to the computer program product and is configured to measure the patient's breathing parameters. Alternatively, a built-in camera 310 of the phone or tablet may be used as sensor to measure the breathing parameters. The device used by the patient 311 may display breathing parameters to the patient, e.g. in the form of a breathing pattern 312. Similarly, as described above based on measured breathing parameter values, the control unit will start to influence the patient's breathing pattern. This may be a good way for the patient to prepare for the upcoming diagnostic exam. Also this preparation may provide the patient and the caregiver with the confidence that the patient will be capable of finishing the imaging exam or therapeutic session without the need for sedation. After a certain breathing rate criterion is met, the control unit stops influencing the patient's breathing pattern and shares a final value of breathing parameters of interest to an image protocol optimizer 350 and/or a patient scheduling module 352 that may be located at the hospital. As described above, based on the shared final value of the breathing parameters the image protocol optimizer may be configured for optimizing the image protocol to the patient specific breathing parameters. This optimization may be done manually or (semi)-automatically. This image protocol may in turn be shared with the patient scheduling module which uses this information for scheduling a timeslot for imaging this patient. For example the patient scheduling module may determine an expected length of the timeslot for this specific patient, also it may determine whether specific (more qualified) personnel may be needed during scanning or preparation of this particular patient or whether access to sedation or anesthesia may be needed. The patient scheduling module does not need to be connected to the imaging protocol optimizer, but may also schedule based on the output of the computer program product alone, e.g. based on these values the patient scheduling module may be able to assess whether additional preparation time may be needed.

[0071] The diagnostic imaging system (e.g. MRI system) 360 will receive input from the image protocol optimizer. The diagnostic imaging system does also comprise the breathing adaptation system according to claim 1 and will hence be configured for influencing the patient's breathing pattern.

[0072] Of course, the breathing pattern at home may not be the same as the breathing pattern during actual diagnostic imaging, but the output from the computer program product may give an indication of the patient's breathing pattern during diagnostic imaging or treatment delivery.

[0073] FIG. 5 diagrammatically shows a method for influencing a breathing parameter according to the invention. The method comprises the steps of: [0074] monitoring a current value of the influenced breathing parameter of the user 501 and; [0075] providing feedback to the user about a preferred value of the influenced breathing parameter, wherein the preferred value is different from the current value of the influenced breathing parameter 502 and; [0076] determining the preferred value of the influenced breathing parameter in a user specific manner by means of an intelligent agent, wherein the intelligent agent is rewarded by means of a reward function after performing the determination of the preferred value of the influenced breathing parameter, wherein the reward function is such that it balances the reward for obtaining the goal and the ability of the user to breathe according to a breathing pattern having the preferred value for the influenced breathing parameter 502.

[0077] Whilst the invention has been illustrated and described in detail in the drawings and foregoing description, such illustrations and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.